A method for monitoring smoking posture alarm in public areas

Abstract

The present invention proposes a method for monitoring smoking posture alarm in public areas, and relates to the technical field of image recognition. The present invention proposes a process for monitoring smoking posture alarm in public areas, which includes obtaining smoking posture image data in public areas, annotating smoking posture in public areas and target annotating, constructing a multi-head QKV generation module, constructing an attention branch, constructing a feature restoration branch, constructing an improved linear attention module, constructing a public area smoking image scaling module, constructing a public area smoking posture detection model, constructing a public area smoking target detection model and public area smoking posture alarm monitoring; at the same time, an improved linear attention module is proposed to guide linear attention to focus on more important features, and at the same time, a lightweight public area smoking image scaling module is proposed, which is applied to the public area smoking posture detection model to reduce the inference cost, and the accuracy of public area smoking detection is improved by applying the public area smoking posture detection model and the public area smoking target detection model.

Description

Technical Field

[0001] The invention belongs to the technical field of image recognition, and in particular relates to a smoking posture alarm monitoring method in a public area. Background Art

[0002] Implementing smoking detection in public medical places such as hospitals and outpatient clinics has important public health and environmental significance. Smoking is not only harmful to the health of smokers themselves, but also poses a secondhand smoke hazard to non-smoking patients and medical staff around them. Through effective smoking detection, illegal smoking behaviors can be discovered and prevented in a timely manner, secondhand smoke exposure can be reduced, air quality can be improved, and public health can be maintained, especially in hospitals and outpatient clinics, which require a high degree of cleanliness and safety. This will not only help improve the quality of medical services, but also create a healthier and safer medical environment.

[0003] Visual detection technology has shown great convenience and accuracy in monitoring smoking behavior in hospitals and outpatient clinics. Relying on modern computer vision and deep learning algorithms, the visual detection system can automatically analyze images and videos captured by the camera to identify and locate smoking behavior in real time. Compared with traditional manual monitoring, the visual detection system not only improves monitoring efficiency, but also reduces missed detections and misjudgments caused by human factors, ensuring high accuracy and reliability of detection, and providing continuous and effective monitoring methods for hospitals and outpatient clinics.

[0004] Combining posture detection with object detection can significantly improve the effect of smoking detection in hospitals and outpatient clinics. Posture detection can accurately capture the position information of each joint of the human body, thereby identifying complex smoking postures; while object detection can quickly locate small target objects such as cigarettes. The combination of the two can effectively make up for the shortcomings of a single method and achieve higher detection accuracy and robustness. Even in crowded and severely occluded situations, it can efficiently identify smoking behavior, provide a more comprehensive monitoring solution, and ensure a smoke-free environment in hospitals and outpatient clinics. Summary of the invention

[0005] The present invention provides a method for monitoring smoking posture alarm in public areas, aiming to propose an improved linear attention module, guide linear attention to focus on more important features, improve the detection capability of the visual Transformer backbone network, and propose a lightweight public area smoking image scaling module, which is applied to a public area smoking posture detection model to reduce inference costs. By applying a public area smoking posture detection model and a public area smoking target detection model, the accuracy of public area smoking detection is improved.

[0006] The present invention aims to propose an improved linear attention module and provide a method for alarming and monitoring smoking postures in public areas, comprising the following steps:

[0007] S1. Obtaining smoking posture image data in public areas, collecting smoking posture images in public areas, including five smoking postures, namely, standing smoking, walking smoking, sitting smoking, squatting smoking, and leaning smoking;

[0008] S2. Smoking posture labeling and target labeling in public areas. In the posture labeling stage, the five smoking postures are labeled with posture key points. In the target labeling stage, the cigarette position is labeled with targets.

[0009] S3, build a multi-head QKV generation module, including linear projection operations;

[0010] S4, build the attention branch, including the multi-head QKV generation module and the guidance function;

[0011] S5, construct feature restoration branch, including multi-head QKV generation module and dynamic convolution;

[0012] S6. Build an improved linear attention module, including an attention branch and a feature restoration branch;

[0013] S7, building a public area smoking image scaling module, including a basic scaler;

[0014] S8. Build a public area smoking posture detection model, which consists of input, public area smoking image scaling module, visual Transformer backbone network, heat map regression detection head and output in sequence, and use the improved linear attention module to replace the self-attention module in the visual Transformer backbone network;

[0015] S9. Build a public area smoking target detection model, which consists of input, visual Transformer backbone network, target detection head and output in sequence, and use the improved linear attention module to replace the self-attention module in the visual Transformer backbone network;

[0016] S10, public area smoking posture alarm monitoring, real-time input of public area surveillance camera video, extracting frames per second and inputting them into the public area smoking posture detection model and the public area smoking target detection model, when the smoking posture and cigarette are detected at the same time, the smoking alarm information is output.

[0017] Preferably, in step S3, for the multi-head QKV generation module, the input feature X∈R N×C , where N = h × w, h and w are the height and width of the image that generates the input feature, respectively, C represents the channel of the input feature X, and the input feature is divided into H heads, each with a dimension of d k =C / H, define Q h , K h and V h is the feature of the h-th head, 1≤h≤H, in and It is to generate Q h , K h and V h The corresponding linear projection operation is implemented by the linear layer, and both

[0018] Preferably, in step S4, for the attention branch, the input feature X∈R N×C , where N = h × w, h and w are the height and width of the image that produces the input feature, C represents the channel of the input feature X, and the output feature F a First, input feature X to the multi-head QKV generation module to obtain the features of H heads, and the feature of the hth head is Q h , K h and V h , 1≤h≤H, for each head h, the attention output feature N = h × w, The query vector Q representing the i-th position h , represents the key vector K at the jth position h , is the value vector V at the jth position h , 1≤i≤N, Lead represents the leading function, which is calculated as in x **p represents the element-wise power p of x, ReLU is the activation function, ||x|| is the norm of x, T represents transposition, and finally the output features of all heads are connected to get F a , F a is the output feature of the attention branch.

[0019] Preferably, in step S5, for the feature reduction branch, input feature X∈R N×C , where N = h × w, h and w are the height and width of the image that produces the input feature, C represents the channel of the input feature X, and the output feature F r First, input feature X to the multi-head QKV generation module to obtain the features of H heads, and the feature of the hth head is Q h , K h and V h , 1≤h≤H, for V h Perform dynamic convolution operation,

[0020] in Represents V hThe result of dynamic convolution, g k To generate weights, W k is the kth convolution kernel, K is the number of convolution kernels, and the output features of all heads are connected to get F r , F r is the output feature of the feature reduction branch.

[0021] Preferably, in step S6, for the improved linear attention module, the feature is input to the attention branch of the module to obtain the feature F a , input the feature to the feature reduction branch of this module, and obtain the feature F r , and then get the output feature F of the improved linear attention module, F = F a +F r .

[0022] Preferably, in step S7, the public area smoking image scaling module inputs the public area smoking image X and outputs the scaled public area smoking image y. Where R is the base scaler, R is implemented by bilinear resampling, and for the lth layer in the k layers, G l and G l+1 represents the Gaussian filter matrix, G l =GaussianKernel(σ l ), G l+1 =GaussianKernel(σ l ), σ l+1 =σ l +Δσ,σ l is the standard deviation of layer l, the initial standard deviation is σ1, Δσ represents the increment, GaussianKernel represents the Gaussian kernel, α l and β l represents the scaling parameter and bias parameter of layer l, and σ represents the nonlinear function.

[0023] Preferably, in step S8, for the heat map regression detection head, it is composed of a 1×1 convolution layer, an upsampling layer and a ReLU activation function, and there are 19 key points, namely, the tip of the nose, left eye, right eye, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left fingertips, right fingertips, neck, center of the chest, left hip, right hip, left knee, right knee, left ankle, and right ankle.

[0024] Compared with the prior art, the present invention has the following technical effects:

[0025] The technical solution provided by the present invention proposes an improved linear attention module, guides linear attention to focus on more important features, improves the detection capability of the visual Transformer backbone network, and proposes a lightweight public area smoking image scaling module, which is applied to the public area smoking posture detection model to reduce the reasoning cost. Compared with using a separate smoking posture detection model or a smoking target detection model, by applying the public area smoking posture detection model and the public area smoking target detection model, the accuracy of public area smoking detection can be improved while keeping the reasoning cost the same. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a flow chart of the public area smoking posture alarm monitoring provided by the present invention.

[0027] Figure 2 This is a structural diagram of the improved linear attention module provided by the present invention.

[0028] Figure 3 It is a structural diagram of the public area smoking image zoom module provided by the present invention.

[0029] Figure 4 This is a structural diagram of the public area smoking posture detection model provided by the present invention.

[0030] Figure 5 It is a structural diagram of the public area smoking target detection model provided by the present invention.

[0031] Figure 6 This is a cigarette detection effect diagram based on the public area smoking target detection model provided by the present invention.

[0032] Figure 7 This is a posture detection effect diagram based on the public area smoking posture detection model provided by the present invention. DETAILED DESCRIPTION

[0033] The present invention aims to propose a method for alarming and monitoring smoking postures in public areas, proposes an improved linear attention module, guides linear attention to focus on more important features, improves the detection capability of the visual Transformer backbone network, and proposes a lightweight public area smoking image scaling module, which is applied to the public area smoking posture detection model to reduce the inference cost. By applying the public area smoking posture detection model and the public area smoking target detection model, the accuracy of public area smoking detection is improved.

[0034] See also Figure 1 As shown, a method for monitoring smoking posture alarm in a public area in an embodiment of the present application:

[0035] S1. Obtain image data of smoking postures in public areas. Collect images of smoking postures in public areas, a total of 300 images, with a width and height of 1024, including five smoking postures, namely standing smoking, walking smoking, sitting smoking, squatting smoking and leaning smoking; S2. Annotation and target annotation of smoking postures in public areas. In the posture annotation stage, the five smoking postures are annotated with posture key points. Each posture has a total of 19 key points, namely the tip of the nose, left eye, right eye, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left fingertips, right fingertips, neck, chest center, left hip, right hip, left knee, right knee, left ankle, right ankle. In the target annotation stage, the cigarette position is annotated with targets, and the annotated position is the hand area holding the cigarette;

[0036] S3, build a multi-head QKV generation module, including linear projection operations;

[0037] S4, build the attention branch, including the multi-head QKV generation module and the guidance function;

[0038] S5, construct feature restoration branch, including multi-head QKV generation module and dynamic convolution;

[0039] S6. Build an improved linear attention module, which includes 1 attention branch and 1 feature restoration branch;

[0040] S7, building a public area smoking image scaling module, including a basic scaler;

[0041] S8. Construct a smoking posture detection model in public areas, whose structure is as follows: Figure 4 As shown, it consists of input, public area smoking image scaling module, visual Transformer backbone network, heat map regression detection head and output. The visual Transformer backbone network is implemented using ViT-B / 16, and the improved linear attention module is used to replace the self-attention module in the visual Transformer backbone network.

[0042] S9. Build a public area smoking target detection model, whose structure is as follows Figure 5 As shown, it consists of input, visual Transformer backbone network, object detection head and output in sequence. The visual Transformer backbone network is implemented using ViT-B / 16, and the improved linear attention module is used to replace the self-attention module in the visual Transformer backbone network.

[0043] S10, public area smoking posture alarm monitoring, real-time input of public area surveillance camera video, extracting frames per second and inputting them into the public area smoking posture detection model and the public area smoking target detection model, when the smoking posture and cigarette are detected at the same time, the smoking alarm information is output.

[0044] Further, in step S3, for the multi-head QKV generation module, the input feature X∈R N×C , where N = h × w, h and w are the height and width of the image that generates the input feature, respectively, C represents the channel of the input feature X, and the input feature is divided into H heads, each with a dimension of d k =C / H, define Q h , K h and V h is the feature of the h-th head, 1≤h≤H, in and It is to generate Q h , K h and V h The corresponding linear projection operation is implemented by the linear layer, and both

[0045] Further, in step S4, for the attention branch, the input feature X∈R N×C , where N = h × w, h and w are the height and width of the image that produces the input feature, C represents the channel of the input feature X, and the output feature F a First, input feature X to the multi-head QKV generation module to obtain the features of H heads, and the feature of the hth head is Q h , K h and V h , 1≤h≤H, for each head h, the attention output feature N = h × w, The query vector Q representing the i-th position h , represents the key vector K at the jth position h , is the value vector V at the jth position h , 1≤i≤N, Lead represents the leading function, which is calculated as in x **p represents the element-wise power p of x, ReLU is the activation function, ||x|| is the norm of x, T represents transposition, and finally the output features of all heads are connected to get F a , F a is the output feature of the attention branch.

[0046] Further, in step S5, for the feature reduction branch, input feature X∈R N×C , where N = h × w, h and w are the height and width of the image that produces the input feature, C represents the channel of the input feature X, and the output feature F rFirst, input feature X to the multi-head QKV generation module to obtain the features of H heads, and the feature of the hth head is Q h , K h and V h , 1≤h≤H, for V h Perform dynamic convolution operation,

[0047] in Represents V h The result of dynamic convolution, g k To generate weights, W k is the kth convolution kernel, K is the number of convolution kernels, and the output features of all heads are connected to get F r , F r is the output feature of the feature reduction branch.

[0048] Further, in step S6, for the improved linear attention module, its structure is as follows Figure 2 As shown, the input feature is fed into the attention branch of the module to obtain the feature F a , input the feature to the feature reduction branch of this module, and obtain the feature F r , and then get the output feature F of the improved linear attention module, F = F a +F r .

[0049] Further, in step S7, for the public area smoking image scaling module, its structure is as follows: Figure 3 As shown, the public area smoking image X is input and the scaled public area smoking image y is output. where R is the base scaler, R is implemented by bilinear resampling, and for the lth layer in the k layers, G l and G l+1 represents the Gaussian filter matrix, G l =GaussianKernel(σl),G l+1 =GaussianKernel(σ l+1 ), σ l+1 =σ l +Δσ,σ l is the standard deviation of layer l, the initial standard deviation is σ1, Δσ represents the increment, GaussianKernel represents the Gaussian kernel, α l and β l represents the scaling parameter and bias parameter of layer l, and σ represents the nonlinear function.

[0050] Furthermore, in step S8, for the heat map regression detection head, which is composed of a 1×1 convolution layer, an upsampling layer and a ReLU activation function, there are 19 key points, namely, the tip of the nose, left eye, right eye, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left fingertips, right fingertips, neck, center of the chest, left hip, right hip, left knee, right knee, left ankle, and right ankle.

[0051] Furthermore, the public area smoking posture detection model and the public area smoking target detection model are written in Python language, based on the PyTorch deep learning framework using NVIDIA's CUDA and cuDNN for GPU acceleration, with the CUDA version being 11.6 and the cuDNN version being 8.4.1.

[0052] Further, in step S8, the width and height of the input public area smoking image are both 1024, and the number of channels is 3. After the public area smoking image scaling module, the width and height of the output scaled public area smoking image are both 512, and the number of channels is 3.

[0053] Furthermore, the cigarette detection effect of the public area smoking target detection model in step S9 is as follows: Figure 6 As shown, in Figure 6 The right area shows the cigarette detection frame. Figure 6 The two larger black rectangles in the figure are covered for the privacy of passers-by and have nothing to do with the detection of smoking objects.

[0054] Furthermore, the posture detection effect of the public area smoking posture detection model in step S8 is as follows: Figure 7 As shown, in Figure 7 The right area shows 19 key points of standing smoking posture. Figure 7 The two larger black rectangles in the figure are covered for the privacy of passers-by and have nothing to do with smoking posture detection.

[0055] The above are only preferred embodiments of the present invention. It should be pointed out that a person skilled in the art can make several modifications and improvements without departing from the inventive concept of the present invention, and these all fall within the protection scope of the present invention.

Claims

1. A method for monitoring smoking posture alarm in public areas, characterized in that: The following steps are involved: S1. Obtaining smoking posture image data in public areas, collecting smoking posture images in public areas, including five smoking postures, namely, standing smoking, walking smoking, sitting smoking, squatting smoking, and leaning smoking; S2. Smoking posture labeling and target labeling in public areas. In the posture labeling stage, the five smoking postures are labeled with posture key points. In the target labeling stage, the cigarette position is labeled with targets. S3, build a multi-head QKV generation module, including linear projection operations; S4, build the attention branch, including the multi-head QKV generation module and the guidance function; S5, construct feature restoration branch, including multi-head QKV generation module and dynamic convolution; S6. Build an improved linear attention module, including an attention branch and a feature restoration branch; S7, building a public area smoking image scaling module, including a basic scaler; S8. Build a public area smoking posture detection model, which consists of input, public area smoking image scaling module, visual Transformer backbone network, heat map regression detection head and output in sequence, and use the improved linear attention module to replace the self-attention module in the visual Transformer backbone network; S9. Build a public area smoking target detection model, which consists of input, visual Transformer backbone network, target detection head and output in sequence, and use the improved linear attention module to replace the self-attention module in the visual Transformer backbone network; S10, public area smoking posture alarm monitoring, real-time input of public area surveillance camera video, extracting frames per second and inputting them into the public area smoking posture detection model and the public area smoking target detection model, and outputting smoking alarm information when both the smoking posture and the cigarette are detected; In step S6, for the improved linear attention module, the feature is input to the attention branch of the module to obtain the feature F a , input the feature to the feature reduction branch of this module, and obtain the feature F r , and then get the output feature F of the improved linear attention module, F = F a +F r ; In step S7, the public area smoking image scaling module inputs the public area smoking image X and outputs the scaled public area smoking image y. where R is the base scaler, R is implemented by bilinear resampling, and for the lth layer in the k layers, G l and G l+1 represents the Gaussian filter matrix, G l =GaussianKernel(σ l ), G l+1 =GaussianKernel(σ l+1 ), σ l+1 =σ l +Δσ,σ l is the standard deviation of layer l, the initial standard deviation is σ1, Δσ represents the increment, GaussianKernel represents the Gaussian kernel, α l and β l represents the scaling parameter and bias parameter of layer l, and σ represents the nonlinear function.

2. A method for monitoring smoking posture in public areas according to claim 1, characterized in that: In step S3, for the multi-head QKV generation module, the input feature X∈R N×C , where N = h × w, h and w are the height and width of the image that generates the input feature, respectively, C represents the channel of the input feature X, and the input feature is divided into H heads, each with a dimension of d k =C / H, define Q h , K h and V h is the feature of the h-th head, 1≤h≤H, in and It is to generate Q h , K h and V h The corresponding linear projection operation is implemented by the linear layer, and both 3. The method for monitoring smoking posture in public areas according to claim 2, characterized in that: In step S4, for the attention branch, the input feature X∈R N×C , where N = h × w, h and w are the height and width of the image that produces the input feature, C represents the channel of the input feature X, and the output feature F a First, input feature X to the multi-head QKV generation module to obtain the features of H heads, and the feature of the hth head is Q h , K h and V h , 1≤h≤H, for each head h, the attention output feature N=h×W Represents the query vector Q at the i-th position h , represents the key vector K at the jth position h , is the value vector V at the jth position h , 1≤i≤N, Lead represents the leading function, which is calculated as in x **p represents the element-wise power p of x, ReLU is the activation function, ||x|| is the norm of x, T represents transposition, and finally the output features of all heads are connected to get F a , F a is the output feature of the attention branch.

4. The method for monitoring smoking posture in public areas according to claim 1, characterized in that: In step S5, for the feature reduction branch, the input feature X∈R N×C , where N = h × w, h and w are the height and width of the image that produces the input feature, C represents the channel of the input feature X, and the output feature F r First, input feature X to the multi-head QKV generation module to obtain the features of H heads, and the feature of the hth head is Q h , K h and V h , 1≤h≤H, for V h Perform dynamic convolution operation, in Represents V h The result of dynamic convolution, g k To generate weights, W k is the kth convolution kernel, K is the number of convolution kernels, and the output features of all heads are connected to get F r , F r is the output feature of the feature reduction branch.

5. The method for monitoring smoking posture in public areas according to claim 1, characterized in that: In step S8, for the heat map regression detection head, it is composed of a 1×1 convolution layer, an upsampling layer and a ReLU activation function. There are 19 key points in total, namely, the tip of the nose, left eye, right eye, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left fingertips, right fingertips, neck, center of the chest, left hip, right hip, left knee, right knee, left ankle, and right ankle.

Images