Smoking target identification method based on context learning

Abstract

The invention discloses a smoking target identification method based on context learning. The method comprises the following steps: S0, performing coarse screening by using skeleton extraction; the method comprises the following steps: S1, inputting an image into a pre-trained ResNet-101 network for feature extraction; s2, constructing a feature pyramid network, and fusing multi-scale features through a top-down path; s3, splicing to form a global feature map; s4, key features are enhanced through an scSE module; s5, candidate areas are generated and screened; s6, performing context alignment on the candidate areas; and S7, performing classification and regression. The false detection rate and the omission rate in a complex scene can be reduced, and high-precision real-time smoking behavior recognition can be realized as far as possible.

Description

Technical Field

[0001] This invention relates to the field of graph neural network deep learning technology, and specifically to a method for identifying smoking targets based on context learning. Background Technology

[0002] For individuals with small pixel counts in images, it's difficult to extract effective features to ensure accuracy. When people are far from the camera, their pixel counts may be even smaller, and cigarette images will have even smaller pixel counts. Most current papers on smoking behavior detection propose methods and experimental scenarios with clear features and relatively close proximity, leaving limited resources for various complex scenarios.

[0003] Traditional smoking detection methods still suffer from high false positive and false negative rates. Because the visual features of small targets are not readily apparent, the features learned during network fitting may resemble the visual characteristics of a person performing actions such as smoking or making a phone call. This leads to numerous false positives based on similar actions like touching the chin, drinking water, resting one hand on the head, or eating. Furthermore, if the smoker doesn't make any similar gestures but simply holds the cigarette, it may result in some false negatives. Additionally, there are often connections between objects and scenes, and between objects themselves. Deep learning networks capable of modeling contextual information can improve target and scene recognition by simulating human interaction with the relationship between targets and their environment, as well as between targets themselves. Therefore, contextual information can provide additional decision support for small targets with poor quality. However, traditional smoking detection methods do not adequately utilize contextual information, resulting in high false positive and false negative rates. Therefore, there is an urgent need for a smoking target recognition method that incorporates deep learning networks that can integrate contextual information. Summary of the Invention

[0004] This invention aims to solve the technical problems existing in the prior art, and innovatively proposes a smoking target recognition method based on context learning, which can reduce the false detection rate and false negative rate in complex scenarios and achieve high-precision real-time smoking behavior recognition as much as possible.

[0005] To achieve the above objectives, the present invention provides a smoking target recognition method based on context learning, comprising the following steps:

[0006] S0: Use skeleton extraction for coarse screening;

[0007] S1: Input the image into a pre-trained ResNet-101 network for feature extraction;

[0008] S2: Construct a feature pyramid network and fuse multi-scale features through a top-down path;

[0009] S3: Concatenate to form a global feature map ;

[0010] S4: Enhance key features through the scSE module;

[0011] S5: Generate candidate regions and filter them;

[0012] S6: Perform context alignment on the candidate regions;

[0013] S7: Classification and Regression.

[0014] In the above scheme, step S1 further includes:

[0015] S1-1: Extract low-level feature representations;

[0016] The input image Let the network be the first The low-level feature extraction output of each residual stage is:

[0017] ;

[0018] in, This represents low-level feature representation. Indicates the height of the original image; Indicates the width of the original image; The residual function representing low-level feature extraction is expressed as: , For the residual function, These are the convolutional layer weight parameters in the residual block. It is input. It is the residual output, which is the original input plus the residual;

[0019] S1-2: Extract mid-to-low-level feature representations based on low-level feature representations;

[0020] ;

[0021] in, This represents low- to mid-level feature representation. Indicates the height of the original image; Indicates the width of the original image; Represents the residual function for low- to mid-level feature extraction;

[0022] S1-3: Extract mid-to-high-level feature representations based on mid-to-low-level feature representations;

[0023] ;

[0024] in, This represents the mid-to-high level features. Represents the residual function for mid-to-high-level feature extraction;

[0025] S1-4: Extract high-level feature representations based on mid-to-high-level feature representations;

[0026] ;

[0027] in, This represents a high-level feature representation. This represents the residual function for high-level feature extraction.

[0028] In the above scheme: S2-1: Feature dimensionality reduction, the process is as follows:

[0029] ;

[0030] in This indicates the first step in step S1 of the network. Feature map of layer Dimensionality reduction features after convolution operation; This represents a 1×1 convolution operation;

[0031] S2-2: Perform top-down feature fusion on the feature maps after dimensionality reduction of each layer;

[0032] The process is as follows:

[0033] ;

[0034] in, For a transposed convolution with kernel size 4, stride 2, and padding 1, This indicates a convolution operation with a kernel of 3. , , , This represents the feature map after the features from each layer have been fused.

[0035] In the above scheme, step S3 further includes:

[0036] Step S3 also includes:

[0037] S3-1: Perform high-level feature upsampling on the feature map after fusing low-level, medium-high-level, and high-level features;

[0038] ;

[0039] in, This represents the feature map after upsampling of high-level features. This indicates transpose convolution processing. This indicates L2 normalization. Represents the ReLU activation function;

[0040] S3-2: Channel splicing to form a global feature map ;

[0041] ;

[0042] in, This indicates a splicing operation. These are feature maps after upsampling at low-to-medium, medium-to-high, and high levels, respectively. This represents the feature map after low-level feature fusion.

[0043] S3-3: Compressing Global Feature Maps ;

[0044] S3-3-1: Global Feature Map The process of performing grouped convolution is as follows:

[0045] ;

[0046] This represents the global feature map after grouping and concatenation. Indicates a connection. This represents the first group of feature maps after grouping. Indicates the first group after grouping Group feature map, This indicates a grouped convolution operation;

[0047] ;

[0048] here Indicates the first The result of the group after 1×1 convolution, and This represents a 1×1 convolution operation. Indicates the first A subset of the channels of the group, i.e., from the original feature map Extracting consecutive elements from Sub-feature map composed of channels;

[0049] S3-3-2: Shuffle the channels of the results of grouped convolution;

[0050] The channel shuffling operation is as follows:

[0051] ;

[0052] in, This represents the output feature map. This represents the result after convolution. A permutation matrix is ​​used to rearrange the channel order, where:

[0053] ;

[0054] in, Indicates the original channel index is The channel index after shuffling is The permutation matrix; is a permutation function.

[0055] In the above scheme, step S4 further includes:

[0056] S4-1: The input feature map is processed through the channel attention module;

[0057] S4-1-1: Feature map after channel splicing Global pooling is performed as follows:

[0058] ;

[0059] This represents the input feature map after channel concatenation. It is the original input of the scSE module and contains multi-scale or multi-channel feature information. This represents the channel-level features obtained after global average pooling. This indicates a spatial dimension compression operation that extracts global semantic information from each channel;

[0060] S4-1-2: Globally sum the spatial dimensions of the channel-level features after global average pooling, as follows:

[0061] ;

[0062] in: Indicates to The spatial dimensions are globally summed to obtain channel-level statistics, which are then used for subsequent channel weight calculations. Indicates the original channel index is The channel index after shuffling is Channel-level features;

[0063] and Each represents the weight matrix of the fully connected layer for channel attention;

[0064] Represents the ReLU activation function. This represents the Sigmoid activation function. This represents the channel attention weight vector after Sigmoid activation. This indicates an element-wise multiplication operation. This represents the feature map output by the cSE module.

[0065] S4-2: The input feature map is processed through the spatial attention module.

[0066] In the above scheme, step S4-2 further includes:

[0067] S4-2-1: Convolve and compress the input feature map, as shown below:

[0068] ;

[0069] in, Represents a single-channel spatial feature map after convolution and compression;

[0070] S4-2-2: Activate the single-channel spatial feature map using the Sigmoid activation function;

[0071] The process is as follows:

[0072] ;

[0073] This represents the spatial attention weight map obtained after activation, where each element at a spatial location represents the importance of that location. This represents the Sigmoid activation function;

[0074] S4-2-3: Perform position-by-position weighting on the input feature map;

[0075] The process is as follows:

[0076] ;

[0077] This represents the weighted feature map. Indicates a space broadcast multiplication operation;

[0078] S4-3: Perform feature fusion:

[0079] ;

[0080] This represents the final output feature map.

[0081] In the above scheme, step S5 further includes:

[0082] S5-1: Propose possible candidate regions through the RPN (Region Proposal Network);

[0083] S5-1-1: Generate anchor points;

[0084] Define anchor frame set at each spatial location The center coordinates are: ;

[0085] The RPN prediction header is:

[0086] ;

[0087] in, This represents the intermediate feature map obtained by performing a 3×3 convolution on the input feature map F and then activating it with ReLU. Representing the foreground probability, for the intermediate feature map The classification prediction branch obtained by performing a 1×1 convolution; For bounding box offset ;

[0088] S5-1-2: Decode the bounding box; decoded as:

[0089] ;

[0090] in: , Indicates the original center coordinates of the anchor frame. , Indicates the width and height of the anchor frame;

[0091] , , , This indicates the coordinates and dimensions of the decoded candidate bounding box;

[0092] , , , This represents the bounding box offset predicted by the RPN regression head, used to adjust the anchor box to a candidate box that is closer to the true target.

[0093] S5-2: Perform maximum suppression on the candidate region;

[0094] ;

[0095] in, This represents the set of candidate regions retained after maximum suppression. This indicates a nonmaximum suppression operation. Indicates the first One candidate region; Indicates the intersection-union ratio threshold; This indicates the total number of candidate regions.

[0096] In the above scheme, step S6 further includes:

[0097] S6-1: For each feature map Candidate regions Extract its in , , , The corresponding regions on the feature map form a local context.

[0098] S6-1-1: At level P2 of the feature pyramid network, the corresponding region is calculated using the following formula:

[0099] ;

[0100] in, This indicates the corresponding region at level P2. , , , This indicates the coordinates and dimensions of the decoded candidate bounding box;

[0101] S6-1-2: At the P3 level, the corresponding region is calculated using the following formula:

[0102] ;

[0103] in, This indicates the corresponding region at level P3;

[0104] S6-1-3: At level P4, the corresponding region is calculated using the following formula:

[0105] ;

[0106] in, This indicates the corresponding area at level P4;

[0107] S6-1-4: At level P5, the corresponding region is calculated using the following formula:

[0108] ;

[0109] in, This indicates the corresponding area at level P5;

[0110] S6-2: Align its size using the context alignment method and align it with the candidate region. The intrinsic features are concatenated on the channels and then fused with contextual features through convolution.

[0111] In the above scheme, step S7 further includes:

[0112] S7-1: Perform feature alignment on candidate regions;

[0113] S7-1-1: Feature extraction for RoI, the process is as follows:

[0114] ;

[0115] in, Indicates the candidate region The corresponding feature map;

[0116] Indicates the candidate region;

[0117] express operate;

[0118] S7-1-2: Feature extraction is performed using convolution, the process is as follows:

[0119] ;

[0120] in, Indicates to The feature map after 3×3 convolution. Indicates the convolution operation;

[0121] S7-2: Feature concatenation is performed with the contextual auxiliary information from S6, as follows:

[0122] ;

[0123] This represents the context-aided information feature map output by step S6; Indicates splicing, This represents the fused feature map after splicing.

[0124] In the above scheme, step S0 further includes:

[0125] S0-1: First, detect the position of the human body in the image and extract the skeleton information;

[0126] Using the YOLOv8-Pose model, the input RGB image is preprocessed and then subjected to end-to-end inference to output the human bounding box and the coordinates of the core key points, ultimately obtaining structured skeleton data bound to the human ID.

[0127] S0-2: Determine whether each person is likely to smoke based on their skeletal posture;

[0128] Based on the human skeleton key point data output by S1-1, a binary judgment system of "spatial constraints + posture geometric features" is constructed. First, key joint points are screened, focusing on core nodes related to smoking behavior. If the number of missing core nodes of the target human body is ≥2, it is judged as incomplete posture information and will not be included in subsequent analysis. For human bodies with complete nodes, two types of key features are extracted through geometric calculation: one is spatial distance features, which are calculated by constructing the circumcircle of the head with the key points of the top of the head and the chin as the diameter, calculating the Euclidean distance between the key points of the left and right wrist joints and the key point of the chin, and judging whether it is less than 1.5 times the radius of the circumcircle of the head; the other is posture angle features, which are calculated by using the vector dot product formula to calculate the angle formed by the elbow joint and the corresponding shoulder and wrist joints. If the angle is in the range of 30° to 120°, it is judged as a valid posture of raising and bending the arm.

[0129] Based on the above features, the following judgment rules are constructed: when both the conditions of "any wrist joint is close to the chin" and "the corresponding arm is bent" are met, it is marked as "high probability of smoking"; when a human skeleton is detected, it is marked as "medium probability of smoking"; when no human skeleton is detected, it is marked as "no probability of smoking".

[0130] S0-3: Subsequent processing strategy;

[0131] During the training phase, the original image is used as the subsequent input image;

[0132] During the reasoning phase, input should follow the logic below:

[0133] For cases where there is no possibility of smoking: no testing will be conducted;

[0134] For moderate smoking probability: perform subsequent detection on the original image;

[0135] For areas with a high likelihood of smoking: Select key areas as specific inputs.

[0136] If a single image contains both objects with a high probability of smoking and objects with a medium probability of smoking, the entire image is input into the detection model first. After the detection is completed, the detection results of the high probability area are used to cover the original detection output of the corresponding object, achieving a dual optimization of precise focusing and global coverage.

[0137] In summary, the beneficial effects of this invention are: 1. It has a multi-level feature fusion and enhancement mechanism: multi-scale features (C2-C5) are extracted through ResNet-101 and FPN is introduced to construct semantically rich pyramid features (P2-P5), effectively integrating low-level details and high-level semantic information. After deconvolution, normalization, and activation operations, P3-P5 are concatenated with P2 to form feature map G, realizing fine-grained feature fusion across levels. By using higher-level abstract features as the context of objects, the ability to represent multi-scale targets is enhanced, especially improving the feature visibility of small targets.

[0138] 2. Dynamic feature optimization design: The scSE module is used to perform dual-path attention weighting on the feature map G, which is both channel and spatial. By autonomously learning channel importance (channel weight) and spatial saliency (spatial weight), it adaptively strengthens key target regions (such as small targets) and suppresses irrelevant background noise, thereby improving feature discriminability and providing a cleaner and more focused feature map F for subsequent detection tasks.

[0139] 3. Local Context-Assisted Decision Framework: A local context-assisted discrimination mechanism is introduced after Region Proposal (RPN). This innovatively extracts the contextual information of candidate regions (RoIs) in high-level features (P4 / P5) and fuses local environmental features through context alignment and channel concatenation. This design combines the RoI features of the target itself with its surrounding semantic context, effectively mitigating occlusion, deformation, and background interference issues, and enhancing the model's robustness to complex scenes. Attached Figure Description

[0140] Figure 1 This is a structural diagram of the present invention.

[0141] Figure 2 Image 1 is the result of processing according to the present invention.

[0142] Figure 3 Image 1 is the ground truth processed.

[0143] Figure 4 Image 2 is the result of processing according to the present invention.

[0144] Figure 5 Image 2 is the ground truth processed. Detailed Implementation

[0145] The present invention will be further described below with reference to the embodiments and accompanying drawings:

[0146] like Figures 1-5 As shown, a method for identifying smoking targets based on context learning includes the following steps:

[0147] S0: Use skeleton extraction for coarse screening;

[0148] S0-1: First, detect the position of the human body in the image and extract the skeleton information;

[0149] Using the YOLOv8-Pose model, the input RGB image is preprocessed and then subjected to end-to-end inference. The output is a human bounding box (filtered with a confidence threshold of 0.5 and deduplicated by NMS) and the coordinates of 17 core key points (valid nodes with a confidence score ≥ 0.3 are selected). Finally, structured skeleton data bound to human ID is obtained.

[0150] S0-2: Determine whether each person is likely to smoke based on their skeletal posture;

[0151] Based on the human skeleton key point data output by S1-1, a binary judgment system of "spatial constraints + posture geometric features" is constructed. First, key joint points are screened, focusing on core nodes related to smoking behavior (chin, left and right shoulder joints, left and right elbow joints, left and right wrist joints). If the number of missing core nodes of the target human body is ≥2, it is judged as incomplete posture information and will not be included in subsequent analysis. For human bodies with complete nodes, two types of key features are extracted through geometric calculation: one is spatial distance features, which are calculated by constructing the circumcircle of the head with the top of the head and the chin key points as the diameter, and calculating the Euclidean distance between the left and right wrist joint key points and the chin key point, and judging whether it is less than 1.5 times the radius of the circumcircle of the head (quantifying the proximity of the hand and the head); the other is posture angle features, which are calculated by using the vector dot product formula to calculate the angle formed by the elbow joint and the corresponding shoulder joint and wrist joint (i.e., the shoulder-elbow-wrist angle). If the angle is in the range of 30°~120°, it is judged as a valid posture of raising and bending the arm. Based on the above features, the following judgment rules are constructed: when both the conditions of "any wrist joint is close to the chin" and "the corresponding arm is bent" are met, it is marked as "high probability of smoking"; when a human skeleton is detected, it is marked as "medium probability of smoking"; when no human skeleton is detected, it is marked as "no probability of smoking".

[0152] S0-3: Subsequent processing strategy;

[0153] During the training phase, the original image is used as the subsequent input image.

[0154] During the reasoning phase, input should follow the logic below:

[0155] For cases where there is no possibility of smoking: no testing will be conducted;

[0156] For moderate smoking probability: perform subsequent detection on the original image;

[0157] For areas with a high likelihood of smoking: Select key areas as specific inputs.

[0158] If a single image contains both objects with a high probability of smoking and objects with a medium probability of smoking, the entire image is input into the detection model first. After the detection is completed, the detection results of the high probability area are used to cover the original detection output of the corresponding object, achieving a dual optimization of precise focusing and global coverage.

[0159] S1: Input the image into a pre-trained ResNet-101 network for feature extraction;

[0160] S1-1: Extract low-level feature representations;

[0161] The input image Let the network be the first The low-level feature extraction output of each residual stage is:

[0162] ;

[0163] in, This represents low-level feature representation. Indicates the height of the original image; Indicates the width of the original image; The residual function representing low-level feature extraction is expressed as: , For the residual function, These are the convolutional layer weight parameters in the residual block. It is input. It is the residual output, which is the original input plus the residual;

[0164] S1-2: Extract mid-to-low-level feature representations based on low-level feature representations;

[0165] ;

[0166] in, This represents low- to mid-level feature representation. Indicates the height of the original image; Indicates the width of the original image; Represents the residual function for low- to mid-level feature extraction;

[0167] S1-3: Extract mid-to-high-level feature representations based on mid-to-low-level feature representations;

[0168] ;

[0169] in, This represents the mid-to-high level features. Represents the residual function for mid-to-high-level feature extraction;

[0170] S1-4: Extract high-level feature representations based on mid-to-high-level feature representations;

[0171] ;

[0172] in, This represents a high-level feature representation. Represents the residual function of high-level feature extraction;

[0173] Since the mainstream camera is 2K (2560) The five residual stages (1440) achieve an optimal balance between preserving sufficient spatial details and extracting strong semantic information, while the receptive field covers the entire image. During this process, the spatial resolution of the feature map decreases progressively while the number of channels increases, forming a hierarchical representation.

[0174] S2: Construct a feature pyramid network and fuse multi-scale features through a top-down path;

[0175] S2-1: Feature dimensionality reduction, the process is as follows:

[0176] ;

[0177] in This indicates the first step in step S1 of the network. Feature map of layer Dimensionality reduction features after convolution operation; This represents a 1×1 convolution operation;

[0178] S2-2: Perform top-down feature fusion on the feature maps after dimensionality reduction of each layer;

[0179] The process is as follows:

[0180] ;

[0181] in, For a transposed convolution with kernel size 4, stride 2, and padding 1, This indicates a convolution operation with a kernel of 3. , , , This represents the feature map after feature fusion from each layer;

[0182] By adaptively reconstructing high-frequency details and sharp edges using learnable parameters, this method effectively mitigates the blurring caused by bilinear interpolation, thereby significantly improving the localization accuracy and boundary clarity of small targets. Simultaneously, this operation can precisely achieve 2x upsampling, enhancing spatial information representation while maintaining feature map size alignment, making it particularly suitable for detail-sensitive small target detection tasks.

[0183] Use ReLU activation. Output feature pyramid. It is now standardized to 256 channels.

[0184] This represents the feature pyramid layer obtained after top-down feature fusion and convolution processing. , , , The output features, corresponding to different scales, form a multi-scale feature pyramid structure, which is used for subsequent tasks such as object detection.

[0185] S3: Concatenate to form a global feature map ;

[0186] S3-1: Perform high-level feature upsampling on the feature map after fusing low-level, medium-high-level, and high-level features;

[0187] ;

[0188] in, This represents the feature map after upsampling of high-level features. This represents the transposed convolution process, and its formula is: (Core size 4×4, step size 2, fill size 1); The L2 normalization process is represented by the following formula: Output ; The ReLU activation function is represented by the following formula: ;

[0189] S3-2: Channel splicing to form a global feature map ;

[0190] ;

[0191] in, This indicates a splicing operation. , , These are feature maps after upsampling at low-to-medium, medium-to-high, and high levels, respectively. This represents the feature map after low-level feature fusion.

[0192] Features retain the highest spatial resolution after stitching. ( ), which utilizes higher-level abstract features as the context of objects.

[0193] S3-3: Compressing Global Feature Maps ;

[0194] The 1024 channels contain a large amount of redundant information. Grouped convolution and channel shuffling methods are used to compress the global feature map of the 1024 channels. ;

[0195] S3-3-1: Global Feature Map The process of performing grouped convolution is as follows:

[0196] Divide the 1024 channels into Groups, each group has Each group has 1 channel. For each group, a 1×1 convolution is applied to separate each group from... The number of channels has been reduced to Each channel. Therefore, the output feature map... The number of channels becomes .

[0197] ;

[0198] here Indicates the first The result of the group after 1×1 convolution, and This represents a 1×1 convolution operation. Indicates the first A subset of channels from the group, i.e., extracting continuous channels from the original feature map G. A sub-feature map composed of channels.

[0199] S3-3-2: Shuffle the channels of the results of grouped convolutions to facilitate information exchange between different groups.

[0200] The channel shuffling operation is as follows:

[0201] ;

[0202] in, This indicates that the channel order of the output feature map is shuffled to ensure cross-group information exchange. This represents the result after convolution. A permutation matrix is ​​used to rearrange the channel order, where:

[0203] ;

[0204] in, Represents the original channel index, indicating the number of a channel in the feature map before shuffling, with a value range of: ; This indicates the channel index after shuffling, representing the new position the channel has been repositioned to after the shuffle. Indicates the original channel index is The channel index after shuffling is The permutation matrix is ​​obtained by the permutation function. Decide; Let be a permutation function, and its expression is:

[0205] ;

[0206] in, This indicates the modulo operation.

[0207] This technical solution adopts =8 and =32, meaning the number of channels has been reduced from 1024 to 256.

[0208] S4: Enhance key features through the scSE module;

[0209] S4-1: The input feature map is processed through the channel attention module;

[0210] S4-1-1: Feature map after channel splicing Global pooling is performed as follows:

[0211] ;

[0212] This represents the input feature map after channel concatenation. It is the original input of the scSE module and contains multi-scale or multi-channel feature information. This represents the channel-level features obtained after Global Average Pooling (GAP). Its function is to compress spatial dimensions and extract global semantic information for each channel;

[0213] S4-1-2: Globally sum the spatial dimensions of the channel-level features after global average pooling, as follows:

[0214] ;

[0215] in: Indicates to The spatial dimensions are globally summed to obtain channel-level statistics, which are then used for subsequent channel weight calculations. Indicates the original channel index is The channel index after shuffling is Channel-level features; Represents the pixel index in the "height direction" of the feature map space, with a value ranging from 1 to... , Represents the pixel index in the "width direction" of the feature map space, with a value ranging from 1 to... ;

[0216] and Both represent the weight matrix of the fully connected layer for channel attention, and , , where r=16 is the compression ratio, used to reduce computational load;

[0217] This represents the ReLU activation function, used to introduce non-linearity and enhance feature representation.

[0218] This represents the Sigmoid activation function, which maps the output to the interval [0, 1] and generates channel attention weights.

[0219] This represents the channel attention weight vector after Sigmoid activation, where each element corresponds to the importance of a channel.

[0220] This indicates an element-wise multiplication operation. This indicates that channel attention weights are applied to the input feature map. We perform channel-by-channel weighting to enhance the characteristics of key channels.

[0221] This represents the feature map output by the cSE module.

[0222] S4-2: Process the input feature map using a spatial attention module;

[0223] S4-2-1: Convolve and compress the input feature map, as shown below:

[0224]

[0225] in, Represents a single-channel spatial feature map after convolution and compression;

[0226] S4-2-2: Activate the single-channel spatial feature map using the Sigmoid activation function;

[0227] The process is as follows:

[0228] ;

[0229] This represents the spatial attention weight map obtained after activation. The elements at each spatial location represent the importance of that location (foreground target areas have high weights, and background areas have low weights). This represents the Sigmoid activation function;

[0230] S4-2-3: Perform position-by-position weighting on the input feature map;

[0231] The process is as follows:

[0232] ;

[0233] This represents the weighted feature map. This represents the spatial broadcast multiplication operation (i.e., the attention weight map is multiplied element-wise with the feature map in the spatial dimension to achieve weighting for different spatial locations).

[0234] S4-3: Perform feature fusion:

[0235] ;

[0236] This represents the final output feature map;

[0237] Channel attention output Spatial attention output By adding element-wise, the final output feature map of the scSE module is obtained. Through this dual-path attention mechanism of "channel + space", the scSE module can simultaneously enhance the semantic information of key channels and the detailed information of key spatial regions, ultimately significantly improving the feature response of small targets.

[0238] S5: Generate candidate regions and filter them;

[0239] S5-1: Propose possible candidate regions through the RPN (Region Proposal Network);

[0240] S5-1-1: Generate anchor points;

[0241] Define anchor frame set at each spatial location (scale Aspect Ratio The center coordinates are:

[0242] The RPN prediction header is:

[0243] ;

[0244] in, This indicates that after performing a 3×3 convolution (512 output channels) on the input feature map F, the intermediate feature map obtained by ReLU activation is used for subsequent classification and regression branches; Representing the foreground probability, for the intermediate feature map The classification prediction branch is obtained by performing a 1×1 convolution (18 output channels). Here, "18" is calculated from "number of anchor boxes (3 scales × 3 aspect ratios = 9) × 2 categories (foreground / background)". For bounding box offset ;

[0245] S5-1-2: Decode the bounding box; decoded as:

[0246] ;

[0247] in: , Indicates the original center coordinates of the anchor frame. , Indicates the width and height of the anchor frame;

[0248] , , , This represents the coordinates and dimensions of the decoded candidate bounding box. The offset is then combined with the original anchor box using a formula to obtain the final candidate bounding box position.

[0249] , , , This represents the bounding box offset predicted by the RPN regression head, used to adjust the anchor box to a candidate box that is closer to the true target.

[0250] S5-2: Perform maximum suppression on the candidate region;

[0251] ;

[0252] This represents the set of candidate regions retained after maximum suppression, which is the final selected non-overlapping, high-confidence target region.

[0253] This represents the non-maximum suppression operation, an algorithm used in object detection to remove overlapping candidate boxes. Its core principle is to retain the candidate box with the highest confidence while suppressing other boxes whose intersection-over-union (IoU) ratio exceeds a threshold.

[0254] Indicates the first Candidate regions.

[0255] This represents the Intersection over Union (IoU) threshold, which is set to 0.7 here. This means that when the IoU between two candidate regions exceeds 0.7, only the region with the higher confidence level is retained.

[0256] This indicates the total number of candidate regions.

[0257] Retain Top-K candidates (K=2000 during training, K=100 during testing).

[0258] Through this series of steps, RPN can generate high-quality candidate regions from the feature map, providing accurate "region proposals" for subsequent object detection tasks.

[0259] S6: Perform context alignment on the candidate regions;

[0260] S6-1: For each feature map Candidate regions Extract its in , , , The corresponding regions on the feature map form a local context.

[0261] S6-1-1: At level P2 of the feature pyramid network, the corresponding region is calculated using the following formula:

[0262] ;

[0263] in, This indicates the corresponding region at level P2. , , , This indicates the coordinates and dimensions of the decoded candidate bounding box;

[0264] S6-1-2: At the P3 level, the corresponding region is calculated using the following formula:

[0265] ;

[0266] in, This indicates the corresponding region at level P3;

[0267] S6-1-3: At level P4, the corresponding region is calculated using the following formula:

[0268] ;

[0269] in, This indicates the corresponding area at level P4;

[0270] S6-1-4: At level P5, the corresponding region is calculated using the following formula:

[0271] ;

[0272] in, This indicates the corresponding area at level P5;

[0273] S6-2: Align its size using the context alignment method and align it with the candidate region. The intrinsic features are concatenated across channels, and then contextual features are fused through convolution.

[0274] Considering that a smaller original ROI (roii) might result in a smaller region being mapped onto the higher-level feature maps (P4, P5) , The context information contained is insufficient, and a minimum size threshold needs to be set: For and If its calculated width or height components ( , , , If the value of a component is less than the preset threshold of 7, then the value of that component is set to 7.

[0275] All mapped regions ( , , , If the boundary of a feature map exceeds the range of the corresponding feature map, it will be truncated to the valid boundary of the feature map.

[0276] For each region after size correction and boundary trimming , , , The RoI Align operation is applied to sample them onto a fixed-size 7 × 7 feature grid.

[0277] S7: Classification and Regression;

[0278] S7-1: Perform feature alignment on candidate regions;

[0279] S7-1-1: Feature extraction for RoI, the process is as follows:

[0280] ;

[0281] in, Indicates the candidate region The corresponding feature map;

[0282] Indicates the candidate region;

[0283] express operate;

[0284] The size of the feature grid;

[0285] S7-1-2: Feature extraction is performed using convolution, the process is as follows:

[0286] ;

[0287] in, Indicates to Feature map after 3×3 convolution;

[0288] S7-2: Feature concatenation is performed with the contextual auxiliary information from S6, as follows:

[0289] ;

[0290] This represents the context-aided information feature map output by step S6. This indicates a splicing operation.

[0291] This represents the fused feature map after splicing.

[0292] The following experiment uses a dataset of real-world smoking surveillance images to illustrate the detection method of this application.

[0293] The on-site smoking monitoring images have a resolution of 2560×1440 pixels. The targets are small targets, less than 1 / 10 the size of the image. A total of 2357 images were collected, of which 2121 were used for training and 236 for validation. The images were taken by high-point cameras at the factory, including images from various indoor and outdoor application scenarios.

[0294] The platform used in this experiment was a 16-core Xeon(R) Platinum 8481C CPU, an RTX 4090D / 24GB GPU, 128GB DDR4 RAM, and a 2TB NVMe SSD. The operating system was Ubuntu 20.04, the deep learning framework was PyTorch 2.0.0, CUDA version was 11.8, Python version was 3.8, and other dependencies included numpy 1.21.0 and opencv-python 4.5.5. During training, the batch size was 4; the entire training process consisted of 100 training epochs, using the Gradient Descent (SGD) optimizer with an initial learning rate of 0.01, momentum of 0.937, and weight decay coefficient of 0.0005. The experimental results are shown in the table below.

[0295]

[0296] Among them, Precision (IoU=0.5): Under the judgment criterion of IoU threshold of 0.5, the proportion of all predicted targets that are actually the target (measures the ability to "predict accurately and not make false predictions").

[0297] Recall (IoU=0.5): The proportion of real targets successfully detected by the model at an IoU threshold of 0.5 (measuring the ability to achieve "full coverage and no missed detections").

[0298] AP50 represents the average precision for a given category with an Intersection over Union (IoU) threshold of 0.5. Specifically, a predicted bounding box is considered a correct detection (TP) when the IoU between the predicted and ground truth bounding boxes is ≥ 0.5. Based on this, precision and recall are calculated at different confidence levels, a precision-recall curve is plotted, and the area under the curve is used to obtain AP50.

[0299] Experimental results show that on a real-world surveillance image dataset, the proposed method outperforms mainstream object detection algorithms in both AP50 and Recall (IoU=0.5). Specifically, the proposed method achieves an AP50 of 85.7%, significantly higher than Fast-RCNN (72.9%) and Faster R-CNN (75.3%), and slightly better than YOLO v5 (84.5%). In terms of precision, the proposed method achieves 81.2%, slightly lower than YOLO v5's 86.3%, but higher than the other two methods. Particularly in terms of Recall, the proposed method, with a recall rate of 87.6%, significantly outperforms the other three methods, demonstrating that it maintains high detection accuracy while possessing stronger false negative suppression capabilities. Its overall performance is excellent, making it suitable for real-world surveillance scenarios with high requirements for detection completeness.

[0300] Comparison images as follows Figures 2-5 As shown in the figure, the blue boxes are the ground truth labels, and the red boxes are the predictions of this technical solution.

Claims

1. A method for identifying smoking targets based on context learning, characterized in that: Includes the following steps: S0: Use skeleton extraction for coarse screening; S1: Input the image into a pre-trained ResNet-101 network for feature extraction; S2: Construct a feature pyramid network and fuse multi-scale features through a top-down path; S3: Concatenate to form a global feature map ; S4: Enhance key features through the scSE module; S5: Generate candidate regions and filter them; S6: Perform context alignment on the candidate regions; S7: Classification and Regression.

2. The smoking target recognition method based on context learning according to claim 1, characterized in that: Step S1 also includes: S1-1: Extract low-level feature representations; The input image Let the network be the first The low-level feature extraction output of each residual stage is: ； in, This represents low-level feature representation. Indicates the height of the original image; Indicates the width of the original image; The residual function representing low-level feature extraction is expressed as: , For the residual function, These are the convolutional layer weight parameters in the residual block. It is input. It is the residual output, which is the original input plus the residual; S1-2: Extract mid-to-low-level feature representations based on low-level feature representations; ； in, This represents low- to mid-level feature representation. Indicates the height of the original image; Indicates the width of the original image; Represents the residual function for low- to mid-level feature extraction; S1-3: Extract mid-to-high-level feature representations based on mid-to-low-level feature representations; ； in, This represents the mid-to-high level features. Represents the residual function for mid-to-high-level feature extraction; S1-4: Extract high-level feature representations based on mid-to-high-level feature representations; ； in, This represents a high-level feature representation. This represents the residual function for high-level feature extraction.

3. The smoking target recognition method based on context learning according to claim 2, characterized in that: Step S2 also includes: S2-1: Feature dimensionality reduction, the process is as follows: ； in This indicates the first step in step S1 of the network. Feature map of layer Dimensionality reduction features after convolution operation; This represents a 1×1 convolution operation; S2-2: Perform top-down feature fusion on the feature maps after dimensionality reduction of each layer; The process is as follows: ； in, For a transposed convolution with kernel size 4, stride 2, and padding 1, This indicates a convolution operation with a kernel of 3. , , , This represents the feature map after the features from each layer have been fused.

4. The smoking target recognition method based on context learning according to claim 2, characterized in that: Step S3 also includes: S3-1: Perform high-level feature upsampling on the feature map after fusing low-level, medium-high-level, and high-level features; ； in, This represents the feature map after upsampling of high-level features. This indicates transpose convolution processing. This indicates L2 normalization. Represents the ReLU activation function; S3-2: Channel splicing to form a global feature map ; ； in, This indicates a splicing operation. These are feature maps after upsampling at low-to-medium, medium-to-high, and high levels, respectively. This represents the feature map after low-level feature fusion. S3-3: Compressing Global Feature Maps ; S3-3-1: Global Feature Map The process of performing grouped convolution is as follows: ； This represents the global feature map after grouping and concatenation. Indicates a connection. This represents the first group of feature maps after grouping. Indicates the first group after grouping Group feature map, This indicates a grouped convolution operation; ； here Indicates the first The result of the group after 1×1 convolution, and This represents a 1×1 convolution operation. Indicates the first A subset of the channels of the group, i.e., from the original feature map Extracting consecutive elements from Sub-feature map composed of channels; S3-3-2: Shuffle the channels of the results of grouped convolution; The channel shuffling operation is as follows: ； in, This represents the output feature map. This represents the result after convolution. A permutation matrix is ​​used to rearrange the channel order, where: ； in, Indicates the original channel index is The channel index after shuffling is The permutation matrix; is a permutation function.

5. The smoking target recognition method based on context learning according to claim 4, characterized in that: Step S4 also includes: S4-1: The input feature map is processed through the channel attention module; S4-1-1: Feature map after channel splicing Global pooling is performed as follows: ； This represents the input feature map after channel concatenation. It is the original input of the scSE module and contains multi-scale or multi-channel feature information. This represents the channel-level features obtained after global average pooling. This indicates a spatial dimension compression operation that extracts global semantic information from each channel; S4-1-2: Globally sum the spatial dimensions of the channel-level features after global average pooling, as follows: ； in: Indicates to The spatial dimensions are globally summed to obtain channel-level statistics, which are then used for subsequent channel weight calculations. Indicates the original channel index is The channel index after shuffling is Channel-level features; and Each represents the weight matrix of the fully connected layer for channel attention; Represents the ReLU activation function. This represents the Sigmoid activation function. This represents the channel attention weight vector after Sigmoid activation. This indicates an element-wise multiplication operation. This represents the feature map output by the cSE module. S4-2: The input feature map is processed through the spatial attention module.

6. The smoking target recognition method based on context learning according to claim 4, characterized in that: Step S4-2 also includes: S4-2-1: Convolve and compress the input feature map, as shown below: ； in, Represents a single-channel spatial feature map after convolution and compression; S4-2-2: Activate the single-channel spatial feature map using the Sigmoid activation function; The process is as follows: ； This represents the spatial attention weight map obtained after activation, where each element at a spatial location represents the importance of that location. This represents the Sigmoid activation function; S4-2-3: Perform position-by-position weighting on the input feature map; The process is as follows: ； This represents the weighted feature map. Indicates a space broadcast multiplication operation; S4-3: Perform feature fusion: ； This represents the final output feature map.

7. The smoking target recognition method based on context learning according to claim 1, characterized in that: Step S5 also includes: S5-1: Propose possible candidate regions through the RPN (Region Proposal Network); S5-1-1: Generate anchor points; Define anchor frame set at each spatial location The center coordinates are: ; The RPN prediction header is: ； in, This represents the intermediate feature map obtained by performing a 3×3 convolution on the input feature map F and then activating it with ReLU. Representing the foreground probability, for the intermediate feature map The classification prediction branch obtained by performing a 1×1 convolution; For bounding box offset ; S5-1-2: Decode the bounding box; decoded as: ； in: , Indicates the original center coordinates of the anchor frame. , Indicates the width and height of the anchor frame; , , , This indicates the coordinates and dimensions of the decoded candidate bounding box; , , , This represents the bounding box offset predicted by the RPN regression head, used to adjust the anchor box to a candidate box that is closer to the true target. S5-2: Perform maximum suppression on the candidate region; ； in, This represents the set of candidate regions retained after maximum suppression. This indicates a nonmaximum suppression operation. Indicates the first One candidate region; Indicates the intersection-union ratio threshold; This indicates the total number of candidate regions.

8. The smoking target recognition method based on context learning according to claim 1, characterized in that: Step S6 also includes: S6-1: For each feature map Candidate regions Extract its in , , , The corresponding regions on the feature map form a local context. S6-1-1: At level P2 of the feature pyramid network, the corresponding region is calculated using the following formula: ； in, This indicates the corresponding region at level P2. , , , This indicates the coordinates and dimensions of the decoded candidate bounding box; S6-1-2: At the P3 level, the corresponding region is calculated using the following formula: ； in, This indicates the corresponding region at level P3; S6-1-3: At level P4, the corresponding region is calculated using the following formula: ； in, This indicates the corresponding area at level P4; S6-1-4: At level P5, the corresponding region is calculated using the following formula: ； in, This indicates the corresponding area at level P5; S6-2: Align its size using the context alignment method and align it with the candidate region. The intrinsic features are concatenated on the channels and then fused with contextual features through convolution.

9. The smoking target recognition method based on context learning according to claim 1, characterized in that: Step S7 also includes: S7-1: Perform feature alignment on candidate regions; S7-1-1: Feature extraction for RoI, the process is as follows: ； in, Indicates the candidate region The corresponding feature map; Indicates the candidate region; express operate; S7-1-2: Feature extraction is performed using convolution, the process is as follows: ； in, Indicates to The feature map after 3×3 convolution. Indicates the convolution operation; S7-2: Feature concatenation is performed with the contextual auxiliary information from S6, as follows: ； This represents the context-aided information feature map output by step S6; Indicates splicing, This represents the fused feature map after splicing.

10. A smoking target recognition method based on context learning according to claim 1, characterized in that: Step S0 also includes: S0-1: First, detect the position of the human body in the image and extract the skeleton information; Using the YOLOv8-Pose model, the input RGB image is preprocessed and then subjected to end-to-end inference to output the human bounding box and the coordinates of the core key points, ultimately obtaining structured skeleton data bound to the human ID. S0-2: Determine whether each person is likely to smoke based on their skeletal posture; Based on the human skeleton key point data output by S1-1, a binary judgment system of "spatial constraints + posture geometric features" is constructed. First, key joint points are screened, focusing on core nodes related to smoking behavior. If the number of missing core nodes of the target human body is ≥2, it is judged as incomplete posture information and will not be included in subsequent analysis. For human bodies with complete nodes, two types of key features are extracted through geometric calculation: one is spatial distance features, which are calculated by constructing the circumcircle of the head with the key points of the top of the head and the chin as the diameter, and calculating the Euclidean distance between the key points of the left and right wrist joints and the key point of the chin, and judging whether it is less than 1.5 times the radius of the circumcircle of the head; the other is posture angle features, which are calculated by using the vector dot product formula to calculate the angle formed by the elbow joint and the corresponding shoulder and wrist joints. If the angle is in the range of 30° to 120°, it is judged as a valid posture of raising and bending the arm. Based on the above features, the following judgment rules are constructed: when both the conditions of "any wrist joint is close to the chin" and "the corresponding arm is bent" are met, it is marked as "high probability of smoking"; when a human skeleton is detected, it is marked as "medium probability of smoking"; when no human skeleton is detected, it is marked as "no probability of smoking". S0-3: Subsequent processing strategy; During the training phase, the original image is used as the subsequent input image; During the reasoning phase, input should follow the logic below: For cases where there is no possibility of smoking: no testing will be conducted; For moderate smoking probability: perform subsequent detection on the original image; For areas with a high likelihood of smoking: Select key areas as specific inputs. If a single image contains both objects with a high probability of smoking and objects with a medium probability of smoking, the entire image is input into the detection model first. After the detection is completed, the detection results of the high probability area are used to cover the original detection output of the corresponding object, achieving a dual optimization of precise focusing and global coverage.

Images