An ai detection and identification method and system for illegal smoking of an operating vehicle driver

Abstract

The application relates to artificial intelligence computer vision, deep learning model fusion and intelligent traffic monitoring technology, and discloses an AI detection and identification method and system for illegal smoking of an operating vehicle driver, which comprises two types of labels of illegal smoking and normal driving, samples of multiple vehicle types, multiple light rays, multiple occlusions and multiple postures are collected, a training set and a verification set are divided according to 4:1, an exclusive data set is constructed through geometric enhancement, pixel enhancement, occlusion simulation and standardization processing, and a YOLO12x-cls local feature extractor and a Swin-Base global feature extractor are built. In the application, the local details and the global context features are complementary, the limitation of single model identification is broken through, and the best verification accuracy reaches 99.83%, the problems that, in the prior art, the model calculation amount is huge, it is difficult to adapt to the low computing power environment of the vehicle terminal, the sensitivity to local fine-grained features is insufficient, and the single model feature extraction capability of the AI detection technology is limited are solved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence computer vision, deep learning model fusion and intelligent traffic monitoring technology, and in particular to an AI detection and identification method and system for drivers of commercial vehicles smoking illegally. Background Technology

[0002] As the intelligent transportation industry demands increasingly sophisticated safety monitoring of operating vehicles, real-time identification of drivers' illegal smoking behavior (holding a cigarette in their hands, smoking actions) has become a key technology for reducing traffic accident risks. Existing technologies suffer from the following core shortcomings, making it difficult to meet practical application needs. According to statistics from the Guangzhou Transportation Association, the accuracy rate for identifying drivers illegally smoking in in-vehicle environments is only 58%. Especially under the conditions of black-and-white pixel cameras in in-vehicle environments, actions such as eating lollipops, sucking fingers, or biting pen caps are easily identified as illegal smoking.

[0003] In existing technologies, traditional YOLO series models, such as YOLOv8 / 9, excel at capturing local detailed features, such as object edges and local actions, but they are insufficient at capturing global context, such as driver posture and ambient lighting in the vehicle. This leads to a sharp drop in recognition accuracy in occluded or changing lighting scenarios. While pure Transformer models can capture global context, their computational demands are enormous, making them unsuitable for the low-computing-power environment of in-vehicle terminals. Furthermore, they lack sensitivity to fine-grained local features, resulting in limitations in the feature extraction capabilities of single models in AI detection technology. Summary of the Invention

[0004] To achieve the above objectives, the present invention provides the following technical solution: An AI-based detection and recognition method and system for illegal smoking by drivers of commercial vehicles includes: classifying illegal smoking and normal driving into two categories; collecting samples from multiple vehicle types, lighting conditions, occlusions, and poses; dividing the training and validation sets into a 4:1 ratio; constructing a dedicated dataset through geometric enhancement, pixel enhancement, occlusion simulation, and standardization; building a YOLO12x-cls local feature extractor and a Swin-Base global feature extractor; normalizing the two types of features and then concatenating the channels; processing the features through 1×1 convolution dimensionality reduction, ReLU activation, and Dropout layers to output fused features; connecting them to a precise classification head; training the model using an adaptive training strategy; quantizing and optimizing the trained model; adapting it to CPU deployment; and achieving accurate real-time recognition of illegal smoking behavior in complex in-vehicle environments with an accuracy rate ≥99.7%.

[0005] The above technical solution divides the labels into two categories. Through the entire process of sample collection, dataset construction, dual model building, feature fusion, training quantization and CPU deployment, the recognition accuracy is ≥99.7%. It is adapted to the complex in-vehicle environment, effectively reduces the risk of false positives and false negatives in cases of illegal smoking, and contributes to traffic safety.

[0006] As a further description of the above technical solution: The sample covers various types of operating vehicles, including trucks, buses, and ride-hailing vehicles; four types of lighting conditions: strong daylight, cloudy days, weak nightlight, and tunnel lighting; three types of occlusion levels: no obstruction, slight occlusion ≤30%, and moderate occlusion 30%-50%; and five types of driving postures: looking straight ahead, looking to the side, looking down, turning one's head, and raising one's hand.

[0007] The above technical solution covers multiple types of operating vehicles, 4 types of lighting, 3 types of occlusion, and 5 types of poses, fully reflecting actual driving conditions, significantly improving the model's generalization ability, and avoiding recognition errors caused by a single scene.

[0008] As a further description of the above technical solution: Geometric enhancement employs random horizontal flipping with a probability of 0.5, random cropping within the range of 0.8-1.0, and slight rotation of ±5°. Pixel enhancement uses ±20% brightness adjustment, ±15% contrast adjustment, and Gaussian noise with a variance of 0.01. Occlusion simulation adds rectangular occlusion blocks of 10-30 pixels. Normalization is performed to unify the image size to 640×640, normalize the pixel values ​​to [0,1], with a mean of 0.485 and a variance of 0.229.

[0009] The above technical solutions employ geometric enhancement by randomly horizontally flipping (probability 0.5), cropping at a ratio of 0.8-1.0, and slightly rotating by ±5° to simulate camera shake during driving; pixel enhancement uses ±20% brightness, ±15% contrast adjustment, and Gaussian noise with a variance of 0.01 to reproduce light fluctuations and device noise; occlusion simulation adds 10-30 pixel rectangular blocks to restore occlusion conditions such as the steering wheel, and standardizes and unifies the 640×640 size and [0,1] pixel normalization, significantly enhancing the model's anti-interference ability and laying a solid foundation for training and recognition accuracy.

[0010] As a further description of the above technical solution: YOLO12x-cls contains 5 convolutional stages and a global average pooling layer, outputting 768-dimensional local features. Swin-Base contains PatchEmbedding, 4 Transformer stages and a global average pooling layer, outputting 1024-dimensional global features.

[0011] The above technical solution uses YOLO12x-cls to focus on 768-dimensional local fine-grained features such as cigarette outline and hand micro-movements, while Swin-Base captures 1024-dimensional global context such as driving posture and ambient light. The two complement each other to break through the limitations of single-model feature extraction, enhance the ability to recognize complex scenes, and lay a solid core support for a high accuracy of 99.83%.

[0012] As a further description of the above technical solution: Training was performed using GPUs, with batch size set to 150 and epochs to 50. The AdamW optimizer and cosine annealing learning rate scheduling were employed, combined with transfer learning, cross-entropy loss with class weights [1.5, 1.0], and an early stopping strategy.

[0013] The above technical solution, based on GPU training, with reasonable parameters and optimization strategies, accelerates model convergence by 3 times, balances sample distribution, suppresses overfitting, and efficiently achieves the goal of high-precision training.

[0014] As a further description of the above technical solution: S71: Export the float32 ONNX model and perform static int8 quantization based on 1000 calibration set samples; S72: Preserve accuracy in key layers, ensuring accuracy loss after quantization is ≤0.1% and model size is ≤80MB.

[0015] The above technical solution, after static int8 quantization and preservation of key layer accuracy, has a model size of ≤80MB and an accuracy loss of ≤0.1%, which is suitable for the low storage requirements of automotive applications and balances accuracy and deployment flexibility.

[0016] As a further description of the above technical solution: S81: Through operator fusion, computation graph pruning, instruction set adaptation and memory reuse optimization, it is compatible with x86_64 and ARM architecture CPUs. The inference latency of 4-core CPU is ≤180ms and that of 8-core CPU is ≤120ms. It can run continuously for 72 hours without memory leaks and the CPU utilization rate is ≤60%.

[0017] The above technical solution is optimized and adapted to dual-architecture CPUs in multiple dimensions. The inference latency of 4 cores / 8 cores meets the standards, and it can run stably for 72 hours without leakage. It has low CPU utilization and reduces deployment costs and hardware dependence.

[0018] As a further description of the above technical solution: S62: The precise classification head is a 2-layer fully connected + Softmax activation architecture, including FC1 layer: 1792→2048, ReLU activation, Dropout layer: p=0.2, FC2 layer: 2048→2, Softmax activation.

[0019] The above technical solution employs a simplified architecture of "2 fully connected layers + Softmax activation" for accurate classification, deeply adapting to 1792-dimensional fusion features: the first FC1 layer maps the 1792-dimensional features to 2048 dimensions, enhancing non-linear expression through ReLU activation; coupled with a Dropout layer with p=0.2, it effectively suppresses overfitting during training; the second FC2 layer focuses on the core classification task, compressing the 2048-dimensional features to 2 dimensions, and outputting the class probabilities of illegal smoking and normal driving through Softmax. Simultaneously, it incorporates a cross-entropy loss with class weights [1.5, 1.0] to balance the sample distribution, accurately distinguishing similar actions such as lollipop, pen biting, and smoking in black and white pixels, further controlling the misclassification rate to within 0.2%, balancing classification efficiency and accuracy.

[0020] The present invention has the following beneficial effects: 1. This invention, by dividing labels into two categories and through the entire process of sample collection, dataset construction, dual-model building, feature fusion, training quantization, and CPU deployment, achieves an accuracy rate of ≥99.7%, adapting to the complex in-vehicle environment, effectively reducing the risk of false positives and false negatives in detecting illegal smoking, and contributing to traffic safety. It achieves complementary features between local details and global context, overcomes the limitations of single-model recognition, and achieves an optimal verification accuracy of 99.83%. This solves the problems of existing technologies where the model computation is huge, making it difficult to adapt to the low-computing-power environment of in-vehicle terminals, and lacking sensitivity to local fine-grained features, thus limiting the feature extraction capabilities of single-model AI detection technologies.

[0021] 2. This invention utilizes samples covering various types of commercial vehicles, including trucks, buses, and ride-hailing vehicles; four lighting conditions: strong daylight, cloudy days, weak nightlight, and tunnel lighting; three levels of occlusion: no obstruction, slight occlusion ≤30%, and moderate occlusion 30%-50%; and five driving postures: looking straight ahead, looking to the side, looking down, turning head, and raising hand. This achieves targeted data augmentation, improves robustness to in-vehicle environments, and solves the problem of inaccurate recognition of models under different samples and lighting conditions in existing technologies. Attached Figure Description

[0022] Figure 1 This is a detailed diagram of the fusion model architecture in this invention; Figure 2 This is a graph showing the training performance of the model in this invention. Figure 3 This is a schematic diagram illustrating the vehicle scene recognition effect in this invention; Figure 4 This is a flowchart of the CPU deployment process in this invention; Figure 5 This is a schematic diagram of the network layer structure in this invention. Detailed Implementation

[0023] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention.

[0024] Reference Figures 1 to 5 The present invention provides an embodiment of an AI-based method and system for detecting and identifying illegal smoking by drivers of commercial vehicles, comprising: S1: Divide into two categories: illegal smoking and normal driving; S2: Collect samples from multiple vehicle types, under various lighting conditions, with multiple occlusions, and in multiple poses; S3: The training set and validation set are divided in a 4:1 ratio, and a dedicated dataset is constructed through geometric augmentation, pixel augmentation, occlusion simulation and normalization. S4: Build the YOLO12x-cls local feature extractor and the Swin-Base global feature extractor; S5: After normalizing the two types of features respectively, the channels are concatenated, and then processed by 1×1 convolution dimensionality reduction, ReLU activation and Dropout layer to output the fused features; S6: Connect the precise classification heads and use an adaptive training strategy to train the model; S7: Perform quantization optimization on the trained model; S8: Adapts to CPU deployment, enabling accurate and real-time identification of illegal smoking behavior in complex in-vehicle environments, with an accuracy rate of ≥99.7%.

[0025] The above embodiments clearly define the entire process technical framework, covering the entire chain of "label definition - data construction - model building - feature fusion - training optimization - quantization deployment - real-time recognition". The core objective is to adapt to the complex in-vehicle environment. Through dual backbone fusion, dedicated dataset construction and CPU quantization optimization, it ultimately achieves accurate real-time recognition with a recognition accuracy of ≥99.7%, while taking into account the requirements of high robustness and low latency deployment.

[0026] Reference Figures 1 to 5 S21: The sample covers various types of operating vehicles such as trucks, buses, and ride-hailing vehicles, and four types of lighting conditions: strong daytime light, cloudy daytime light, weak nighttime light, and tunnel lighting; three types of occlusion: no obstruction, slight occlusion ≤30%, and moderate occlusion 30%-50%; and five types of driving postures: looking straight ahead, looking to the side, looking down, turning head, and raising hand.

[0027] The above embodiments focus on the comprehensiveness and relevance of the sample scenarios, clearly covering 3 types of operating vehicles: trucks, buses, ride-hailing vehicles, etc.; 4 types of lighting conditions: strong daytime light, cloudy days, weak nighttime light, tunnel lighting; 3 types of occlusion levels: no occlusion, mild occlusion ≤30%, moderate occlusion 30%-50%; and 5 types of driving postures: looking straight ahead, looking to the side, looking down, turning head, and raising hand, laying a data foundation for the model's generalization ability.

[0028] Reference Figures 1 to 5 S31: Geometric enhancement employs random horizontal flipping with a probability of 0.5, random cropping within the range of 0.8-1.0, and a slight rotation of ±5°; S32: Pixel enhancement uses ±20% brightness adjustment, ±15% contrast adjustment, and 0.01 Gaussian noise addition; S33: Occlusion simulation adds a 10-30 pixel rectangular occlusion block; S34: The standardization process resulted in a uniform image size of 640×640, with pixel values ​​normalized to [0,1], a mean of 0.485, and a variance of 0.229.

[0029] The above embodiments refine key parameters for data preprocessing to ensure data quality and resistance to interference: Geometric enhancement: random horizontal flip probability of 0.5, random cropping (0.8-1.0) or slight rotation of ±5°; Pixel enhancement: ±20% brightness adjustment, ±15% contrast adjustment, and 0.01 Gaussian noise addition; Occlusion simulation: 10-30 pixel rectangular occlusion block; Standardization: Uniform size 640×640, pixel values ​​normalized to [0,1], mean 0.485, variance 0.229.

[0030] Reference Figures 1 to 5 S41: YOLO12x-cls contains 5 convolutional stages and a global average pooling layer, outputting 768-dimensional local features; S42: Swin-Base contains PatchEmbedding, 4 TransformerStages and a global average pooling layer, and outputs 1024-dimensional global features.

[0031] The above embodiments clearly define the core structure and functional division of the dual-model approach, achieving feature complementarity: YOLO12x-cls: Contains 5 convolutional stages + global average pooling layers, focusing on extracting 768-dimensional local fine-grained features: cigarette outline, hand movements; Swin-Base: Includes PatchEmbedding + 4 TransformerStages + global average pooling layer, focusing on capturing 1024-dimensional global contextual features: driving posture and ambient light.

[0032] Reference Figures 1 to 5 S61: GPU-based training with batch size=150 and epochs=50. The AdamW optimizer and cosine annealing learning rate scheduling are used, combined with transfer learning, cross-entropy loss with class weights [1.5,1.0], and an early stopping strategy.

[0033] The above embodiments clearly define the quantitative process and core indicators to adapt to deployment requirements: Quantization process: Optimize the parameters and mechanisms of the entire training process, balancing convergence speed and accuracy: Hardware and basic parameters: GPU training, batch size=150, epochs=50; Core configuration: AdamW optimizer, cosine annealing learning rate scheduler; Optimization methods: transfer learning to accelerate convergence, cross-entropy loss with class weights [1.5, 1.0] to balance sample distribution, and early stopping strategy to suppress overfitting.

[0034] Reference Figures 1 to 5 S71: Export the float32ONNX model and perform static int8 quantization based on 1000 calibration set samples; S72: Preserve accuracy in key layers, ensuring accuracy loss after quantization is ≤0.1% and model size is ≤80MB.

[0035] The above embodiments clearly define the quantitative process and core indicators to adapt to deployment requirements: Quantization process: First, export the float32ONNX model, and then perform static int8 quantization based on 1000 calibration set samples; Accuracy Guarantee: Preserve accuracy in key layers, ensuring accuracy loss after quantization is ≤0.1%; Storage optimization: The quantized model size is ≤80MB, meeting the low storage requirements of automotive applications.

[0036] Reference Figures 1 to 5 S81: Through operator fusion, computation graph pruning, instruction set adaptation and memory reuse optimization, it is compatible with x86_64 and ARM architecture CPUs. The inference latency of 4-core CPU is ≤180ms and that of 8-core CPU is ≤120ms. It can run continuously for 72 hours without memory leaks and the CPU utilization rate is ≤60%.

[0037] The above embodiments achieve efficient CPU deployment through multiple optimizations, meeting the real-time and stability requirements of automotive applications. Optimization methods: operator fusion, computation graph pruning, instruction set adaptation (AVX2 / ARMNEON), and memory reuse; Supported architectures: x86_64 (Intel / AMD) and ARM (Cortex-A78 / A55) architectures; Performance metrics: 4-core CPU inference latency ≤180ms, 8-core CPU ≤120ms, no memory leaks after 72 hours of continuous operation, and CPU utilization ≤60%.

[0038] Reference Figures 1 to 5 S62: The precise classification head is a 2-layer fully connected + Softmax activation architecture, including FC1 layer: 1792→2048, ReLU activation, Dropout layer: p=0.2, FC2 layer: 2048→2, Softmax activation.

[0039] The above embodiments feature a streamlined and efficient classification head structure that balances accuracy and resistance to overfitting. Architecture composition: End-to-end design with 2 layers of full connectivity and Softmax activation; Parameters for each layer: FC1 layer: 1792-dimensional input → 2048-dimensional output, ReLU activation; Dropout layer; FC2 layer: 2048-dimensional input → 2 classes output, Softmax activation, to achieve accurate classification of illegal smoking and normal driving.

[0040] The above embodiments are merely illustrative of the technical solutions of the present invention and are not intended to limit it. Anyone skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. An AI-based method and system for detecting and identifying illegal smoking by drivers of commercial vehicles, characterized in that, include: S1: Divide into two categories: illegal smoking and normal driving; S2: Collect samples from multiple vehicle types, under various lighting conditions, with multiple occlusions, and in multiple poses; S3: The training set and validation set are divided in a 4:1 ratio, and a dedicated dataset is constructed through geometric augmentation, pixel augmentation, occlusion simulation and normalization. S4: Build the YOLO12x-cls local feature extractor and the Swin-Base global feature extractor; S5: After normalizing the two types of features respectively, the channels are concatenated, and then processed by 1×1 convolution dimensionality reduction, ReLU activation and Dropout layer to output the fused features; S6: Connect the precise classification heads and use an adaptive training strategy to train the model; S7: Perform quantization optimization on the trained model; S8: Adapts to CPU deployment, enabling accurate and real-time identification of illegal smoking behavior in complex in-vehicle environments, with an accuracy rate of ≥99.7%.

2. The AI ​​detection and identification method and system for illegal smoking by drivers of commercial vehicles according to claim 1, characterized in that, S2 further includes the following steps: S21: The sample covers various types of operating vehicles such as trucks, buses, and ride-hailing vehicles, and four types of lighting conditions: strong daytime light, cloudy daytime light, weak nighttime light, and tunnel lighting; three types of occlusion: no obstruction, slight occlusion ≤30%, and moderate occlusion 30%-50%; and five types of driving postures: looking straight ahead, looking to the side, looking down, turning head, and raising hand.

3. The AI ​​detection and identification method and system for illegal smoking by drivers of commercial vehicles according to claim 1, characterized in that, S3 further includes the following steps: S31: Geometric enhancement uses random horizontal flipping with a probability of 0.5, random cropping range of 0.8-1.0, and slight rotation of ±5°; S32: Pixel enhancement uses ±20% brightness adjustment, ±15% contrast adjustment, and 0.01 Gaussian noise addition; S33: Occlusion simulation adds a 10-30 pixel rectangular occlusion block; S34: The standardization process resulted in a uniform image size of 640×640, with pixel values ​​normalized to [0,1], a mean of 0.485, and a variance of 0.

229.

4. The AI ​​detection and identification method and system for illegal smoking by drivers of commercial vehicles according to claim 3, characterized in that, S4 further includes the following steps: S41: YOLO12x-cls contains 5 convolutional stages and a global average pooling layer, outputting 768-dimensional local features; S42: Swin-Base contains PatchEmbedding, 4 TransformerStages and a global average pooling layer, and outputs 1024-dimensional global features.

5. The AI ​​detection and identification method and system for illegal smoking by drivers of commercial vehicles according to claim 4, characterized in that, S6 also includes the following steps: S61: GPU-based training with batch size=150 and epochs=50. It uses the AdamW optimizer, cosine annealing learning rate scheduling, and combines transfer learning, cross-entropy loss with class weights [1.5,1.0], and an early stopping strategy.

6. The AI ​​detection and identification method and system for illegal smoking by drivers of commercial vehicles according to claim 4, characterized in that, S7 further includes the following steps: S71: Export the float32 ONNX model and perform static int8 quantization based on 1000 calibration set samples; S72: Preserve accuracy in key layers, ensuring accuracy loss after quantization is ≤0.1% and model size is ≤80MB.

7. The AI ​​detection and identification method and system for illegal smoking by drivers of commercial vehicles according to claim 4, characterized in that, S8 further includes the following steps: S81: Through operator fusion, computation graph pruning, instruction set adaptation and memory reuse optimization, it is compatible with x86_64 and ARM architecture CPUs. The inference latency of 4-core CPU is ≤180ms and that of 8-core CPU is ≤120ms. It can run continuously for 72 hours without memory leaks and the CPU utilization rate is ≤60%.

8. The AI ​​detection and identification method and system for illegal smoking by drivers of commercial vehicles according to claim 4, characterized in that: The serial precision classification head in S6 also includes the following steps: S62: The precise classification head is a 2-layer fully connected + Softmax activation architecture, including FC1 layer: 1792→2048, ReLU activation, Dropout layer: p=0.2, FC2 layer: 2048→2, Softmax activation.

Images