A low-power consumption driving rule violation behavior detection method and system

Abstract

The application discloses a kind of low-power consumption driving training illegal behavior detection method and system, to solve the problems of high terminal running power consumption, large data transmission delay, high single-dimensional detection false and missing report rate, and privacy desensitization difficult to balance compliance requirements and business availability of existing driving training illegal detection technology.The method is applied to the system-on-a-chip hardware layer integrated with CPU and neural network processor NPU: screening by CPU running lightweight illegal classifier, outputting abnormal confidence and intermediate layer feature map;when the abnormal confidence reaches the threshold, the feature data and vehicle operating state data are transmitted through shared memory zero-copy, and the NPU is awakened to perform multi-modal feature fusion precision illegal detection;after detection, the NPU is powered off and hibernates.The application can effectively reduce the terminal whole machine power consumption, shorten the detection response delay, improve the composite rule identification accuracy, and balance the compliance protection and detection function of personal sensitive information.

Description

Technical Field

[0001] This invention relates to the field of driver training time management and edge heterogeneous computing technology, specifically to a low-power driver training violation detection method and system. Background Technology

[0002] As regulations in the driver training industry continue to tighten, national and local traffic management departments have set clear requirements for the authenticity and standardization of the driver training process. Driver training terminals must be able to identify violations to curb cheating such as unauthorized training, parking to accumulate training hours, and training outside of registered locations, thus ensuring the quality of driver training. Under this regulatory trend, violation detection capabilities are gradually shifting from the cloud to the terminal side to reduce transmission dependence and improve response speed. However, existing violation detection solutions for driver training terminals still have several technical shortcomings in actual vehicle power supply and operating environments, making them difficult to adapt to the comprehensive requirements of large-scale deployment.

[0003] Existing detection solutions generally suffer from high power consumption due to algorithm stacking. Current mainstream solutions often employ a dual-model architecture with a target detection network and a classification network connected in series. Some solutions using a heterogeneous architecture of a central processing unit (CPU) and a neural network processor (NPU) maintain the NPU running at full speed. To ensure detection accuracy, the model needs to continuously infer at a rate of 25 to 30 frames per second, resulting in an average power consumption of 3.5W to 5W for the entire system. Since the driver training terminal is directly powered by the vehicle battery, in idling and low-speed training scenarios such as driving test part two, the vehicle engine output power is limited, and the battery itself is undercharged. The continuous high power consumption of the driver training terminal will further exacerbate battery depletion, and long-term use will significantly reduce battery health, even leading to serious malfunctions such as the vehicle failing to start.

[0004] Meanwhile, existing heterogeneous collaborative solutions also suffer from high latency and high bandwidth consumption due to data transfer, and a single detection dimension easily leads to high false alarms and false negatives. When traditional CPUs and NPUs work together, they need to frequently transfer raw images or a large amount of intermediate feature data. A single detection requires the transfer of about 230KB of raw image data, which not only severely encroaches on memory bandwidth but also generates a transmission delay of 15 to 25 milliseconds, making it difficult to meet the real-time alarm requirements for urgent violations. Moreover, existing detection logic is mostly based on image data for judgment, without effectively integrating multi-source heterogeneous data from the vehicle's on-board diagnostic system (OBD) and positioning module. This makes it impossible to cross-validate complex violations and difficult to identify cheating methods such as forgery of vehicle data and positioning data, resulting in high false alarm and false negative rates.

[0005] Furthermore, existing solutions struggle to balance compliance and business usability in sensitive information desensitization. According to the Personal Information Protection Law, terminals must desensitize sensitive information such as facial features. However, existing technologies generally suffer from two problems: first, over-desensitization, where blurred facial features result in insufficient facial recognition, leading to a 1:1 face comparison accuracy rate below 90% on the platform, directly impacting the determination of training hours validity; second, delayed desensitization, where raw image data is temporarily stored in Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM) before desensitization, posing a security risk of personal privacy leakage. Meanwhile, regulatory authorities in various regions require that desensitization not affect the effectiveness of liveness detection. Existing technologies cannot simultaneously meet the dual requirements of compliant desensitization and business functionality usability, making them generally unsuitable for the current regulatory and operational needs of the driver training industry. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a low-power method and system for detecting violations in driver training. It solves the technical problems of existing driver training violation detection technologies, such as excessive terminal power consumption, large data transmission latency, high false positive and false negative rates in single-dimensional detection, and the inability of privacy desensitization to simultaneously meet compliance requirements and business availability. This invention utilizes a heterogeneous processor hierarchical wake-up computing power scheduling mechanism, a shared memory zero-copy data transmission method, a feature fusion detection logic for multi-source heterogeneous data, and front-end parallel desensitization and irreversible feature extraction by the edge Internet Service Provider (ISP). This effectively reduces the overall power consumption of the driver training terminal, shortens the violation detection response latency, and improves the accuracy of identifying complex violations, while simultaneously achieving a balance between compliance protection of sensitive personal information and the effectiveness of the detection function.

[0007] To achieve the above technical objectives, this invention provides a low-power method for detecting violations during driver training, applied to the built-in hardware layer of a driver training terminal. The hardware layer integrates a system-on-a-chip (SoC) containing a central processing unit (CPU) and a neural network processor (NPU). The method includes the following steps:

[0008] S1: The driver training terminal loads a preset violation detection model from the cloud and stores it in the model cache of the NPU. The model cache is configured with an independent reserved power supply domain to ensure that the model data is not lost during a complete power outage of the NPU. Subsequently, the CPU acquires vehicle operating status data, video streams and their corresponding image frames, runs a lightweight violation classifier for processing, and outputs anomaly confidence and intermediate layer feature maps. If the anomaly confidence is lower than a first threshold, the current state is determined to be normal, and the NPU remains in a (completely) power-off sleep state with the power supply domain voltage of the NPU at 0V. If the anomaly confidence reaches or exceeds the first threshold, proceed to step S2.

[0009] S2: The CPU stores the intermediate layer feature map in the shared memory area between the CPU and the NPU, records the memory address pointer of the intermediate layer feature map, writes the memory address pointer to a fixed address in the shared memory area, and can set a read-only lock state for the fixed address to prevent it from being overwritten or modified by other processes; then it sends a wake-up signal to the NPU.

[0010] S3: After the NPU is powered on and started from a (completely) power-off hibernation state, it first reads the memory address pointer from the fixed address in the shared memory region, and then reads the intermediate layer feature map from the shared memory region based on the memory address pointer. At the same time, it reads the vehicle operation status data cached in the shared memory region. The NPU is equipped with a feature fusion layer, which performs multimodal fusion of visual features and vehicle operation status features to output violation detection results.

[0011] S4: After the NPU writes the violation detection result back to the shared memory area, it sets a completion flag and sends an interrupt notification to the CPU; the CPU responds to the interrupt, reads and verifies the violation detection result, and then controls the NPU to power off; the CPU performs processing based on the read result.

[0012] This invention employs a tiered computing power scheduling mechanism: a CPU runs a lightweight violation classifier for pre-detection; the NPU remains in a zero-voltage (complete) sleep state under normal conditions and is only woken up as needed when the anomaly confidence threshold is triggered; and the CPU controls power-off after detection. This mechanism significantly reduces the overall power consumption of the driver training terminal while ensuring detection capabilities, effectively alleviating the problems of power depletion and lifespan degradation caused by long-term high loads in vehicle battery-powered scenarios. Furthermore, relying on the shared memory area of ​​the CPU and NPU to store intermediate layer feature maps and using a zero-copy transmission method for data interaction via memory address pointers, it eliminates the need for extensive copying of original images and feature data. The reduced memory bandwidth consumption and transmission latency caused by data transfer significantly improve the real-time response speed of violation detection. Simultaneously, the multimodal detection logic, which integrates visual intermediate features and multi-dimensional vehicle operating status data on the NPU side, enables multi-source cross-verification of violations, effectively identifying data forgery-related cheating methods and reducing false alarm and false negative rates. Furthermore, the handshake-style power-down mechanism, which notifies the CPU via an interrupt after the NPU writes back the results and then controls power-off after CPU verification, eliminates the risk of result loss due to interrupted operation. Overall, it simultaneously meets the application requirements of low-power operation, real-time alarm, and high-precision recognition for driver training terminals.

[0013] As a technical solution, the loading of the preset violation detection model from the cloud in step S1 specifically includes: the driver training terminal loading a three-level model library with a hierarchical federated index from the cloud to the NPU; the three-level model library includes a general base model, a region-adapted model, and a driving school-customized model. The driver training terminal constructs a hash index based on the region code and driving school code, automatically downloads the corresponding model combination from the cloud, and loads it into the NPU, while using a file lock mechanism to manage the reading and writing of model files. After loading, the model is fixed in the model cache area of ​​the NPU. The cache area uses an independent reserved power supply domain to maintain the model data without loss during a complete power outage of the NPU, eliminating the need to reload the model every time the NPU is woken up. By constructing a three-tiered model library comprising a general base model, a regionally adapted model, and a driving school-customized model, the system can flexibly adapt to the differences in regulatory rules across different regions and the personalized management needs of driving schools, while covering general violation detection scenarios. This effectively compensates for the shortcomings of existing single detection models in terms of scenario adaptability, improving the compliance and detection accuracy of violation detection in different deployment environments. By constructing a hash index using regional codes and driving school codes and automatically matching and downloading corresponding model combinations, the system can achieve precise and automated deployment of terminal models, significantly improving model loading efficiency while avoiding the full model consuming terminal storage space and NPU computing resources. In conjunction with a file lock mechanism to manage the reading and writing process of model files, the system can effectively ensure the consistency and integrity of file data during model updates and loading, avoiding anomalies such as model corruption and loading failures caused by concurrent reading and writing. The model is embedded in a reserved power supply domain cache, ensuring that the model remains usable even after the NPU core is completely powered off (0V), without needing to be reloaded every time the NPU is woken up, thus reducing wake-up latency and avoiding the power consumption overhead caused by repeated loading.

[0014] As a technical solution, in step S1, the CPU acquires the video stream and its corresponding image frames, and runs a lightweight violation classifier for processing. Specifically, the CPU reads the raw video stream captured by the in-vehicle camera through the MIPI-CSI or USB peripheral interface integrated in the system-on-a-chip (SoC) and inputs it to the image signal processing unit (ISP) integrated in the SoC. The ISP has a face detection acceleration unit embedded in it. The ISP performs parallel processing on the raw video stream: one path generates full-image desensitized image frames for behavior pre-detection, and the other path extracts irreversible face feature vectors for identity verification. The raw face data is destroyed after the ISP pipeline is completed and is not written to shared memory. Parallel processing is achieved at the image acquisition front end through the ISP hardware unit integrated on the SoC, eliminating the need to utilize the CPU's main computing power for image desensitization and feature extraction operations. This avoids additional pre-detection latency and computational load on the lightweight violation classifier, ensuring the operational efficiency and low power consumption of the terminal pre-detection process. The parallel processing design meets the different data requirements of behavior detection and identity verification. While meeting personal information protection compliance requirements, it retains the effective visual features required for violation detection—a full-image desensitized image for behavior pre-detection and an irreversible facial feature vector for identity verification. This avoids the problem of reduced recognition accuracy due to excessive full-image desensitization, achieving a balance between privacy compliance and the usability of detection functions. Simultaneously, after generating desensitized image frames, the original image data in the shared memory area is directly overwritten. This prevents original sensitive information such as faces from residing in memory for a long time from the source of data flow, eliminating the security risks of leakage of original sensitive data and achieving privacy and security control throughout the entire edge image processing chain. Furthermore, image acquisition, desensitization, and caching are all completed based on the shared memory area, eliminating the need to allocate additional independent memory space. This saves terminal memory resources and adapts to the subsequent CPU-NPU zero-copy data interaction architecture, further ensuring the operational efficiency of the overall detection process.

[0015] As a technical solution, the irreversible blurring process employs at least one of the following methods: using a Gaussian blur algorithm to replace the pixel values ​​within the sensitive area with the weighted average of its neighboring pixels; using a mosaic algorithm to divide the sensitive area into fixed-size pixel blocks and unify the pixel value of each block to the average pixel value of that block. The ISP dynamically switches the desensitization intensity based on the current detection task type of the lightweight violation classifier. A strong desensitization mode is used when the detection task is a behavior-related determination, and a weak desensitization mode is used when the detection task is an identity-related determination. The desensitization intensity is configured in conjunction with the detection task.

[0016] As a technical solution, the intensity of the irreversible fuzzing process is configured in conjunction with the recognition requirements of the lightweight violation classifier to meet the requirements of motor vehicle driver training supervision for determining the validity of training hours: when the lightweight violation classifier only needs to determine whether the training process actually exists (such as identifying "no one in the driver's seat while the vehicle is in motion" or "both hands off the steering wheel"), the fuzzing radius is increased, and only the key features of the human skeleton are retained; when the lightweight violation classifier needs to determine whether the training was conducted by the individual (such as identifying "not the student driving"), the fuzzing radius is decreased, and feature information for liveness detection is retained in the facial eye and nose areas to ensure that the authenticity of the training process is verified.

[0017] As a technical solution, the video stream in step S1 is collected by an in-vehicle camera and includes information on the driver's face, behavior, and environmental background. The vehicle operating status data includes an in-vehicle time-series data stream and a positioning data stream. The in-vehicle time-series data stream is obtained through the vehicle's on-board diagnostic (OBD) interface and includes information on vehicle speed, engine speed, and seat belt status. The positioning data stream is obtained by a positioning module built into the driver training terminal. The positioning module supports the Global Positioning System or the BeiDou Navigation Satellite System and includes coordinates, driving speed, and heading angle information.

[0018] As a technical solution, in step S2, the CPU stores the generated intermediate layer feature map in the shared memory region to prevent it from being overwritten or modified by other processes; the memory address pointer is written to a fixed address in the shared memory region to achieve zero-copy data transmission. This avoids the process of copying and moving the original image or large-volume feature data across memory in the traditional CPU-NPU collaborative architecture, significantly reducing memory bandwidth usage, effectively eliminating processing latency introduced by data transmission, and greatly improving the response speed of violation detection. The memory address pointer is 8 bytes long, corresponding to a very small data transmission volume, which hardly occupies system bus bandwidth or processor computing power. This shortens the feature reading preparation time after NPU wake-up and avoids the additional power consumption caused by frequent data copying, further meeting the design goal of low-power operation of the driver training terminal.

[0019] As a technical solution, in step S3, the NPU does not perform any data copying or format conversion when reading data. It directly reads the intermediate layer feature map and vehicle operation status data in the shared memory area through the memory access controller. This completely eliminates the additional overhead of copying and format adaptation in the cross-processor data transfer process. This further reduces the processing latency of the data reading process and improves the overall response speed of violation detection. It also avoids the computing power consumption and memory bandwidth occupation caused by the data copying process, further reducing the additional power consumption of the heterogeneous collaboration process. At the same time, it reduces the degree of CPU intervention in the data interaction process and ensures the efficient use of processor computing power resources.

[0020] As a technical solution, the NPU in step S3 has a feature fusion layer that performs multimodal fusion on the following features: visual features, which are driver facial features, liveness features, and motion features obtained by further processing the intermediate layer feature map through the NPU's internal network layer; OBD temporal features, which are vehicle motion pattern features extracted from vehicle speed sequence, RPM sequence, and seat belt status sequence; and positioning temporal features, which are position movement features extracted from coordinate sequence, driving speed sequence, and heading angle sequence. The fused features are input into a standardized feature extraction model, and the violation detection result is output. The NPU internally incorporates a feature fusion layer to perform multimodal fusion of visual features, OBD temporal features, and positioning temporal features. This enables cross-validation of video, vehicle diagnostic, and positioning data, overcoming the limitations of single-visual-dimensional detection. It effectively identifies complex cheating behaviors such as vehicle data forgery and positioning data tampering, significantly reducing the false positive and false negative rates of violation detection. Furthermore, the feature fusion process is entirely completed within the NPU, eliminating the need to transfer intermediate feature data across processors. This ensures efficient multimodal feature fusion processing and avoids the latency and power consumption overhead associated with cross-core data transfer from multiple sources, resulting in highly efficient and accurate violation detection results.

[0021] As a technical solution, the violation detection results in step S3 include three types: unauthorized training, parking violations for credit hours, and training outside the registered location, along with the corresponding confidence levels for each type. The detection of unauthorized training employs a dual-dimensional judgment method combining identity verification and behavioral verification: the identity verification score is calculated using the following formula: The formula for calculating the behavioral verification score is: The confidence level for calculating parking credit hours is as follows: The confidence level calculation formula for training outside of registered venues is as follows:

[0022] In the formula, Confidence score for face matching For confidence level of liveness detection. To train the confidence level of proper movement, The external environment change index is represented by v, the vehicle speed of the on-board automatic diagnostic system is v, the engine speed is r, and the displacement distance is D. For the speed of the Global Positioning System, For markings within the electronic fence, Match confidence levels to the scene. Scoring is given for trajectory anomalies. For vehicle speed threshold, This refers to the engine speed threshold. Displacement threshold For indicator functions, , , , , , , , , , , , The weights are multimodal fusion weights, and the sum of the weights in each formula is 1. The above formula incorporates multi-dimensional feature parameters, indicator functions, and threshold judgment mechanisms, which can combine multi-source data to output judgment results, improving the accuracy of identification in various violation scenarios. At the same time, the weights in the formula can be flexibly adjusted according to the regulatory requirements of different regions and driving school management standards to adapt to the differences in judgment scales under different scenarios, thereby improving the scenario adaptability and compliance of the solution.

[0023] As a technical solution, in step S4, the CPU reads the violation detection results in the shared memory area and performs processing based on the violation detection results. Specifically, this includes: the CPU reads the violation detection results in the shared memory area via an interrupt; if a violation exists, a local alarm is triggered, the desensitized video streams before and after the violation occur are captured, and the vehicle time-series data stream, positioning data stream, and video stream for the corresponding time period are captured simultaneously, packaged and uploaded to the management platform, and the corresponding learning time is marked as abnormal; if normal, the cached data in the shared memory area is cleared, and the process returns to step S1 to continue monitoring. By using an interrupt-driven approach to read detection results from shared memory, timely detection of violations can be achieved, ensuring timely response to violation alarms and handling. When a violation is detected, a local alarm is automatically triggered, and multi-source data is simultaneously captured, packaged, uploaded, and marked as abnormal training hours. This ensures the complete preservation of the violation evidence chain, meeting the requirements for driver training supervision in terms of evidence collection and training hour management, and realizing a closed-loop process of detection, alarm, evidence collection, and reporting. When a normal result is detected, the shared memory cache data is cleared in a timely manner, freeing up memory space, preventing historical data accumulation from interfering with subsequent detection processes, and ensuring the stability of the system's continuous cyclical monitoring.

[0024] As a technical solution, the SoC has a built-in power management unit (PMU) to perform dynamic computing power allocation, specifically including: when the normal state is determined in step S1, controlling the CPU to run in a low-frequency, low-power mode of no more than 400MHz, and keeping the NPU power supply domain voltage at 0V; after the wake-up signal is triggered in step S2, increasing the power supply voltage of the CPU and the shared memory region to full voltage within no more than 5ms, and switching the CPU to a high-frequency mode of no less than 1.2GHz to improve the deterministic response of pointer writing and wake-up signal; during the NPU's inference task execution in step S3, the available time slice of a single CPU core in a unit scheduling cycle is hard-limited to 20% of the maximum theoretical time through scheduler quota control, releasing power budget to prioritize the NPU; after the NPU is powered off in step S4, restoring the CPU to a low-frequency, low-power mode of no more than 400MHz within no more than 2ms. The power management unit implements phased dynamic computing power allocation, matching corresponding computing power and power supply strategies for different stages of the detection process: during the normal pre-detection stage, the CPU runs at a low frequency and the NPU is completely powered off to minimize standby power consumption; during the wake-up stage, the power supply and CPU frequency are rapidly increased to ensure deterministic wake-up response; during the NPU inference stage, the CPU time slice quota is limited, and the power budget is released to prioritize NPU computing power supply; after detection is completed, it quickly falls back to low-power mode. This mechanism can significantly reduce the average operating power consumption of the entire machine while ensuring detection real-time performance and inference performance, alleviating the power depletion problem in vehicle battery-powered scenarios, and achieving a precise balance between power consumption and performance. As a technical solution, the dynamic computing power allocation step also includes a temperature sensing and adjustment mechanism. This mechanism is designed to address the heat accumulation characteristics within the sealed plastic casing of the driver training terminal under summer sun exposure conditions in training vehicles. When the SoC junction temperature exceeds a preset first temperature threshold T1, a T1 stage adjustment is executed: the NPU's computing core frequency is reduced to 30%~60% of its rated frequency, and the video sampling interval for the CPU running the lightweight violation classifier is lengthened from Δt1 to Δt2 (Δt2≥2·Δt1), while ensuring uninterrupted power supply to the NPU. When the junction temperature exceeds a preset second temperature threshold T2, a T2 stage adjustment is executed: the NPU's inference task is paused, while the CPU's lightweight violation classifier remains operational until the junction temperature drops below T1, returning to the T1 stage. When the junction temperature exceeds a preset third temperature threshold T3, a T3 stage adjustment is executed: the SoC's overall power-off protection is triggered. To address the challenges of training vehicles operating under summer sun exposure and the tendency for heat to accumulate in enclosed environments, a three-tiered progressive temperature sensing and adjustment mechanism is implemented. This mechanism can gradually adopt a graded strategy as the chip junction temperature rises, including frequency reduction, extended sampling intervals, suspension of high-power modules, and overall power-off protection, thus balancing detection capabilities and thermal protection requirements across different temperature ranges.This mechanism can retain core detection functions and avoid detection interruption when the temperature is slightly high, while gradually reducing system heat output when the temperature increases, preventing the terminal from crashing or being damaged due to overheating. It effectively improves the stability and reliability of the system operation in high-temperature environments, extends the service life of terminal hardware, and adapts to the complex temperature environment of vehicles.

[0025] The violation detection system based on the aforementioned low-power driver training violation detection method includes: a data acquisition unit for acquiring video streams and vehicle operating status data; a circular video buffer module connected to the data acquisition unit for continuously caching the anonymized video stream data in a circular queue to support the interception of the video stream before the violation occurs; and a SoC integrating a CPU, NPU, image signal processing unit (ISP), power management unit (PMU), and shared memory area. The ISP embeds a face detection acceleration unit for parallel processing of the original video stream in the image signal processing pipeline: one path generates full-image anonymized image frames for behavior pre-detection, and the other path extracts irreversible face feature vectors for identity verification. The original face data is destroyed after processing in the ISP pipeline and is not written to the shared memory. The CPU is connected to the data acquisition unit and runs a lightweight violation classifier to process the video stream, generating intermediate layer feature maps while outputting anomaly confidence levels. If the anomaly confidence level is lower than a first threshold, the current state is determined to be normal, and the NPU remains in a completely power-off sleep state. The intermediate layer feature map is not stored. If the anomaly confidence level reaches or exceeds a first threshold, the intermediate layer feature map is stored in the shared memory region, the memory address pointer of the intermediate layer feature map is written to a fixed address in the shared memory region, and a wake-up signal is sent. After powering on from a completely power-off state, the NPU reads the intermediate layer feature map from the shared memory region based on the memory address pointer, and simultaneously reads the vehicle operation status data cached in the shared memory region. The NPU has a feature fusion layer, which performs multimodal fusion of the visual features obtained by further processing the intermediate layer feature map through the internal network layer of the NPU with the OBD time-series features and positioning time-series features corresponding to the vehicle operation status data, outputs the violation detection result and writes it back to the shared memory region, sets a completion flag and sends an interrupt notification to the CPU. In response to the interrupt, the NPU reads and verifies the violation detection result in the shared memory region. After verification, it sends an NPU power-off command to the PMU. After receiving the power-off command, the NPU enters a completely power-off sleep state. The CPU performs corresponding processing according to the detection result. The aforementioned technical solution constructs an edge-side driver training violation detection system based on a heterogeneous system-on-a-chip. Through a collaborative architecture of a data acquisition unit, a central processing unit, a neural network processor, and shared memory, it achieves a two-level detection mechanism of "lightweight pre-inspection - high-precision fine inspection," balancing detection accuracy and operating power consumption. Relying on shared memory to achieve zero-copy data interaction eliminates the data transmission bottleneck of traditional heterogeneous architectures and reduces detection latency. The neural network processor adopts a normal power-off and on-demand wake-up operating mode, significantly reducing the average system power consumption and adapting to the deployment environment powered by vehicle batteries. Overall, it can achieve low power consumption, low latency, and high-precision driver training violation detection, meeting the comprehensive needs of industry supervision and terminal deployment.

[0026] As a technical solution, it also includes a three-level model library stored in the cloud, which is used to store the features of basic face recognition, liveness detection and scene classification models. The three-level model library includes a general base model, a region-adapted model and a driving school customized model. When the system starts, the CPU builds a hash index based on the region code and driving school code, automatically downloads and loads the corresponding model combination from the cloud to the NPU, and uses a file lock mechanism to manage the reading and writing of model files.

[0027] As a technical solution, the system-on-a-chip also integrates an image signal processing unit (ISP); the CPU obtains the original video stream through the peripheral interface of the driver training terminal, calls the ISP to locate sensitive areas of the original video stream to perform irreversible blurring processing, generates desensitized image frames, and uses the desensitized image frames to cover the original image frames in the original shared memory area, and uses the desensitized image frames as the input of the lightweight violation classifier.

[0028] As a technical solution, the lightweight violation classifier running on the CPU and the deep detection network on the NPU side are the front and back segments of a cascaded network that are jointly trained and uniformly segmented. The lightweight violation classifier includes a lightweight convolutional neural network, whose network structure includes an input layer, two or more convolutional layers, pooling layers, and an output layer. The intermediate layer feature map is the feature tensor output by the lightweight violation classifier at the last convolutional layer, and the anomaly confidence is a scalar value output by the output layer. As a technical solution, the CPU locks the generated intermediate layer feature map in a designated area of ​​the shared memory region to prevent it from being overwritten or modified by other processes. The memory address pointer is written to a fixed address in the shared memory region, and only an 8-byte memory address pointer is transmitted, without transmitting the original image data or the intermediate layer feature map data, thus achieving zero-copy data transmission.

[0029] As a technical solution, it also includes a power management unit, which is configured to: when the system is determined to be in a normal state, control the CPU to run in a low-frequency, low-power mode of ≤400MHz and ensure that the NPU power supply domain voltage is 0V; after triggering a wake-up signal, increase the CPU and shared memory region voltage to full voltage within ≤5ms and switch the CPU to a high-frequency mode of ≥1.2GHz to improve the deterministic response of pointer write and wake-up signal; during the NPU's execution of inference tasks, through the scheduler quota control mechanism, hard limit the available time slice of a single CPU core in a unit scheduling cycle to 20% of the maximum theoretical time, thereby releasing the power budget and prioritizing the computing power supply of the NPU; after the NPU is powered off, restore the CPU to a low-frequency, low-power mode of ≤400MHz within ≤2ms; and execute a temperature sensing and adjustment mechanism.

[0030] The present invention provides a low-power driver training violation detection method and system. It employs a heterogeneous hierarchical detection architecture with a CPU running a lightweight classifier for pre-detection and an NPU in normal zero-voltage sleep mode with on-demand wake-up. This is combined with a zero-copy data interaction mechanism using shared memory with fixed address pointers, and a power management unit with phased dynamic frequency and voltage adjustment and CPU time slice quota control for computing power scheduling. This significantly reduces the average power consumption of the entire system while ensuring real-time detection, effectively alleviating the problems of power depletion and lifespan degradation in vehicle battery-powered scenarios. Simultaneously, through the NPU's internal feature fusion layer, multimodal cross-validation of visual features, OBD timing features, and positioning timing features is achieved. Coupled with a quantifiable violation confidence calculation formula, it can accurately identify complex cheating behaviors such as data forgery, significantly reducing the false alarm rate and false negative rate of violation detection. This is further enhanced by combining a cloud-based three-level model library and hash index for automatic adaptation. The system features a flexible matching mechanism that adapts to different regional regulatory rules and the personalized management needs of driving schools, enhancing the solution's scenario adaptability and compliance assessment. Furthermore, relying on on-chip ISP hardware, it performs targeted desensitization of sensitive areas at the image acquisition front end. The desensitization intensity is configured in conjunction with the detection task requirements, preventing original sensitive information from residing in memory at the data flow source and meeting personal information protection compliance requirements. It also selectively retains necessary features such as liveness detection, addressing the pain points of traditional solutions where excessive desensitization leads to detection failure and insufficient desensitization poses compliance risks. Coupled with a three-level progressive temperature sensing adjustment mechanism and data protection mechanisms such as feature map memory locking and file locking, it can adapt to the high-temperature conditions of in-vehicle operation under summer sun exposure, ensuring data consistency and long-term system stability. Overall, it achieves low-power, high-precision, high-compliance, and high-reliability end-side detection of driver training violations, fully meeting the regulatory requirements of the driver training industry and the application needs of in-vehicle terminals. Attached Figure Description

[0031] Appendix Figure 1 This is a schematic diagram illustrating the connection relationship between the driver training terminal and external devices in an embodiment of the present invention.

[0032] Appendix Figure 2 This is a schematic diagram of the architecture of the low-power driver training violation detection system according to an embodiment of the present invention.

[0033] Appendix Figure 3 This is a flowchart of a low-power driver training violation detection method according to an embodiment of the present invention.

[0034] Appendix Figure 4 This diagram illustrates the shared memory data interaction and zero-copy transmission mechanism in the low-power driver training violation detection method of this invention. (Detailed implementation details follow.)

[0035] Example 1

[0036] This embodiment provides a low-power driver training violation detection system, applied to a driver training terminal installed in a training vehicle, such as... Figure 1 As shown, the driver training terminal communicates with in-vehicle camera 1, in-vehicle camera 2, and the vehicle's on-board diagnostic (OBD) interface via peripheral interfaces to acquire raw video streams and vehicle operating status data. The driver training terminal integrates a heterogeneous SoC chip with a CPU, NPU, ISP, and PMU. The SoC chip also includes a shared memory area accessible by both the CPU and NPU. The mainboard of the driver training terminal also has peripheral chips connected to the SoC, including a GPS / BeiDou positioning chip, a 4G / 5G / Wi-Fi communication chip, and a DDR memory chip (with a ring-shaped video buffer area within the DDR memory). The driver training terminal connects to a cloud management platform via wireless communication to receive and upload violation detection results from a three-level model library. The system is used to detect three types of driver training violations in real time: unauthorized training, parking to accumulate training hours, and training outside of registered locations, achieving high-precision violation monitoring with low power consumption.

[0037] like Figure 2 As shown, this system includes a data acquisition unit, a ring video buffer module, a heterogeneous computing SoC unit, a shared memory unit, a cloud model library unit, and an alarm and upload unit.

[0038] The data acquisition unit includes a video acquisition module, an OBD data acquisition module, and a positioning acquisition module. The video acquisition module uses a 2-megapixel in-vehicle camera facing the driver's seat to capture raw video streams including the driver's face, driving actions, and driving environment, which are then connected to the SoC via a MIPI-CSI interface. The OBD data acquisition module collects vehicle-mounted time-series data through the vehicle's OBD diagnostic interface, including parameters such as vehicle speed, engine speed, and seatbelt status. The positioning acquisition module uses a Beidou / GPS dual-mode positioning module to collect positioning data such as vehicle latitude and longitude coordinates, real-time driving speed, heading angle, and cumulative displacement distance.

[0039] The circular video buffer module is connected to the video acquisition module of the data acquisition unit. It continuously caches the video stream data after the first de-identification process by the ISP in the DDR using a circular queue. In this embodiment, it is configured to cache 30 seconds of video data before and after the violation occurs, corresponding to a cache capacity of about 20MB. The circular buffer module uses the hardware video encoder built into the SoC to complete the real-time compression encoding of the de-identified video. The normal operating power consumption is about 60mW.

[0040] The heterogeneous computing SoC unit employs a dedicated edge AISoC chip, integrating a CPU subsystem, an NPU neural network processor, an ISP image signal processor (with an embedded face detection acceleration unit), and a power management unit (PMU). The CPU subsystem normally operates in a 400MHz low-power mode, responsible for running a lightweight violation classifier, performing pre-detection, and scheduling system processes; after waking up, it switches to a high-frequency mode of at least 1.2GHz to improve the deterministic response to pointer writes and wake-up signals. The NPU neural network processor has a computing power of 1 TOPS, normally completely powered off and in sleep mode, only waking up on demand, responsible for multimodal feature fusion inference and accurate violation judgment. The ISP image signal processor is a hardware-level image processing unit responsible for performing sensitive area image desensitization without consuming CPU computing power. The power management unit is responsible for power supply control, frequency adjustment, and dynamic power consumption scheduling for each module.

[0041] The shared memory unit is the same physical memory space that is uniformly addressed by the CPU and NPU, and is divided into five major areas: image desensitization storage area, face feature vector buffer, OBD data cache area, positioning data cache area, and pointer fixed address area (fixed starting address 0x2000_0000), realizing zero-copy data interaction between the CPU and NPU.

[0042] The cloud-based model library unit is deployed on a cloud management platform, storing three levels of detection models: a general base model, a regional adaptation model, and a driving school customized model, which can be downloaded and loaded on demand by driver training terminals.

[0043] The alarm and upload unit includes a local audible and visual alarm module, an evidence data packaging module, and a learning time marking module. When a violation is triggered, it performs alarm, evidence retention, abnormal learning time marking, and data upload.

[0044] This embodiment also provides a low-power method for detecting violations during driver training, such as... Figure 3 As shown, the process includes four core stages: pre-screening, wake-up preparation, accurate detection, and result processing. The specific process is as follows:

[0045] Step A1: Multi-source data acquisition and CPU pre-test.

[0046] (1) System initialization and model loading: After the driver training terminal is started, the CPU loads the preset violation detection model from the cloud and stores it in the model cache of the NPU. The cache is configured with an independent reserved power supply domain to maintain the model data without loss during the complete power failure of the NPU. After the model is loaded, the NPU enters a complete power failure hibernation state.

[0047] (2) Data acquisition and caching: After the driver training terminal is started, the data acquisition unit continuously acquires video streams, OBD timing data and positioning data, and caches the raw data in real time to the corresponding area of ​​shared memory.

[0048] (3) Hardware-level parallel image processing and feature extraction: After receiving the raw video stream, the ISP starts parallel processing in the image signal processing pipeline. The raw RAW data is destroyed after processing and does not enter the DDR.

[0049] First path (full image strong desensitization): The ISP performs Gaussian blur (8-pixel radius) + 10×10 mosaic overlay processing on the full image to generate a full image desensitization image frame with completely unrecognizable face, which is written to shared memory area 1 for the CPU lightweight classifier to perform behavioral pre-detection, and at the same time for the circular video buffer module to store violation evidence video.

[0050] The second approach (direct facial feature vector extraction): The ISP's embedded face detection accelerates verification by detecting face regions. The detected face regions are directly fed into the NPU's dedicated feature extraction unit at the end of the ISP pipeline via an internal high-speed bus. This feature extraction unit calculates and outputs a 512-dimensional irreversible facial feature vector in real time, without the face data passing through DDR. This feature vector is generated using the activation values ​​of the intermediate layers of a neural network through a one-way hash mapping. It cannot be reverse-engineered from the original face image and is written to shared memory area 2 for the NPU to calculate face matching confidence and... .

[0051] The original face image data is destroyed after being split at the end of the ISP pipeline. All data written to the shared memory are irreversible desensitized images or irreversible feature values. There are no original face images that can be restored in the terminal memory. (4) Lightweight classifier pre-detection: The CPU runs a lightweight violation classifier (a lightweight convolutional neural network based on MobileNetV3-small cropped optimization, retaining 8 depthwise separable convolutional blocks, with a model parameter size of about 0.8M and a size of about 2.3MB). It inputs desensitized image frames and real-time vehicle operation data, and outputs two results simultaneously: ① the abnormal confidence of the current scene (scalar value, range 0-1); ② the intermediate layer feature map output by the last depthwise separable convolutional block of the network.

[0052] (5) Threshold grading determination: The first threshold is preset to 0.6.

[0053] If the anomaly confidence level is <0.6, the current training status is determined to be normal. The NPU remains in a completely powered-off sleep state (power supply domain voltage is 0V, static power consumption is <1mW), and the CPU maintains a low frequency mode of 400MHz and continuously cycles through pre-checks.

[0054] If the anomaly confidence level is ≥0.6, it is determined that there is a suspected violation, and proceed to step A2.

[0055] In this embodiment, when the driver's seat is unoccupied and the vehicle speed is greater than 5 km / h, the CPU outputs an anomaly confidence level of 0.85, which reaches the first threshold, and proceeds to step A2.

[0056] Step A2: Feature caching and NPU wake-up triggering.

[0057] (1) Feature data storage: The CPU writes the generated intermediate layer feature map into the feature map storage area of ​​shared memory, and at the same time updates the OBD timing data and positioning data in the current time window to the corresponding cache area of ​​shared memory.

[0058] (2) Address pointer writing: The CPU records the memory starting address pointer (8 bytes long) of the intermediate layer feature map, writes the pointer to the fixed address area of ​​the shared memory, and sets the address to a read-only lock state to prevent other processes from tampering with the data.

[0059] (3) Send wake-up signal: The CPU sends the NPU wake-up command to the PMU. The PMU completes the power-on of the NPU power supply domain, clock reset and initialization within 3ms. The NPU enters the working state from the power-off sleep state.

[0060] Step A3: NPU multimodal fusion for accurate detection.

[0061] (1) Zero-copy data reading: After the NPU is powered on, it first reads the memory address pointer from the fixed address area of ​​the shared memory through the direct memory access controller (DMA controller), and then directly reads the intermediate layer feature map, irreversible face feature vector, OBD timing data and positioning data in the shared memory according to the pointer; there is no data copying or format conversion operation throughout the process.

[0062] (2) Multi-dimensional feature extraction:

[0063] Visual features: The deep network inside the NPU continues to reason about the intermediate layer feature maps, and combines the irreversible face feature vector to extract three types of visual features: driver facial features, liveness detection features, and driving action features.

[0064] OBD temporal features: extracting vehicle motion pattern features from vehicle speed sequence, speed sequence, and seat belt status sequence;

[0065] Positioning temporal features: Extract displacement trajectory and position matching features from coordinate sequence, velocity sequence, and heading angle sequence.

[0066] (3) Multimodal feature fusion: The feature fusion layer inside the NPU adopts a weighted splicing strategy to align and fuse the three types of features in dimensions and output a unified fused feature vector; the fusion process is completed entirely inside the NPU without the need to transfer data across processors.

[0067] (4) Violation type determination: The fusion feature input classification head outputs the confidence scores of three types of violations. The specific determination rules and calculation formulas are as follows:

[0068] Category 1: Training not conducted by the individual, which will be determined using a two-dimensional approach of identity verification and behavioral verification.

[0069] The formula for calculating the identity verification score is:

[0070]

[0071] in =0.55, =0.45, + =1, Confidence score for face matching This represents the confidence level for liveness detection. When... Authentication is successful when the value is ≥0.55.

[0072] The formula for calculating the behavioral verification score is:

[0073]

[0074] in =0.40, =0.35, =0.25, + + =1. v is the OBD vehicle speed. =5km / h is the vehicle speed threshold; r is the engine speed. =800rpm is the engine speed threshold; D is the displacement distance. =50m is the displacement threshold; This is an indicator function; it takes the value 1 when the condition within the parentheses is true, and 0 otherwise. Behavioral validation passes when the value is ≥0.50.

[0075] If both identity verification and behavioral verification pass, the training is determined to be genuine; otherwise, it is determined to be training conducted by someone other than the individual.

[0076] The second category: parking for credit hours, integrating multi-source data for cross-validation.

[0077] The confidence formula for calculating the parking credit hours is as follows:

[0078]

[0079] in =0.15, =0.10, =0.20, =0.20, =0.20, + + + + =1. To train the confidence level of proper movement, For GPS speed, .when A score of ≥0.60 is considered a violation of parking regulations for accumulating study hours.

[0080] Where v is the OBD vehicle speed, =5km / h is the vehicle speed threshold; r is the engine speed. =800rpm is the engine speed threshold; D is the displacement distance. =50m is the displacement threshold; To train the confidence score for proper driving technique, a lightweight violation classifier extracts key hand features from desensitized image frames. This score reflects whether the driver is properly holding the steering wheel and performing effective driving maneuvers. A value less than 0.40 is considered non-standard. GPS speed; This is an external environment change index, obtained by the CPU calculating the structural similarity index from consecutive image frames. When <0.15, it is stationary; This is an indicator function; it takes the value 1 when the condition within the parentheses is true, and 0 otherwise. The sum of all weight coefficients is 1.

[0081] when A value ≥0.60 is considered a violation of parking regulations for accumulating study hours. When the weighted formula does not reach the threshold but there are obvious contradictions in the multi-source data, the system further determines the violation through logical rules. These logical rules include: when... <0.40 and <0.15 and When the value is greater than 0, it is determined that the GPS signal simulation device is cheating; when... <0.40 and <0.15 and v> When, it is determined that the OBD signal simulator is cheating; when <0.40 and <0.15 and When =0 and v=0, it is determined to be a real parking violation for credit hours.

[0082] The third category: training is not conducted at the registered site, but combines electronic fences with scene recognition.

[0083] The formula for calculating the confidence level of training at locations not registered is as follows:

[0084]

[0085] in =0.05, =0.05, =0.10, =0.55, =0.25, + + + + =1. v is the OBD vehicle speed. =5km / h is the vehicle speed threshold; r is the engine speed. =800rpm is the engine speed threshold; The marker indicates the area inside the electronic fence (1 indicates inside the fence, 0 indicates outside the fence). Match confidence levels to the scene; Scoring is given for trajectory anomalies. When A score of ≥0.50 indicates a violation of regulations, specifically training outside of a registered training area.

[0086] In this embodiment, the weighting coefficients are set as follows based on the regulatory requirements of the driver training industry and actual application scenarios:

[0087] Authentication detection weight: =0.55, =0.45;

[0088] Behavioral verification detection weights: =0.40, =0.35, =0.25

[0089] Parking time-based learning hour assessment weighting: =0.15, =0.10, =0.20, =0.20, =0.20,

[0090] Weighting of training and testing outside of registered venues: =0.05, =0.05, =0.10, =0.55, =0.25.

[0091] Step A4: Result feedback and NPU power-off hibernation, such as... Figure 4 As shown.

[0092] (1) Detection result write-back and interrupt notification: After the NPU completes inference, it writes the violation detection result (including violation type, corresponding confidence level, and timestamp) back to the result cache area of ​​shared memory, executes the memory barrier instruction to ensure that the result data is written to disk, writes the completion flag bit at a fixed address in shared memory, and sends a completion interrupt signal to the CPU.

[0093] (2) CPU interrupt response and NPU safe power-off: After the CPU responds to the interrupt, it immediately reads the result data in the shared memory and performs data integrity verification; after the verification is successful, the CPU sends an NPU power-off command to the PMU; after the PMU confirms that the NPU has no unfinished bus transactions, it cuts off the power supply to the NPU power domain within 1ms; the NPU returns to a completely power-off sleep state. At the same time, the CPU retains a 500ms timeout polling mechanism. If no interrupt is received, it will actively poll the completion flag bit after the timeout as a fallback guarantee for interrupt exceptions; in this embodiment, the NPU's single wake-up working time is <50ms.

[0094] (3) CPU post-processing: After the CPU reads the detection results:

[0095] If a violation is detected, a local audio-visual alarm is triggered. The full-image desensitized video before and after the violation is captured from the circular video buffer module. The OBD time-series data and location data at the corresponding time are captured simultaneously, encrypted, packaged, and uploaded to the management platform. The corresponding learning time is marked as abnormal learning time. If no violation is detected, the temporary feature data and result data in the shared memory are cleared, and the process returns to step A1 to continue the pre-inspection loop.

[0096] In this embodiment, the NPU power-off adopts a handshake-style safe power-down mechanism controlled by the CPU, which differs from the traditional module self-power-off scheme and eliminates the risk of data loss: Phase 1: After completing inference, the NPU writes the results to shared memory, executes a memory barrier instruction to ensure the data is visible to the CPU, and sets a write completion flag; Phase 2: The NPU notifies the CPU that the results are ready via a hardware interrupt, rather than powering off itself; Phase 3: The CPU responds to the interrupt, reads and verifies the result data (CRC check), confirming the data is complete and valid; Phase 4: The CPU sends an NPU power-off command to the PMU, and the PMU waits for the current bus transaction to complete before cutting off the NPU power supply; Phase 5: The NPU is completely powered off and will be powered on again when the next CPU wake-up signal is triggered. This mechanism ensures that the detection results are 100% written to disk before power-off, eliminating the risk of result loss due to power-off state transitions, and conforms to the heterogeneous architecture design principle of "CPU as the system scheduling master."

[0097] Real-world testing has verified that under normal pre-inspection conditions, the SoC core power consumption is approximately 250mW (including quad-core A53@400MHz continuous inference, ISP processing, and DDR read / write), and the average power consumption of the entire terminal is approximately 500mW (including camera, GPS / BeiDou dual-mode, OBD, and power conversion), which is only 10%~15% of the power consumption of traditional NPU always-on solutions (3.5W~5W). In the idling training scenario of driving test subject 2, continuous operation for 8 hours consumes approximately 4Wh of power, accounting for approximately 0.56% of the power consumption of a 60Ah (720Wh) vehicle battery. Combined with alternator idling charging, the battery voltage drop can be controlled within 0.5V, with no risk of battery depletion. The end-to-end latency for a single violation detection is <60ms, meeting the requirements for real-time alarms. The comprehensive recognition accuracy of the three types of violations reaches 96.2%, with a false alarm rate of less than 2.1%, which is significantly better than traditional single-vision detection solutions.

[0098] Example 2

[0099] Based on Example 1, this embodiment further optimizes the model loading and adaptation mechanism. It uses a three-level cloud-based model library to adapt differentiated violation detection rules for different regions and driving schools, thereby improving the scenario compatibility of the solution.

[0100] 1. Three-tier model library architecture

[0101] The cloud-based model library adopts a layered architecture of a general base model, regionally adapted models, and driving school-customized models, storing detection models at different granularities, with clearly defined responsibilities for each level. The general base model is a nationwide, basic violation detection model covering the common features of three core violation behaviors. It is 12MB in size and adaptable to most common driver training scenarios, serving as the foundational model for all terminals and undergoing unified iterative optimization by the platform. Regionally adapted models are fine-tuned models customized for local regulatory rules in various provinces and cities. For example, some regions have added local detection rules such as "verification of the co-driver," "restrictions on training hours," and "prohibition of training in specific areas." These models are 5MB in size and are derived from transfer learning of the general base model, with each region corresponding to a unique 6-digit regional code. Driving school-customized models are lightweight models customized for the specific management needs of driving schools. For example, some driving schools have added internal management detection items such as "using a phone while driving," "not wearing a seatbelt," and "smoking in the car." These models are 2MB in size, with each driving school corresponding to a unique 8-digit driving school code.

[0102] 2. Automatic loading process for fusion models:

[0103] (1) Index generation and query: After the terminal is powered on and connected to the network, it reads the region code and driving school code configured by itself, generates the model index key value using the SHA-256 hash algorithm, and sends a model query request to the cloud without manual configuration.

[0104] (2) Model combination distribution: The cloud matches the corresponding combination of "general base model + regional adaptation model + driving school customized model" according to the index key value, and encrypts and distributes the model file to the terminal.

[0105] (3) File lock security management: The terminal adopts a file lock mechanism to ensure the integrity of the model file: an exclusive write lock is added during the model download and loading process to prevent concurrent writing from causing damage to the model file; after loading is completed, the write lock is released and a shared read lock is added. During the NPU inference process, the model file can only be read and cannot be modified.

[0106] (4) Model fusion loading: The CPU merges the three-level models into the final inference model according to the weights and loads it into the model cache of the NPU; among them, the general base model accounts for 70% of the weight, the regional adaptation model accounts for 20% of the weight, and the driving school customized model accounts for 10% of the weight, so as to achieve smooth fusion of general rules with local and driving school rules.

[0107] 3. Dynamic update mechanism

[0108] When local regulatory rules or driving school management rules change, the cloud updates the corresponding adaptation model for the region or driving school and pushes the update notification to the terminal. The terminal automatically downloads the updated model when the vehicle is turned off and the terminal is idle. The update process adopts a dual-partition backup mechanism. If the update fails, it will automatically roll back to the previous stable version to ensure uninterrupted system availability.

[0109] 4. Implementation Results

[0110] Through a three-tier model library architecture, we achieve one-time development and multi-location adaptation, improve model deployment efficiency by 80%, reduce terminal storage space usage by 60%, eliminate the need to develop complete models separately for different scenarios, significantly reduce customized development and operation and maintenance costs, and can quickly adapt to regulatory requirements in different regions across the country.

[0111] Example 3

[0112] Based on Example 1, this embodiment further optimizes the image desensitization strategy to achieve dynamic linkage between desensitization intensity and detection requirements, maximizing the effectiveness of the detection function while meeting the compliance requirements of the Personal Information Protection Law.

[0113] 1. Establish a two-level desensitization linkage mechanism. Based on the current detection task type of the lightweight violation classifier, the ISP dynamically adjusts the fuzzing intensity, which is divided into two levels of desensitization mode. The automatic switching does not require manual intervention.

[0114] Mode 1: Behavioral-level Strong Desensitization Mode

[0115] When the detection task only needs to determine the authenticity of the training process, such as detecting purely behavioral violations like "driver unattended," "hands off the steering wheel," or "making or receiving phone calls," there is no need to identify the driver's identity, and a strong desensitization strategy is adopted:

[0116] Processing algorithm: Gaussian blur + mosaic overlay processing, the blur radius of the face area is set to 15 pixels, and a 10×10 pixel mosaic block is overlaid at the same time.

[0117] Desensitization effect: Facial features are completely unrecognizable, and the person's identity cannot be restored from the image, fully meeting the compliance requirements for desensitization of personal sensitive information; at the same time, behavioral characteristics such as the outline of the human head, shoulder and neck posture, and key points of arm bones are preserved.

[0118] Functional verification: The lightweight classifier's accuracy in identifying behavioral violations decreased by less than 2%, without affecting the core detection function;

[0119] Application scenarios: Scenarios where no identity verification is required, such as parking for credit hours or detecting dangerous driving behavior.

[0120] Mode 2: Identity-Level Weak Desensitization Mode

[0121] When the detection task requires determining whether the person trained is the actual individual, and facial comparison and liveness detection are necessary, a weak desensitization strategy is adopted:

[0122] Processing algorithm: Only the mouth and cheek areas of the face are subjected to Gaussian blur with a radius of 5 pixels, while the texture features and contour information of the eyes and bridge of the nose are preserved; at the same time, other sensitive areas such as ID cards and mobile phone screens in the image are automatically detected and subjected to full-screen blur processing.

[0123] Desensitization effect: It is impossible to clearly identify a complete face with the naked eye, reducing the risk of privacy leakage; at the same time, the features of the eye and nose areas are sufficient to support the operation of facial feature comparison and liveness detection algorithms.

[0124] Functional verification: Face matching accuracy decreased by <3%, liveness detection accuracy decreased by <2%, which does not affect the core function of identity verification;

[0125] Application scenarios: Scenarios requiring identity verification, such as student check-in / check-out and training / testing for individuals not personally identified.

[0126] 2. Build end-to-end privacy protection

[0127] The entire de-identification process is completed in the ISP hardware pipeline, without the need for CPU computation, with a processing latency of <1ms and no increase in system power consumption. Furthermore, the de-identified image directly overwrites the original image, and the original facial data does not reside in memory for a long time, eliminating the risk of privacy leakage from the source of data flow. The terminal does not store the original facial image locally, but only uploads the de-identified evidence of violations, and the entire chain complies with the requirements of the Personal Information Protection Law for the processing of biometric information.

[0128] 3. Implementation Results

[0129] According to a third-party compliance assessment, both levels of anonymization meet the requirements for personal information protection compliance. At the same time, compared with the full-image unified strong anonymization solution, the accuracy of identity detection scenarios is improved by 27%, achieving a balance between privacy compliance and business usability.

[0130] Example 4

[0131] Based on Example 1, this embodiment adds a temperature sensing power consumption adjustment mechanism for high-temperature in-vehicle environments, solving the problems of terminal overheating and crashing and shortened hardware lifespan in summer sun exposure scenarios, and improving environmental adaptability.

[0132] When training vehicles are parked outdoors in summer, the temperature in the driver's cab can reach over 70°C, and the junction temperature of the SoC inside the sealed casing of the terminal can easily exceed the rated operating threshold. This embodiment sets three temperature thresholds, corresponding to a progressively increasing adjustment strategy:

[0133] 1. First temperature threshold T1 = 60℃ (mildly high temperature)

[0134] When the SoC junction temperature exceeds 60°C, T1 level regulation is triggered:

[0135] On the NPU side: the core frequency was reduced from the rated 1.0GHz to 400MHz (40% of the rated frequency), reducing power consumption by about 60%; at the same time, the NPU power supply and wake-up functions were kept normal, only the inference speed was reduced, and the single inference time was increased to less than 80ms, still meeting the real-time requirements.

[0136] On the CPU side: The video sampling interval of the lightweight violation classifier was increased from 100ms to 300ms, and the equivalent processing frame rate was reduced from 10fps to about 3fps, reducing the computational power consumption of image processing; at the same time, the CPU frequency was reduced from 1.2GHz to 800MHz, further reducing heat generation.

[0137] Adjustment objective: To reduce system heat output and suppress continuous temperature rise while retaining complete detection functions.

[0138] 2. Second temperature threshold T2 = 70℃ (moderately high temperature)

[0139] When the junction temperature continues to rise above 70°C, T2 level regulation is triggered:

[0140] Pause all NPU inference tasks, block the CPU's NPU wake-up signal, keep the NPU in a power-off sleep state, and no longer respond to wake-up requests triggered by preflight;

[0141] The CPU lightweight pre-detection function is retained, the sampling interval is further extended to 500ms, the CPU frequency is reduced to the lowest power mode of 400MHz, and only basic anomaly monitoring and data recording functions are maintained.

[0142] When the junction temperature is detected to drop below 60°C, the wake-up shield is automatically removed, and the CPU sampling rate and NPU frequency are gradually restored, returning to the T1 level adjustment state.

[0143] Adjustment objective: To pause high-power modules, rapidly reduce system heat generation, prevent continuous temperature increases, and retain minimum detection capabilities.

[0144] 3. The third temperature threshold T3 = 85℃ (severe high temperature)

[0145] When the junction temperature abnormally rises above 85°C, the T3 level hardware protection is triggered.

[0146] The PMU performs overall SoC power-off protection, shutting down power to all modules except power management and temperature monitoring to prevent the chip from burning out due to overheating;

[0147] When the temperature naturally drops below 50℃, the system will automatically restart and resume normal detection functions.

[0148] Adjustment objective: Prioritize hardware safety under extreme high temperatures to avoid permanent damage.

[0149] 4. Auxiliary stabilization mechanism

[0150] This embodiment incorporates a temperature hysteresis mechanism, where there is a 5°C difference between the threshold trigger point and the recovery threshold. This prevents frequent state switching caused by temperature fluctuations near the threshold, thereby improving system stability.

[0151] 5. Implementation Results

[0152] After being exposed to the sun in summer, the highest internal temperature of the terminal was 76℃ when the ambient temperature was 42℃. The system could operate stably in the T2 level regulation state without any failures such as crashes or restarts. Compared with the solution without temperature regulation, the average fault-free working time of the terminal in high-temperature environment is increased by more than 3 times and the hardware lifespan is extended by 40%.

[0153] Example 5

[0154] This embodiment verifies the detection effect of the method on three types of violations through a specific test scenario. The test environment is a compliant training vehicle of a driving school. The terminal uses the low-power detection system of this invention. The comparison scheme is the traditional single-vision NPU constant-on detection system.

[0155] 1. Training and testing not conducted by the individual.

[0156] Example 1: Clear front view + normal driving.

[0157] Face matching confidence =0.85, confidence level for liveness detection =0.90, authentication score ==0.55×0.85+0.45×0.90=0.8725≥0.55, identity verification passed.

[0158] Vehicle speed v=45km / h>5km / h, engine speed r=2000rpm≥800rpm, displacement distance D=500m>50m, behavioral verification score. =0.40×1+0.35×1+0.25×1=1.0≥0.50, the behavior verification passed.

[0159] If both dimensions are passed, it is determined that the training was genuine and will be counted as enrollment hours.

[0160] Example 2: Photo or video attack. Face matching confidence level. =0.85, but the confidence level for liveness detection is... =0.10, authentication score =0.55×0.85+0.45×0.10=0.5125<0.55, identity verification failed, and it was determined that the training was not conducted by the person in question.

[0161] Example 3: Training conducted by someone other than the trainee. Face matching confidence level. =0.10, confidence level for liveness detection =0.85, authentication score =0.55×0.10+0.45×0.85=0.4375<0.55, identity verification failed, and it was determined that the training was not conducted by the person in question.

[0162] Example 4: Students are wearing masks. Face matching confidence level. =0.50, confidence level for liveness detection =0.15, authentication score ==0.55×0.50+0.45×0.15=0.3425<0.55, identity verification failed, determined to be training for someone other than the person in question, trainee required to remove mask and re-verify.

[0163] Example 5: Clear frontal face + vehicle stationary for an extended period. Face matching confidence level. =0.85, confidence level for liveness detection =0.90, authentication score =0.55×0.85+0.45×0.90=0.8725≥0.55, identity verification passed.

[0164] Vehicle speed v=0km / h<5km / h, engine speed r=2000rpm≥800rpm, displacement distance D=0m<50m, behavioral verification score. =0.40×0+0.35×1+0.25×0=0.35<0.50, the behavior verification failed, and it was determined that the training was not conducted by the person in question.

[0165] 2. Parking credits for learning hour assessment

[0166] Example 1: Normal driving, standardized actions.

[0167] The vehicle is in normal driving condition, and the speed is v=45km / h ( (v> =1), Engine speed r = 2000 rpm is higher than the threshold ( (r> )=1), Displacement distance D=500m exceeds the threshold ( (D> =1), Confidence level of action standardization =0.85((1− (=0.15), GPS speed =45km / h ( =0)=0), External Environment Change Index =0.85=((1− =0.15).

[0168] Substitute into the confidence formula for parking credit hours:

[0169] =0.15×1+0.10×1+0.20×0+0.20×0.15+0.20×0+0.15×0.15=0.15+0.10+0+0.03+0+0.0225=0.3025<0.60, therefore it is not considered a violation of parking for credit hours.

[0170] =0.3175, which is far below the parking time threshold of 0.60, so the system judges it as normal driving and the time recording will not be interrupted.

[0171] Example 2: The OBD signal simulator and GPS signal simulation device are used together to forge the signal, and the operation is not standardized.

[0172] The vehicle is actually stationary, but the OBD signal simulator simulates a vehicle speed of v=45km / h. (v> =1), Engine speed r = 2000 rpm ( (r> )=1), GPS speed =45km / h ( =0)=0), displacement distance D=500m ( (D> =1), Confidence of action specification =0.30((1− (=0.70), External Environment Change Index =0.10((1- =0.90).

[0173] =0.15×1+0.10×1+0.20×0+0.20×0.70+0.20×0+0.15×0.90=0.525<0.60, the weighted formula did not reach the violation threshold. However, the system further executed logical rule detection and identified the contradictory combination of "the video shows the vehicle is stationary and its movements are irregular, but the OBD and GPS simultaneously show that the vehicle is moving" (…). <0.40、 <0.15、 0), which was determined to be a joint forgery of the OBD signal simulator and GPS signal simulator, and a violation of parking time-farming regulations.

[0174] Example 3: Accumulating training hours while actually parking. The vehicle is completely stationary, and the OBD speed is v=0km / h (…). (v> =0), engine speed r = 600 rpm ( (r> =0), displacement distance D = 0m ( (D> )=1), GPS speed =0km / h ( =0)=1), Confidence of action specification =0.20((1- (=0.80), External Environment Change Index =0.05((1− =0.95).

[0175] =0.15×0+0.10×0+0.20×1+0.20×0.80+0.20×1+0.15×0.95=0.7025≥0.60, therefore it is determined to be a violation of parking time-farming regulations.

[0176] 3. Training and testing outside of registered venues.

[0177] Example 1: Normal training within the registered venue – not considered a violation

[0178] The vehicle was training normally at the registered site, and all characteristic parameters were within the normal range.

[0179] Vehicle speed v = 45 km / h (v> =1), Engine speed r = 2000 rpm ( (r> =1), Markings within the electronic fence =1((1− =1))=0, the vehicle is actually within the electronic fence of the registered site), scene matching confidence level =0.85((1- =0.15, the background environment captured by the camera is consistent with the scene of the registered site, and the trajectory abnormality score is 0.15. =0.10 (The driving trajectory is consistent with the training route of the registered site).

[0180] Substitute the formula for confidence level of training at locations not registered:

[0181] =0.05×1+0.05×1+0.10×0+0.55×0.15+0.25×0.10=0.2075<0.50, therefore it is not considered a violation.

[0182] 4. Overall Test Conclusion

[0183] This system achieves a comprehensive recall rate of 96.2% and an accuracy rate of 95.8% across the three types of violations. Compared to traditional single-dimensional detection solutions, the accuracy rate is improved by 18.7%, effectively identifying data forgery-related cheating methods. At the same time, the average power consumption of the entire device is reduced by 92%, fully demonstrating the technical advantages of this invention: low power consumption, high precision, and strong anti-cheating capabilities.

[0184] The specific embodiments of the present invention have been described above with reference to the accompanying drawings. However, those skilled in the art should understand that the above embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of the present invention. The scope of protection of the present invention is defined by the appended claims.

Claims

1. A low-power method for detecting violations during driver training, characterized in that, An embedded hardware layer for use in a driver training terminal, the hardware layer integrating a system-on-a-chip (SoC) including a central processing unit (CPU) and a neural network processor (NPU), the method comprising the following steps: S1: The driver training terminal loads a preset violation detection model from the cloud and stores it in the model cache of the NPU. The model cache is configured with an independent reserved power supply domain. The CPU acquires vehicle operating status data, video streams and their corresponding image frames, runs a lightweight violation classifier for processing, and outputs anomaly confidence and intermediate layer feature maps. If the anomaly confidence is lower than a first threshold, the current state is determined to be normal, and the NPU remains in a power-off sleep state. If the anomaly confidence reaches or exceeds the first threshold, proceed to step S2. S2: The CPU stores the intermediate layer feature map in the shared memory area between the CPU and the NPU, records the memory address pointer of the intermediate layer feature map, writes the memory address pointer into a fixed address in the shared memory area, and sends a wake-up signal to the NPU; S3: After the NPU is powered on from a power-off hibernation state, it reads the memory address pointer from a fixed address in the shared memory region, reads the intermediate layer feature map from the shared memory region based on the memory address pointer, and reads the vehicle operation status data cached in the shared memory region at the same time; the NPU is equipped with a feature fusion layer, which performs multimodal fusion of visual features and vehicle operation status features, and outputs violation detection results; S4: After the NPU writes the violation detection result back to the shared memory area, it sets a completion flag and sends an interrupt notification to the CPU; the CPU responds to the interrupt, reads and verifies the violation detection result, and then controls the NPU to power off; the CPU performs processing based on the read result.

2. The low-power driver training violation detection method according to claim 1, characterized in that, The loading of the preset violation detection model from the cloud in step S1 specifically includes: the driver training terminal loading a three-level model library from the cloud to the NPU; the three-level model library includes a general base model, a regional adaptation model, and a driving school customized model. The driver training terminal constructs a hash index based on the regional code and the driving school code, automatically downloads the corresponding model combination from the cloud and loads it into the NPU, and at the same time uses a file lock mechanism to manage the reading and writing of model files.

3. The low-power driver training violation detection method according to claim 1, characterized in that, In step S1, the CPU acquires the video stream and its corresponding image frames, and runs a lightweight violation classifier for processing. Specifically, the CPU obtains the original video stream through a peripheral interface and inputs it to the image signal processing unit (ISP) integrated within the SoC. The ISP has an embedded face detection acceleration unit. The ISP processes the original video stream in parallel: one path generates a full-image desensitized image frame for behavior pre-detection, and the other path extracts irreversible face feature vectors for identity verification. The original face data is destroyed after the ISP pipeline is completed and is not written to shared memory.

4. The low-power driver training violation detection method according to claim 3, characterized in that, The irreversible blurring process employs at least one of the following methods: using a Gaussian blur algorithm to replace the pixel values ​​within the sensitive area with the weighted average of its neighboring pixels; using a mosaic algorithm to divide the sensitive area into fixed-size pixel blocks and unify the pixel value of each block to the average pixel value of that block; the ISP dynamically switches the desensitization intensity based on the current detection task type of the lightweight violation classifier, using a strong desensitization mode when the detection task is a behavior-related determination and a weak desensitization mode when the detection task is an identity-related determination, with the desensitization intensity configured in conjunction with the detection task.

5. The low-power driver training violation detection method according to claim 1, characterized in that, The video stream mentioned in step S1 is collected by the in-vehicle camera and includes the driver's face, behavior, and environmental background information; the vehicle operating status data includes an in-vehicle time-series data stream and a positioning data stream. The in-vehicle time-series data stream is obtained through the vehicle's on-board diagnostic (OBD) system interface and includes vehicle speed, engine speed, and seat belt status information; the positioning data stream is obtained by the positioning module built into the driver training terminal. The positioning module supports the Global Positioning System or the BeiDou Navigation Satellite System and includes coordinates, driving speed, and heading angle information.

6. The low-power driver training violation detection method according to claim 1, characterized in that, In step S2, the CPU stores the generated intermediate layer feature map in the shared memory region and writes the memory address pointer to a fixed address in the shared memory region to achieve zero-copy data transmission.

7. The low-power driver training violation detection method according to claim 1, characterized in that, In step S3, the NPU directly reads the intermediate layer feature map and vehicle operating status data in the shared memory area through the memory access controller.

8. The low-power driver training violation detection method according to claim 1, characterized in that, In step S3, the NPU internally sets up a feature fusion layer to perform multimodal fusion of the following features: visual features, which are driver facial features, liveness features, and motion features obtained by further processing the intermediate layer feature map through the NPU internal network layer; OBD temporal features, which are vehicle motion pattern features extracted from vehicle speed sequence, RPM sequence, and seat belt status sequence; and positioning temporal features, which are position movement features extracted from coordinate sequence, driving speed sequence, and heading angle sequence. The fused features are input into the standardized feature extraction model, and the output is the violation detection result.

9. The low-power driver training violation detection method according to claim 1, characterized in that, The violation detection results in step S3 include three types: unauthorized training, parking violations for credit hours, and training outside the registered location, along with the corresponding confidence levels. The detection of unauthorized training employs a two-dimensional judgment method combining identity verification and behavioral verification. The identity verification score is calculated using the following formula: The formula for calculating the behavioral verification score is: The confidence level for calculating parking credit hours is as follows: The confidence level calculation formula for training outside of registered venues is as follows: In the formula, Confidence score for face matching For confidence level of liveness detection. To train the confidence level of proper movement, The external environment change index is represented by v, the vehicle speed of the on-board automatic diagnostic system is v, the engine speed is r, and the displacement distance is D. For the speed of the Global Positioning System, For markings within the electronic fence, Match confidence levels to the scene. Scoring is given for trajectory anomalies. For vehicle speed threshold, This refers to the engine speed threshold. Displacement threshold For indicator functions, , , , , , , , , , , These are the multimodal fusion weight coefficients, and the sum of the weights in each formula is 1.

10. The low-power driver training violation detection method according to claim 1, characterized in that, In step S4, the CPU performs processing based on the violation detection result, specifically including: the CPU reads the violation detection result in the shared memory area via an interrupt; if a violation exists, a local alarm is triggered, the desensitized video stream before and after the violation occurs is captured, and the vehicle time-series data stream and positioning data stream of the corresponding time period are captured simultaneously, packaged and uploaded to the management platform, and the corresponding learning time is marked as abnormal; if normal, the cached data in the shared memory area is cleared, and the process returns to step S1 to continue monitoring.

11. A violation detection system based on the low-power driver training violation detection method of claim 1, characterized in that, include: The data acquisition unit is used to collect video streams and vehicle operating status data; The SoC integrates a CPU, NPU, image signal processing unit (ISP), shared memory region, and power management unit (PMU); the CPU is connected to the data acquisition unit and is used to run a lightweight violation classifier to process the video stream, generating intermediate layer feature maps while outputting anomaly confidence. If the anomaly confidence level is lower than the first threshold, the current state is determined to be normal, and the NPU remains in a completely power-off sleep state and does not store the intermediate layer feature map; If the anomaly confidence level reaches or exceeds a first threshold, the intermediate layer feature map is stored in the shared memory region, the memory address pointer of the intermediate layer feature map is written to a fixed address in the shared memory region, and a wake-up signal is sent. The NPU, after powering on from a completely power-off state, reads the memory address pointer from the fixed address in the shared memory region, reads the intermediate layer feature map from the shared memory region based on the memory address pointer, and simultaneously reads the vehicle operating status data cached in the shared memory region. The NPU has an internal feature fusion layer, used to perform multimodal fusion of the visual features obtained from the intermediate layer feature map through further processing by the NPU's internal network layer with the OBD temporal features and positioning temporal features corresponding to the vehicle operating status data, outputs the violation detection result, writes it back to the shared memory region, sets a completion flag, and sends an interrupt notification to the CPU. The CPU also responds to the interrupt by reading and verifying the violation detection result in the shared memory region. After successful verification, it sends an NPU power-off command to the PMU. Upon receiving the power-off command, the NPU enters a completely power-off sleep state. The CPU performs corresponding processing based on the detection result.

Images