A target detection system and method based on a Linux kernel V4L2 driver

Abstract

The application provides a target detection system and method based on a Linux kernel V4L2 driver, the system comprising a hardware layer, a kernel space performing target detection preprocessing, and a user space for image display; the kernel space comprising: a V4L2 driving module acquiring original video frames, using a lightweight CNN model as an AI preprocessing model, extracting and directly outputting a feature vector; an asynchronous inference module using an asynchronous inference mechanism to run an AI inference task of the AI preprocessing model, determining an inference result of the feature vector, and writing the inference result into an extension field of a V4L2 buffer area so that the user space reads the inference result for image display. The application can avoid the redundancy of copying frame data from the kernel state to the user state for reprocessing, avoid the problems of delay and high CPU resource occupancy rate caused by user space AI processing, reduce data transmission delay and CPU occupancy rate, and improve system stability and multitasking processing capacity.

Description

Technical Field

[0001] This application belongs to the field of target detection and processing technology, and in particular relates to a target detection system and method based on Linux kernel V4L2 driver. Background Technology

[0002] V4L2 (Video for Linux Two) is a set of APIs provided by the Linux kernel for interacting with video devices such as webcams, TV tuners, and USB cameras, supporting video capture and output.

[0003] Currently, the method of using V4L2 to drive a camera to acquire video data and then calling a neural network model (such as YOLOv5) in user space for object detection mainly includes the following steps: Step 1: The camera acquires raw video data through V4L2 and transmits it to user space. Step 2: The user space calls a deep neural network framework (such as YOLOv5) to perform feature extraction and object detection on the user space data. Step 3: The processed results are returned to the application layer for image display.

[0004] This method suffers from the following technical drawbacks: 1) High latency: After video data is transmitted from kernel space to user space for AI processing, multiple memory copies are involved, resulting in latency exceeding 100ms, making it unsuitable for real-time monitoring. 2) High CPU usage: The deep learning model running in user space consumes a significant amount of CPU resources, leading to decreased system performance and impacting multitasking. Therefore, to address these issues, a target detection scheme that can reduce latency and CPU usage is urgently needed. Summary of the Invention

[0005] This application discloses a target detection system and method based on the Linux kernel V4L2 driver, which can reduce the latency of video target detection, reduce CPU utilization, and improve multi-tasking capabilities and system stability.

[0006] Other objectives and advantages of this application can be further understood from the technical features disclosed herein.

[0007] To achieve one, some, or all of the above objectives or other objectives, in a first aspect, this application provides a target detection system based on the Linux kernel V4L2 driver, including a hardware layer, a kernel space for managing the hardware layer to acquire raw video data and perform target detection preprocessing, and a user space for image display; wherein, the kernel space includes:

[0008] The V4L2 driver module acquires raw video frames, uses a lightweight CNN model as an AI preprocessing model, and extracts and directly outputs feature vectors.

[0009] The asynchronous inference module uses an asynchronous inference mechanism to run the AI ​​inference task of the AI ​​preprocessing model, determines the inference result of the feature vector, and writes it into the extended field of the V4L2 buffer; the user space reads the inference result for image display.

[0010] In some embodiments, the hardware layer includes an image sensor, which is used to generate 1080P YUV420 video frames from the original video frames, receive configuration instructions from the kernel space via I2C, and control the transmission of video frame data to the kernel space via a DMA controller.

[0011] In some embodiments, the lightweight CNN model is a YOLOv5 model, and the feature extraction layer of the YOLOv5 model is used to extract feature vectors, including:

[0012] Memory is allocated using kmalloc, pre-trained model parameters are loaded and inference is performed. The 1080P YUV420 video frames are converted to RGB format and the resolution is scaled to 224*224. Forward propagation is performed to extract 128-dimensional feature vectors and store the feature vectors in the extended field of the V4L2 buffer.

[0013] In some embodiments, the asynchronous reasoning mechanism is kthread or workqueue.

[0014] In some embodiments, the asynchronous inference module uses an asynchronous inference mechanism to run the AI ​​inference task of the AI ​​preprocessing model, including: creating an inference thread using kthread_create, setting the scheduling policy to SCHED_FIFO, setting the priority to be higher than that of ordinary tasks, and writing the inference result into the extended field of the V4L2 buffer.

[0015] In some embodiments, the kernel space includes an error handling module for detecting and handling inference anomalies, including: real-time monitoring of camera I2C communication errors, asynchronously calling the reset_camera function via workqueue, detecting inference anomalies, and controlling the error handling recovery time to be less than 300ms.

[0016] In some embodiments, the user space includes an application layer and a v4l2-ctl tool module, wherein the application layer is used to read inference results for image matching and image display;

[0017] The v4l2-ctl tool module creates a parameter directory through the sysfs interface to configure the model parameters of the AI ​​preprocessing model. The parameter directory includes at least the model enable flag, feature vector dimension, and object detection confidence threshold. The v4l2-ctl tool module dynamically adjusts the model parameters of the AI ​​preprocessing model through the sysfs interface and synchronizes the changes in the model parameters in the kernel space to the user space through kobject and uevent.

[0018] In some embodiments, the kernel space includes a status monitoring module for setting load thresholds and sleep thresholds, monitoring and acquiring CPU load and video stream idle time in real time, reducing the inference frequency of the AI ​​inference task in response to the CPU load exceeding the load threshold, and adjusting the sleep mode of the AI ​​inference task in response to the video stream idle time exceeding the sleep threshold.

[0019] In some embodiments, the inference task takes less than 10ms per frame, and the inference frequency ranges from 5 to 10fps.

[0020] Secondly, this application provides a target detection method based on the Linux kernel V4L2 driver, implemented based on a target detection system based on the Linux kernel V4L2 driver as described in any of the first aspects, the method comprising:

[0021] Initialize the V4L2 driver module and AI preprocessing model in the kernel space;

[0022] Based on the V4L2 driver module, the original video frames are obtained, and a lightweight CNN model is used as an AI preprocessing model to extract and directly output feature vectors.

[0023] The AI ​​inference task of the AI ​​preprocessing model is run using an asynchronous inference mechanism to determine the inference result of the feature vector and write it into the extended field of the V4L2 buffer; the user space reads the inference result for image display.

[0024] The aforementioned target detection system and method based on the Linux kernel V4L2 driver have at least the following beneficial effects: 1) It avoids the redundancy of copying frame data from kernel space to user space for processing, enabling the acquisition and preprocessing of video frame data directly in kernel space. Inference tasks are executed through an asynchronous inference mechanism, and the processed images are then passed to user space, avoiding time-consuming inference blocking and the latency and high CPU resource consumption issues caused by AI processing in user space. This reduces data transmission latency and CPU utilization, while improving system stability and multi-task processing capabilities, meeting the real-time application requirements of intelligent monitoring, drones, etc.; 2) The AI ​​preprocessing scheme of this application relies solely on the CPU, avoiding the reliance on NPU / GPU in existing technologies and reducing hardware costs; 3) This application... Through dynamic inference frequency adjustment and idle sleep mechanisms, the standby and operating power consumption of users is further reduced, extending the battery life of battery-powered devices; 4) The application provides dynamic configuration functions (such as feature dimensions and confidence thresholds) through the sysfs interface, which can adapt to different complex hardware, support mainstream sensors (such as OV5640, IMX219) and platforms (ARM, RISC-V), shorten the development cycle, and simplify the hardware adaptation process; 5) To address the problem of long hardware fault recovery time, the fault recovery time is shortened and the continuity of system monitoring is improved through error detection and asynchronous recovery mechanisms; 6) The status monitoring and parameter configuration of this application can support different application scenarios (such as face detection, vehicle recognition, and liveness detection), enhancing the versatility and market competitiveness of the solution. To make the above and other objects, features, and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the specific embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a system block diagram of the target detection system based on the Linux kernel V4L2 driver in this application.

[0027] Figure 2 This is a schematic diagram of the target detection system based on the Linux kernel V4L2 driver in this application.

[0028] Figure 3 This is a flowchart of the feature extraction process for the target detection system based on the Linux kernel V4L2 driver in this application.

[0029] Figure 4This is a timing diagram of asynchronous inference for the target detection system based on the Linux kernel V4L2 driver in this application.

[0030] Figure 5 This is a flowchart illustrating the target detection method based on the Linux kernel V4L2 driver in this application. Detailed Implementation

[0031] The foregoing and other technical contents, features, and effects of this application will be clearly presented in the following detailed description of a preferred embodiment with reference to the accompanying drawings. The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate; this is merely a method of distinguishing objects with the same attributes in the embodiments of this application.

[0032] In their research, the inventors of this application discovered the following drawbacks of existing methods that use V4L2 to drive cameras to collect video data and then call neural network models in user space for object detection: The AI ​​processing of video data after it is transferred from kernel space to user space involves multiple memory copies, resulting in latency exceeding 100ms, making it unsuitable for real-time monitoring; the high CPU resource consumption of deep learning models running in user space leads to decreased system performance and affects multitasking; many solutions (such as RK3588) rely on dedicated NPUs or GPUs to accelerate AI inference, increasing hardware costs and making them unsuitable for low-cost devices; user-space AI models require re-optimization for different hardware, resulting in long development cycles and poor compatibility; continuous user-space processing leads to high power consumption (>50mW), making it unsuitable for battery-powered edge devices.

[0033] To solve the above problems, such as Figure 1 and Figure 2 As shown in the figure, this application provides a target detection system based on the Linux kernel V4L2 driver, including a kernel space, a user space, and a hardware layer. The kernel space is used to manage the hardware layer to acquire raw video data and perform target detection preprocessing based on the raw video data. The user space is used for image display. The kernel space includes a V4L2 driver module and an asynchronous inference module. The V4L2 driver module acquires raw video frames, uses a lightweight CNN model as an AI preprocessing model, extracts and directly outputs feature vectors. The asynchronous inference module uses an asynchronous inference mechanism to run the AI ​​inference task of the AI ​​preprocessing model, determines the inference result of the feature vectors, and writes it to an extended field of the V4L2 buffer. The user space reads the inference result for image display.

[0034] This object detection system, based on the Linux kernel V4L2 driver, interacts with the hardware layer via the Linux kernel V4L2 driver module. It captures raw video data in real-time through a camera at the hardware layer. A lightweight CNN model is embedded in the driver kernel space to directly extract features from the raw video frames, outputting feature vectors. An asynchronous inference mechanism separates the AI ​​inference task from the main video acquisition process. After the V4L2 driver module acquires a video frame, it submits the AI ​​inference task to an asynchronous queue, while the main process continues to acquire the next frame. The inference result of the feature vector is written to an extended field of the V4L2 buffer, and then the user space reads the inference result for image display. This avoids the redundancy of copying frame data from kernel space to user space for processing, enabling video frame data acquisition and preprocessing to be completed directly in kernel space. The inference task is executed through the asynchronous inference mechanism, and the processed image is then passed to user space. This avoids time-consuming inference blocking and the latency and high CPU resource consumption issues caused by AI processing in user space, reducing data transmission latency, decreasing CPU utilization, and improving system stability and multi-tasking capabilities. Meanwhile, this solution is an AI preprocessing method that relies solely on the CPU, avoiding the high cost problem caused by relying on NPU / GPU in existing technologies.

[0035] In some embodiments, the hardware layer includes an image sensor used to generate 1080p YUV420 video frames from raw video frames. The image sensor receives configuration instructions from the kernel space via I2C and controls the transmission of video frame data to the kernel space via a DMA controller. Specifically, the V4L2 driver module acquires raw video data from the camera hardware through a standard interface, ensuring efficient input of the raw data. For example, video data is acquired from a camera (such as an OV5640 sensor) using the VIDIOC_QBUF and VIDIOC_DQBUF interfaces at a resolution of 1080p in YUV420 format. The video data is stored in a kernel buffer and mapped to user space via mmap to reduce memory copying from kernel to user space. The frame rate is set to 20fps by default and can be dynamically adjusted from 10-30fps.

[0036] In some embodiments, the lightweight CNN model is a YOLOv5 model. The feature extraction layer of the YOLOv5 model is used to extract feature vectors, including: allocating memory through kmalloc, loading pre-trained model parameters and performing inference, converting the 1080P YUV420 video frames to RGB format and scaling the resolution to 224*224, performing forward propagation, extracting 128-dimensional feature vectors, and storing the feature vectors in the extended field of the V4L2 buffer. For example, the YOLOv5 feature extraction layer has a parameter size of <1MB and uses INT8 quantization to adapt to embedded devices with memory limitations of less than 4MB. During initialization via the `probe` function, memory is allocated using `kmalloc`, pre-trained model parameters are loaded (e.g., stored as a static array or loaded from the file system via the firmware API), and inference is performed. 1080p YUV420 frames are converted to RGB format, the resolution is scaled to 224*224, forward propagation is performed, and 128-dimensional feature vectors (such as face or vehicle features) are extracted. These feature vectors are stored in an extended field of the V4L2 buffer, such as `v4l2_buffer.reserved`. This method enables real-time preprocessing in kernel mode, reducing video target detection latency and meeting the real-time application requirements of intelligent surveillance, drones, and other applications.

[0037] In some embodiments, the asynchronous reasoning mechanism is kthread or workqueue.

[0038] Furthermore, the asynchronous inference module uses an asynchronous inference mechanism to run the AI ​​inference task of the AI ​​preprocessing model, including: creating an inference thread using kthread_create, setting the scheduling policy to SCHED_FIFO, setting its priority to be higher than that of ordinary tasks, writing the inference result to the extended field of the V4L2 buffer, and then the user space reads the inference result through VIDIOC_DQBUF. This ensures real-time performance and system stability. Optionally, the inference task takes less than 10ms per frame, and the inference frequency ranges from 5 to 10fps.

[0039] In some embodiments, the kernel space includes an error handling module for detecting and handling inference anomalies, including: real-time monitoring of camera I2C communication errors, asynchronously calling the `reset_camera` function via a workqueue to detect inference anomalies, and controlling the error handling recovery time to be less than 300ms. For example, monitoring camera I2C communication errors (such as `VIDIOC_DQBUF` returning -EIO), asynchronously calling the `reset_camera` function via a workqueue to detect inference anomalies, such as invalid feature vectors, and controlling the error handling recovery time to within 300ms via ` / sys / class / v4l2 / ai_params / camera_error_log`. This design, by integrating error detection and asynchronous recovery mechanisms, solves the problem of long hardware fault recovery times (>1s), improving the continuity and reliability of the intelligent monitoring system.

[0040] In some embodiments, the user space includes an application layer and a v4l2-ctl tool module, wherein the application layer is used to read inference results for image matching and image display.

[0041] The v4l2-ctl tool module creates a parameter directory through the sysfs interface to configure the model parameters of the AI ​​preprocessing model. The parameter directory includes at least the model enable flag, feature vector dimension, and object detection confidence threshold. The v4l2-ctl tool module dynamically adjusts the model parameters of the AI ​​preprocessing model through the sysfs interface and synchronizes changes in model parameters in the kernel space to the user space via kobject and uevent. Specifically, exposing AI model parameters through the sysfs interface supports dynamic adjustment in user space, enhancing flexibility. For example, the / sys / class / v4l2 / ai_params parameter directory is created, containing: model_enable: 0 (disable AI preprocessing) or 1 (enable), default 1; feature_dim (feature vector dimension, 64-256, default 128); and confidence_threshold (object detection confidence threshold, 0.5-0.9, default 0.7). kobject and uevent are used to notify of parameter changes, ensuring synchronization in user space. Simultaneously, it addresses the lack of dynamic parameter adjustment capabilities in existing technologies, supporting different application scenarios (such as face detection, vehicle recognition, and liveness detection) through parameter configuration, thereby enhancing the system's application flexibility, versatility, and market competitiveness. Furthermore, it avoids the complexity of adapting AI models to different hardware by providing dynamic configuration functions (such as feature dimensions and confidence thresholds) through the sysfs interface, supporting mainstream sensors (such as OV5640 and IMX219) and platforms (ARM and RISC-V), simplifying the hardware adaptation process and shortening the development cycle.

[0042] In some embodiments, the kernel space includes a status monitoring module for setting load thresholds and sleep thresholds, monitoring and acquiring CPU load and video stream idle time in real time, reducing the inference frequency of the AI ​​inference task in response to the CPU load exceeding the load threshold, and adjusting the sleep mode of the AI ​​inference task in response to the video stream idle time exceeding the sleep threshold. The load threshold is 80%, and the sleep threshold is 100ms. For example, the status monitoring module acquires the CPU load through the cpufreq interface; if the CPU load exceeds 80%, the inference frequency is reduced. The status monitoring module uses jiffies and timer to detect the video stream space; if the video stream is idle for more than 100ms without data frames, the AI ​​inference task is paused, and the pm_runtime_put function is called to enter low-power mode, notifying the user of the space status change via VIDIOC_SUBSCRIBE_EVENT.

[0043] This design addresses the issue of high power consumption (>50mW) caused by user space target detection processing. By using dynamic inference frequency adjustment and idle sleep mechanisms, standby power consumption is reduced to below 20mW, thus reducing operating power consumption and extending the battery life of battery-powered devices.

[0044] To more clearly illustrate the target detection system based on the Linux kernel V4L2 driver in this application, such as Figure 3 The diagram shown is a flowchart of the feature extraction process for this system, which includes the following steps:

[0045] Step a: Initialize the V4L2 driver module and AI preprocessing model: Configure the parameters of the image sensor OV5640 / IMX219 via I2C, load the AI ​​preprocessing model YOLOV5 (kmalloc), create the inference thread kthread / workqueue, and initialize sysfs.

[0046] Step b: Acquire video frames: The V4L2 driver module sends the acquisition control command VIDIOC_QBUF / DQBUF to the hardware layer to acquire 1080p / 720p YUV420 video frames and transfers the frame data to the kernel buffer via DMA.

[0047] Step c: AI preprocessing and feature extraction: Convert the frame format, run the AI ​​preprocessing model YOLOv5, and output the feature vector.

[0048] Step d: Asynchronous inference scheduling: Tasks are scheduled via kthread / workqueue to ensure real-time performance.

[0049] Step e: Dynamic configuration and monitoring: Dynamically adjust the inference frequency (5-10fps), detect idle time (>100ms); implement NEON accelerated inference for the CPU, and control the power consumption status of the power management unit.

[0050] Step f: Error handling: Detect I2C errors, asynchronously reset hardware, and log the error.

[0051] More specifically, such as Figure 4 The diagram shown is an asynchronous inference timing diagram of the target detection system in this application. Key timing parameters and functions are as follows:

[0052] a) t0: User space configures parameters via sysfs (feature_dim=128).

[0053] b) t1-t2: V4L2 driver initializes OV5640, sets the format and startup stream.

[0054] c) t3: Driver acquires frame data (DMA transfer).

[0055] d) t4-t6: Drive the submission of inference tasks, run YOLOv5 in kthread / workqueue, and write feature vectors.

[0056] e)t7: Read feature vectors from user space.

[0057] f) t8-t9: Monitor load and idle time, adjust frequency or hibernate.

[0058] g) t10-t12: Detect I2C errors, asynchronously reset hardware, and log.

[0059] h) Program-controlled hardware:

[0060] I2C: t1-t2 configure the sensor, t11 reset the hardware.

[0061] DMA: t3 transfer frame.

[0062] CPU: t5 performs NEON inference.

[0063] PMU: t9 controls hibernation.

[0064] The implementation steps corresponding to the asynchronous inference timing are as follows:

[0065] 1): t1-t3 (initialization and acquisition).

[0066] 2): t4-t6 (AI preprocessing and asynchronous inference).

[0067] 3): t0, t8-t9 (configuration and monitoring).

[0068] 4): t10-t12 (error handling).

[0069] In summary, the target detection system based on the Linux kernel V4L2 driver proposed in this application, with kernel-mode AI preprocessing as its core, combined with asynchronous inference, no hardware dependency, dynamic configuration and error handling, solves the performance, cost and power consumption problems of traditional solutions.

[0070] Based on the same inventive concept, this application also provides a target detection method based on the Linux kernel V4L2 driver. The method is implemented based on a target detection system based on the Linux kernel V4L2 driver described in any of the above embodiments, and the method includes at least:

[0071] Step 1: Initialize the V4L2 driver module and AI preprocessing model in the kernel space;

[0072] Step 2: Based on the V4L2 driver module, obtain the original video frames, use the lightweight CNN model as the AI ​​preprocessing model, and extract and directly output the feature vectors;

[0073] Step 3: Run the AI ​​inference task of the AI ​​preprocessing model using the asynchronous inference mechanism, determine the inference result of the feature vector, and write it into the extended field of the V4L2 buffer;

[0074] Step 4: The user space reads the inference result for image display.

[0075] The implementation scheme of this method is similar to the implementation scheme described in the above system. Therefore, for specific limitations, please refer to the limitations of the target detection system based on the Linux kernel V4L2 driver mentioned above, which will not be repeated here.

[0076] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It should be noted that for those skilled in the art, several improvements and modifications can be made to this application without departing from the principles of this application, and these improvements and modifications also fall within the scope of protection of the claims of this application. It should be understood that certain features of this disclosure described in the context of individual embodiments for clarity can also be provided in combination in a single embodiment. Conversely, various features of this disclosure described in the context of individual embodiments for clarity can also be provided individually or in any suitable combination or as part of any other described embodiment of this disclosure.

Claims

1. A target detection system based on Linux kernel V4L2 driver, characterized in that, It includes a hardware layer, a kernel space that manages the hardware layer to acquire raw video data and perform object detection preprocessing, and a user space for image display; wherein, the kernel space includes: The V4L2 driver module acquires raw video frames, uses a lightweight CNN model as an AI preprocessing model, and extracts and directly outputs feature vectors. The asynchronous inference module uses an asynchronous inference mechanism to run the AI ​​inference task of the AI ​​preprocessing model, determines the inference result of the feature vector, and writes it into the extended field of the V4L2 buffer; the user space reads the inference result for image display.

2. The target detection system based on Linux kernel V4L2 driver according to claim 1, characterized in that, The hardware layer includes an image sensor, which is used to generate 1080P YUV420 video frames from the original video frames, receive configuration instructions from the kernel space via I2C, and control the transmission of video frame data to the kernel space via a DMA controller.

3. The target detection system based on Linux kernel V4L2 driver according to claim 2, characterized in that, The lightweight CNN model is a YOLOv5 model. Feature vectors are extracted using the feature extraction layer of the YOLOv5 model, including: Memory is allocated using kmalloc, pre-trained model parameters are loaded and inference is performed. The 1080P YUV420 video frames are converted to RGB format and the resolution is scaled to 224*224. Forward propagation is performed to extract 128-dimensional feature vectors and store the feature vectors in the extended field of the V4L2 buffer.

4. The target detection system based on Linux kernel V4L2 driver according to claim 1, characterized in that, The asynchronous reasoning mechanism is kthread or workqueue.

5. A target detection system based on Linux kernel V4L2 driver according to claim 4, characterized in that, The asynchronous inference module uses an asynchronous inference mechanism to run the AI ​​inference task of the AI ​​preprocessing model, including: creating an inference thread using kthread_create, setting the scheduling policy to SCHED_FIFO, setting the priority to be higher than that of ordinary tasks, and writing the inference result to the extended field of the V4L2 buffer.

6. The target detection system based on Linux kernel V4L2 driver according to claim 4, characterized in that, The kernel space includes an error handling module for detecting and handling inference anomalies, including: real-time monitoring of camera I2C communication errors, asynchronously calling the reset_camera function through the workqueue to detect inference anomalies, and controlling the error handling recovery time to be less than 300ms.

7. A target detection system based on Linux kernel V4L2 driver according to claim 1, characterized in that, The user space includes an application layer and a v4l2-ctl tool module. The application layer is used to read inference results for image matching and image display. The v4l2-ctl tool module creates a parameter directory through the sysfs interface to configure the model parameters of the AI ​​preprocessing model. The parameter directory includes at least the model enable flag, feature vector dimension, and object detection confidence threshold. The v4l2-ctl tool module dynamically adjusts the model parameters of the AI ​​preprocessing model through the sysfs interface, and synchronizes the changes in the model parameters in the kernel space to the user space through kobject and uevent.

8. A target detection system based on Linux kernel V4L2 driver according to claim 1, characterized in that, The kernel space includes a status monitoring module for setting load thresholds and sleep thresholds, monitoring and acquiring CPU load and video stream idle time in real time, reducing the inference frequency of the AI ​​inference task in response to the CPU load exceeding the load threshold, and adjusting the sleep mode of the AI ​​inference task in response to the video stream idle time exceeding the sleep threshold.

9. A target detection system based on Linux kernel V4L2 driver according to claim 8, characterized in that, The inference task takes less than 10ms per frame, and the inference frequency ranges from 5 to 10fps.

10. A target detection method based on the Linux kernel V4L2 driver, characterized in that, Based on the target detection system based on the Linux kernel V4L2 driver as described in any one of claims 1-9, the method includes: Initialize the V4L2 driver module and AI preprocessing model in the kernel space; The original video frames are acquired based on the V4L2 driver module. A lightweight CNN model is used as an AI preprocessing model to extract and directly output feature vectors. An asynchronous inference mechanism is used to run the AI ​​inference task of the AI ​​preprocessing model to determine the inference result of the feature vectors and write it into the extended field of the V4L2 buffer. The user space reads the inference result for image display.

Images