Method for reducing driver memory copy in data transmission process and device driver
By creating a memory pool on the embedded Linux platform and modifying the positioning module of the video device driver, directly storing the data to the target location, the performance overhead caused by memory copy in the V4L2 driver is solved, and efficient data transmission and system performance improvement are achieved.
Patent Information
- Application Number
- CN202411477492.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-22
- Publication Date
- 2025-02-28
AI Technical Summary
On embedded Linux platform, in image processing applications, the buffer designed by the V4L2 driver cannot be directly used for data processing within the application, resulting in memory copying, which increases the system's performance overhead.
By creating a memory pool at the application layer and modifying the positioning module in the video device driver, positioning and physical address acquisition are performed every time the buffer physical address is obtained, so that data is stored directly at the target location and data copying is avoided.
It effectively reduces unnecessary driver memory copies, optimizes data processing flow, improves data transmission efficiency and system performance, and is especially suitable for application scenarios with extremely high requirements for real-time performance.
Smart Images
Figure CN119536636A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of software storage, and in particular to a method for reducing driver memory copying during data transmission and a device driver. Background Art
[0002] In the field of image processing on embedded Linux platforms, V4L2 (Video for Linux 2) or similar drivers are widely used for data access of image sensors. The reason behind this choice is that V4L2 provides a powerful and flexible framework for efficient transmission of image data. Specifically, the V4L2 driver works by creating a queue consisting of multiple buffers for each connected device. These buffers are designed for recycling. When a buffer is filled, it is marked as "ready" and waits for the application to read its contents. Once the data is read, the buffer is re-queued and ready to receive new image data again. This approach not only ensures the continuity of the data stream, but also improves the efficiency of data transmission.
[0003] However, although V4L2 and similar drivers are designed with efficiency in mind, there are still some challenges in practical applications. One notable problem is that since these buffers are designed with a specific role as temporary storage areas for data transmission, they are usually not directly used for data processing within the application. This means that in order to ensure data consistency and integrity, the data in the buffer must be copied to the memory space pre-allocated by the application. This process is called memory copy, and although it seems simple, it is a performance overhead that cannot be ignored for embedded systems with limited resources. Memory copy operations not only take up valuable processor time, but can also cause the overall throughput of the system to decrease, especially when processing high-resolution or high-speed image data. This effect is particularly evident.
[0004] There are currently many optimization strategies for the above problems, such as ROS2's zero-copy technology, which attempts to improve data structures to reduce the number of copies, and directly use the buffer in V4L2 temporarily in the application. However, under the current technical framework, completely eliminating memory copies when receiving data is still a very challenging goal. Especially in those application scenarios with extremely high real-time requirements, how to balance data transmission efficiency and system performance has become an important issue that developers must face.
[0005] In summary, since the V4L2 driver is a universal driver framework, simply reducing the number of copies from the application layer cannot affect the data copying from V4L2; and directly using the buffer of the V4L2 driver in the application is considered unsafe in actual applications due to stability issues caused by life cycle limitations. Therefore, for image processing applications on embedded Linux platforms, optimizing the data processing process, especially reducing unnecessary memory copies, is still an important direction to improve system performance. Summary of the invention
[0006] In view of this, an embodiment of the present application provides a method and a device driver for reducing driver memory copies during data transmission, so as to reduce unnecessary driver memory copies and improve data transmission efficiency and system performance.
[0007] The embodiment of the present application provides the following technical solution: a method for reducing driver memory copies during data transmission, applied to the Ubuntu platform, comprising:
[0008] Creating a memory pool at the application layer of the video device driver for data transfer with the video device driver, wherein the memory pool is provided with a plurality of memory blocks that can be transferred to the video device driver;
[0009] Initializing the video device driver, obtaining a plurality of the memory blocks from the memory pool of the application layer, and listing the memory blocks as a plurality of buffers in a buffer queue of the video device driver, wherein the video device driver obtains a target buffer from the buffer queue and obtains a physical address of the buffer, and according to the obtained physical address of the buffer, saves the corresponding image data to the buffer;
[0010] Obtaining the buffer after image data is saved through the application layer of the video device driver, and passing the buffer to the application process for use;
[0011] After the application process is finished using, the buffer is returned to the memory pool as the memory block.
[0012] According to an embodiment of the present application, it also includes: modifying the positioning module in the video device driver for obtaining the physical address of the current buffer zone, so that the positioning module can locate the buffer zone and obtain the physical address of the buffer zone each time the video device driver obtains the physical address of the buffer zone.
[0013] According to an embodiment of the present application, each of the memory blocks in the memory pool is provided with a usage mark, and the usage mark indicates the current usage status of the memory block. After initializing the video device driver, a number of the memory blocks are obtained from the memory pool of the application layer according to the usage marks of the memory blocks; wherein the usage status includes: the memory block is currently in the device driver, the memory block is currently in the application, and the memory block is currently idle.
[0014] According to an embodiment of the present application, the video device driver obtains a target buffer from the buffer queue and obtains a physical address of the buffer, including:
[0015] The video device driver obtains the target buffer from the buffer queue, and obtains the physical address of the buffer through the configuration information of the buffer; the configuration information of the buffer includes the use mark, physical address and space size of the buffer.
[0016] According to an embodiment of the present application, obtaining a plurality of the memory blocks from the memory pool of the application layer further includes: updating the usage marks of the memory blocks in the memory pool.
[0017] According to an embodiment of the present application, it also includes: after the video device driver saves the corresponding image data to the target buffer, the buffer is placed in an outbound data queue, and the outbound data queue and the corresponding buffer index are simultaneously passed to the application layer.
[0018] According to an embodiment of the present application, the buffer index corresponds to the physical address of the buffer in the data output queue.
[0019] According to an embodiment of the present application, the video device driver is a V4L2 device driver.
[0020] The present application also provides a device driver, which executes the method for reducing driver memory copying in the data transmission process as described above when transmitting video data.
[0021] Compared with the prior art, the beneficial effects that can be achieved by at least one of the above-mentioned technical solutions adopted in the embodiments of this specification include at least the following: The embodiments of the present invention are limited to use on the Ubuntu platform of the self-developed Jingzhe Arch64 chip, and are implemented by modifying the video device driver, such as the V4L2 device driver. The embodiments of the present invention modify the meaning of the traditional buffer, so that each time the V4L2 device driver obtains the buffer target address, the buffer is located and the physical address is obtained. At the same time, a memory pool is created at the application layer, and memory blocks are obtained from the application pool, and the life cycle of the buffer is extended to the application level, so that the data in the entire process is directly stored in the target location after being produced from the driver, and data copying is successfully avoided in subsequent application processes. The embodiments of the present invention effectively reduce unnecessary driver memory copies, optimize the data processing process, and improve data transmission efficiency and system performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0023] Figure 1 It is a flowchart of a method for reducing driver memory copies during data transmission according to an embodiment of the present invention;
[0024] Figure 2 is a schematic diagram of the overall framework of the video device driver in an embodiment of the present invention;
[0025] Figure 3 It is a schematic diagram of the process of obtaining and using video data in an embodiment of the present invention. DETAILED DESCRIPTION
[0026] The embodiments of the present application are described in detail below with reference to the accompanying drawings.
[0027] The following describes the implementation methods of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that, in the absence of conflict, the following embodiments and the features in the embodiments can be combined with each other. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work belong to the scope of protection of the present application.
[0028] like Figure 1 and Figure 2 As shown, an embodiment of the present invention provides a method for reducing driver memory copies during data transmission, which is applied to an Ubuntu platform and includes:
[0029] S101. Creating a memory pool for data transfer with the video device driver at the application layer of the video device driver, wherein the memory pool is provided with a plurality of memory blocks that can be transferred to the video device driver;
[0030] S102. Initialize the video device driver, obtain a plurality of memory blocks from the memory pool of the application layer, and list them as a plurality of buffers in a buffer queue of the video device driver, the video device driver obtains a target buffer from the buffer queue, and obtains a physical address of the buffer, and according to the obtained physical address of the buffer, saves the corresponding image data to the buffer;
[0031] S103. Obtain the buffer after the image data is saved through the application layer of the video device driver, and pass the buffer to the application process for use;
[0032] S104: After the application process is finished, the buffer is returned to the memory pool as the memory block.
[0033] The video device driver in the embodiment of the present invention adopts the V4L2 device driver. V4L2 is the abbreviation of Video for Linux 2, which is the kernel driver for video devices in Linux. The embodiment of the present invention is used on the Ubuntu platform of the self-developed Jingzhe Arch64 chip, and is implemented by transforming the video device driver, such as the V4L2 device driver. The embodiment of the present invention modifies the meaning of the traditional buffer, so that each time the V4L2 device driver obtains the buffer target address, the buffer is located and the physical address is obtained. At the same time, a memory pool is created at the application layer, and memory blocks are obtained from the application pool. The life cycle of the buffer is extended to the application level, so that the data in the entire process is directly stored in the target location after being produced from the driver, and data copying is successfully avoided in subsequent application processes. The embodiment of the present invention effectively reduces unnecessary driver memory copies, optimizes the data processing process, and improves data transmission efficiency and system performance.
[0034] The V4L2 device driver is provided with a positioning model, which is used for the V4L2 device driver to obtain the address of the current data block. In a specific implementation of the embodiment of the present invention, the method further includes: modifying the positioning module used for obtaining the physical address of the current buffer in the video device driver, so that the positioning module locates the buffer and obtains the physical address of the buffer each time the video device driver obtains the physical address of the buffer, so as to directly store the data in the target location, avoiding data copying in subsequent application processes.
[0035] In order to enable the V4L2 device driver to obtain the buffer of available memory from the application layer, in a specific implementation of the embodiment of the present invention, each memory block in the memory pool is provided with a usage mark, and the usage mark is displayed as the current usage status of the memory block. After initializing the video device driver, a plurality of memory blocks are obtained from the memory pool of the application layer according to the usage mark of the memory block; wherein the usage status includes: the memory block is currently in the device driver, the memory block is currently in the application, and the memory block is currently idle.
[0036] In a specific implementation of the embodiment of the present invention, the video device driver obtains the target buffer from the buffer queue and obtains the physical address of the buffer, including: the video device driver obtains the target buffer from the buffer queue and obtains the physical address of the buffer through the configuration information of the buffer; the configuration information of the buffer includes the use mark, physical address and space size of the buffer. The use mark is used as a state judgment of whether the memory block (buffer) is currently in the device driver, in the application, or in idle state, the physical address is the physical address of the memory block that can be used by the device driver to store data, and the space size is the memory size of the memory block.
[0037] In a specific implementation of an embodiment of the present invention, obtaining a plurality of the memory blocks from the memory pool of the application layer also includes: updating the usage mark of the memory block in the memory pool, so as to facilitate subsequent acquisition of a plurality of the memory blocks from the memory pool of the application layer according to the usage mark of the memory block.
[0038] In a specific implementation of the embodiment of the present invention, the method further includes: after the video device driver saves the corresponding image data to the target buffer, the buffer is placed in an outbound data queue, and the outbound data queue and the corresponding buffer index are simultaneously transmitted to the application layer. Furthermore, the buffer index corresponds to the physical address of the buffer in the outbound data queue.
[0039] Next, the present invention is further described with reference to specific examples.
[0040] like Figure 3 As shown, the method for reducing driver memory copies during data transmission in an embodiment of the present invention specifically includes the following steps:
[0041] Step 1, customize and modify the V4L2 device driver implementation, modify the positioning module for obtaining the current data block address in the V4L2 device driver, and perform buffer positioning and physical address acquisition every time the V4L2 device driver obtains the buffer target address.
[0042] Step 2: Establish a memory pool in the application layer that can be used for data transfer by the V4L2 device driver. The memory pool is bound to the corresponding V4L2 device driver. There are several memory blocks in the memory pool that can be transferred to the V4L2 device driver.
[0043] The following describes the process of obtaining and using video data. After the application starts running, the V4L2 device driver is initialized. Here, the maximum capacity of the memory pool is declared, but the V4L2 device driver does not initialize the possible buffers in the queue. Instead, a memory pool that meets the space requirements is created at the application layer. An appropriate number of buffers are obtained from the memory pool and queued into the buffer queue of the V4L2 device driver. At the same time, the usage mark of the buffer in the memory pool is updated. The V4L2 device driver starts working, takes a buffer from the buffer queue, obtains the physical address from the buffer configuration information, and then saves the data to the corresponding location. The buffer is then placed in the data output queue. The application layer obtains the buffer and passes it directly to the application process for use. After use, the buffer is returned to the memory pool.
[0044] In the entire process of the embodiment of the present invention, after the data is produced from the driver, it is directly stored in the target location, successfully avoiding data copying in the subsequent application process, optimizing the data processing process, improving data transmission efficiency and system performance, and is particularly suitable for application scenarios with extremely high real-time requirements.
[0045] The embodiment of the present invention also provides a device driver, which performs the method of reducing driver memory copy in the data transmission process as described above when performing video data transmission. The device driver is used on the Ubuntu platform of the self-developed Jingzhe Arch64 chip, wherein the specific execution process of the device driver includes: (1) creating a memory pool for data transmission with the video device driver in the application layer of the video device driver, wherein the memory pool is provided with a plurality of memory blocks that can be transmitted to the video device driver; (2) initializing the video device driver, obtaining a plurality of the memory blocks from the memory pool of the application layer, and listing them as a plurality of buffers in the buffer queue of the video device driver, wherein the video device driver obtains the target buffer from the buffer queue, and obtains the physical address of the buffer, and saves the corresponding image data to the buffer according to the obtained physical address of the buffer; (3) obtaining the buffer after the image data is saved through the application layer of the video device driver, and passing the buffer to the application process for use; (4) after the application process is used, returning the buffer as the memory block to the memory pool.
[0046] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims
1. A method for reducing driver memory copies during data transmission, applied to the Ubuntu platform, characterized in that: include: Creating a memory pool at the application layer of the video device driver for data transfer with the video device driver, wherein the memory pool is provided with a plurality of memory blocks that can be transferred to the video device driver; Initializing the video device driver, obtaining a plurality of the memory blocks from the memory pool of the application layer, and listing the memory blocks as a plurality of buffers in a buffer queue of the video device driver, wherein the video device driver obtains a target buffer from the buffer queue and obtains a physical address of the buffer, and according to the obtained physical address of the buffer, saves the corresponding image data to the buffer; Obtaining the buffer after image data is saved through the application layer of the video device driver, and passing the buffer to the application process for use; After the application process is finished using, the buffer is returned to the memory pool as the memory block.
2. The method for reducing driver memory copies during data transmission according to claim 1, characterized in that: Also includes: The positioning module in the video device driver for obtaining the physical address of the current buffer is modified so that the positioning module can locate the buffer and obtain the physical address of the buffer each time the video device driver obtains the physical address of the buffer.
3. The method for reducing driver memory copies during data transmission according to claim 1, characterized in that: Each of the memory blocks in the memory pool is provided with a usage mark, and the usage mark indicates the current usage status of the memory block. After initializing the video device driver, a number of the memory blocks are obtained from the memory pool of the application layer according to the usage marks of the memory blocks; wherein the usage status includes: the memory block is currently in the device driver, the memory block is currently in the application, and the memory block is currently idle.
4. The method for reducing driver memory copies during data transmission according to claim 3, characterized in that: The video device driver obtains a target buffer from the buffer queue and obtains a physical address of the buffer, including: The video device driver obtains the target buffer from the buffer queue, and obtains the physical address of the buffer through the configuration information of the buffer; the configuration information of the buffer includes the use mark, physical address and space size of the buffer.
5. The method for reducing driver memory copies during data transmission according to claim 3, characterized in that: Acquiring a plurality of the memory blocks from the memory pool of the application layer also includes: updating the usage marks of the memory blocks in the memory pool.
6. The method for reducing driver memory copies during data transmission according to claim 1, characterized in that: Also includes: After the video device driver saves the corresponding image data to the target buffer, it puts the buffer into an outgoing data queue, and simultaneously transmits the outgoing data queue and the corresponding buffer index to the application layer.
7. The method for reducing driver memory copies during data transmission according to claim 6, characterized in that: The buffer index corresponds to the physical address of the buffer in the outgoing data queue.
8. The method for reducing driver memory copies during data transmission according to claim 1, characterized in that: The video device driver is a V4L2 device driver.
9. A device driver, characterized in that: The device driver executes the method for reducing driver memory copying in the data transmission process as described in any one of claims 1 to 8 when performing video data transmission.