Optimization method for improving real-time image data input processing and output efficiency

Abstract

The invention relates to an optimization method for improving the input processing and output efficiency of real-time image data, belongs to the technical field of embedded systems, and is particularly suitable for optimizing the input processing and output efficiency of the real-time image data in a resource-constrained environment. According to the method, parallel execution of image data processing tasks is realized by starting a multi-thread programming model; a hardware acceleration unit arranged in the embedded processor is used for carrying out batch processing on the image data, and the format conversion speed of the image data is increased; and direct access and transmission of the image data are realized by using a memory mapping technology and a DMA controller, so that the copying times of the image data in a memory are reduced, and the data transmission delay is reduced. Specific implementation details of how to realize a multi-thread synchronization mechanism, a hardware acceleration function and a zero copy technology are described, and efficient and real-time image data processing is ensured to be realized under the condition that resources are limited.

Description

Technical Field

[0001] The invention belongs to the technical field of embedded systems, and in particular relates to an optimization method for improving the input processing and output efficiency of real-time image data under a resource-constrained environment. Background Art

[0002] With the development of information technology, embedded systems have been widely used in aerospace and other fields due to their small size, low power consumption, and high reliability. However, in a resource-constrained environment, how to efficiently complete image data processing and transmission has become a key issue. Traditional image data processing methods perform poorly in real-time, especially when processing image data, due to multiple data copies and serial format conversion, the processing efficiency is low. Therefore, it is of great practical significance to study and propose an efficient image data processing method suitable for embedded platforms. Summary of the invention

[0003] In order to solve the above technical problems, the present invention provides an optimization method for improving the efficiency of real-time image data input processing and output, which can improve the efficiency of image data processing and transmission under resource-constrained conditions, and is particularly suitable for image data processing in embedded platforms. By introducing multi-threaded operation, hardware acceleration and zero-copy technology, it can meet the needs of high real-time image data processing.

[0004] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0005] 1. Multithreaded operation:

[0006] • Enable the multi-threaded programming model on the embedded platform, write multi-threaded applications corresponding to data reception, data processing and data output, define the communication protocol and data sharing mechanism between threads, and improve the speed of image data processing through parallel processing.

[0007] •Design and implement thread synchronization mechanism based on mutex locks and condition variables to ensure data consistency in multi-threaded environment.

[0008] • The specific steps include: a) Create multiple threads, each thread is responsible for processing different parts of the image data; b) When threads need to access shared resources, use mutex locks to prevent data competition; c) Use condition variables to synchronize threads and ensure orderly execution of threads; d) Define thread synchronization strategies, including initialization, destruction, locking, unlocking and other operations;

[0009] 2. Hardware acceleration: If image processing needs to be accelerated, use the built-in hardware acceleration unit of the embedded platform processor to write batch pixel processing functions related to image data format conversion and copy access;

[0010] • Use the built-in hardware acceleration technology of modern processors to speed up the conversion of image data formats.

[0011] •Especially during the image data format conversion process, efficient data loading and storage can be achieved through hardware acceleration technology.

[0012] •The specific steps include: a) Splitting complex operations into single instruction multiple data stream operations according to different application scenarios, calling different NEON instructions related to bit operations, logical operations and data access; b) During the image data format conversion process, use the hardware acceleration instruction set to batch load and store data.

[0013] 3. Zero-copy technology: If image data needs to be updated, the image data can be directly accessed and transferred without additional consistency checks based on memory mapping technology and DMA controller;

[0014] • Use zero-copy technology to reduce the number of times image data is copied in memory, thereby reducing data transmission delays.

[0015] •Use DMA technology to directly read and write data.

[0016] •The specific steps include: a) Using memory mapping technology to achieve direct access to data and reduce unnecessary data copying; b) Using DMA technology to achieve direct reading and writing of image data from external devices to memory; c) Implementing dynamic memory consistency mapping based on kmalloc and dma_map_single and serial mapping of fixed-size DMA buffers based on dma_alloc_coherent respectively, and the latter can further reduce the delay of data transmission by reducing one manual consistency check.

[0017] The beneficial effects of the present invention are:

[0018] The present invention realizes synchronization between threads through multi-threaded operation to ensure orderly execution between threads; improves the speed of image data format conversion through hardware acceleration level; realizes direct access to data through zero copy technology to reduce data transmission delay. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A flow chart of an optimization method for improving the efficiency of real-time image data input processing and output according to the present invention;

[0020] Figure 2 This is a practical workflow diagram of the present invention when performing image data transmission and processing tasks. DETAILED DESCRIPTION

[0021] In order to facilitate the understanding of those skilled in the art, the present invention is further described below in conjunction with embodiments and drawings. The contents mentioned in the implementation modes are not intended to limit the present invention.

[0022] like Figure 1 As shown, the present invention provides an optimization method for improving the efficiency of real-time image data input processing and output. After the system is started, sensors and other peripherals start to collect image data; threads are created on demand, and mutex locks and conditional variables are used to achieve thread synchronization to ensure the consistency and integrity of data in a multi-threaded environment; when receiving data, if the platform supports DMA, a driver framework is built, and zero-copy technology is used to improve transmission efficiency; if it is not supported, the traditional method is used; when processing image data, the present invention decomposes complex operations into simple steps, and uses NEON acceleration (if supported by the platform) to improve processing speed; an intelligent reading strategy is implemented in the read cache thread, data readability is determined according to the write cache thread synchronization identifier, and data is quickly read out to the disk to ensure real-time and accurate data; during the entire recording process, the write cache thread and the read cache thread run in parallel until a termination signal is received.

[0023] like Figure 2 As shown, the steps of the real-time image processing system when performing transmission and processing tasks are as follows:

[0024] Step 1: Prepare and select the hardware platform. When the system starts, sensors and other peripherals begin to collect and input image data. According to the functional performance requirements, select an embedded platform with a high-performance multi-core processor, large-capacity RAM, high-speed storage devices, and support for MMU (memory management unit) and DMA (Direct Memory Access) technology as the development platform. Ensure that the selected hardware platform has certain processing power, sufficient memory space, and rich peripheral interfaces, especially support for DMA and memory mapping functions, to lay the foundation for the subsequent implementation of zero-copy technology;

[0025] Step 2: System configuration and testing: Reconfigure and solidify the Linux operating system on the selected hardware platform. Test the peripheral drivers on the platform to confirm that they function properly, especially the driver support for high-speed peripherals such as PCIE. At the same time, ensure that DMA and memory mapping support are enabled in the kernel configuration. Work closely with the project leader and technical support team to ensure the feasibility of functional requirements and perform necessary system optimization according to requirements;

[0026] Step 3: Implement multi-threaded programming and synchronization mechanism. In user space, create threads according to requirements, and create and start threads through the pthread_create() function. For example, two threads, A_handler and PCIE_handler, can be created to be responsible for image processing tasks in user space and data transmission tasks of PCIE peripherals respectively. Use synchronization mechanisms such as semaphores to build an environment suitable for multi-threaded operation, and improve the relevant architecture from thread creation to destruction, including using the pthread_join() function to wait for the end of the thread. At the same time, implement a thread synchronization mechanism based on relevant mutexes and conditional variables such as image resources and CPU resources to ensure data consistency and integrity. The corresponding conditional variable operations include waiting for the condition to be met (pthread_cond_wait()), notifying a thread that the condition has been met (pthread_cond_signal()), and notifying all waiting threads that the condition has been met (pthread_cond_broadcast()). Define clear communication protocols and data sharing mechanisms between threads to ensure the orderly execution of tasks.

[0027] Step 4: Hardware acceleration and image processing optimization, determine whether the platform supports NEON acceleration technology. If supported, use the NEON instruction set to optimize the image processing algorithm, such as dividing the pixel format modification operation of the image data into multiple bit operations, and using vld1q_u16, vandq_u16, vorrq_u16 and other instructions for efficient bit operations;

[0028] Step 5. Implement zero-copy technology and optimize data transmission, build a PCIE driver framework, and select the appropriate memory mapping method according to needs, such as using kmalloc and dma_map_single for dynamic memory allocation and mapping, or using dma_alloc_coherent to apply for consistent memory and map it. By calling the corresponding ioctl and mmap functions of the driver, direct mapping and access of image data from peripherals to memory is achieved, thereby reducing the intermediate copy links and improving data transmission efficiency. During the data transmission process, the DMA controller is used to directly read and write image data. Ensure consistency checks before and after data transmission, such as using dma_sync_single_for_cpu and dma_sync_single_for_device functions to ensure the correctness and integrity of the data;

[0029] Step 6: System termination and resource release: After receiving the termination signal, the system terminates all threads and processes in an orderly manner and releases related resources, such as memory mapping, DMA channels, etc., to ensure that the system can exit cleanly and safely.

[0030] The above embodiments are preferred implementation schemes of the present invention. In addition, the present invention may also be implemented in other ways. Any obvious replacement without departing from the concept of the present technical solution is within the protection scope of the present invention.

[0031] In order to make it easier for ordinary technicians in the field to understand the improvements of the present invention over the prior art, some drawings and descriptions of the present invention have been simplified, and some parts of the common technologies known to those skilled in the art have not been elaborated in detail. For the sake of clarity, some other elements are omitted in this application document. Ordinary technicians in the field should realize that these omitted elements may also constitute the content of the present invention.

[0032] Those skilled in the art should recognize that the above embodiments are only used to illustrate the present invention and are not intended to limit the present invention. As long as they are within the spirit of the present invention, any changes or modifications to the above embodiments will fall within the scope of the claims of the present invention.

Claims

1. An optimization method for improving the efficiency of real-time image data input processing and output, characterized in that: The method comprises the following steps: Enable the multi-threaded programming model by calling the kernel API on the embedded platform, write multi-threaded applications corresponding to data reception, data processing and data output, and define the communication protocol and data sharing mechanism between threads; If image processing acceleration is required, use the built-in hardware acceleration unit of the embedded platform processor to write batch pixel processing functions related to image data format conversion and copy access; If the image data needs to be updated, the image data can be directly accessed and transmitted without additional consistency checks based on memory mapping technology and DMA controller.

2. The optimization method for improving the efficiency of real-time image data input processing and output as claimed in claim 1, characterized in that: The inter-thread communication protocol and data sharing mechanism include a thread synchronization mechanism based on mutex locks and condition variables: a. Create multiple threads, each thread is responsible for processing different parts of the image data; b. Use mutex locks to protect shared resources in a multi-threaded environment to prevent data inconsistency caused by multiple threads accessing the same resource at the same time; c. Use condition variables to achieve synchronization between threads; d. Define thread synchronization strategies, including initialization, destruction, locking, and unlocking.

3. The optimization method for improving the efficiency of real-time image data input processing and output as claimed in claim 1, characterized in that: The batch pixel processing functions related to the image data format conversion and copy access are used to perform batch loading and storage of data through an instruction set.

4. The optimization method for improving the efficiency of real-time image data input processing and output as claimed in claim 3, characterized in that: The batch loading and storing of data by the instruction set includes: a. Split complex operations into single instruction multiple data stream operations for different application scenarios, and call different NEON instructions related to bit operations, logical operations, and data access; b. During the image data format conversion process, use the hardware acceleration instruction set to batch load and store data.

5. The optimization method for improving the efficiency of real-time image data input processing and output as claimed in claim 1, characterized in that: The direct access and transmission of image data without additional consistency check based on memory mapping technology and DMA controller includes: Use memory mapping technology to achieve direct access to data; Use DMA technology to directly read and write image data from external devices to memory; Through the cooperation of memory mapping and DMA technology, the consistent mapping of dynamically allocated memory based on kmalloc and dma_map_single and the serial mapping of fixed-size DMA buffer based on dma_alloc_coherent are realized respectively.

Images