Image algorithm embedding method and device based on MTK Android HAL layer and storage medium

Abstract

The invention relates to an image algorithm embedding method based on an MTK Android HAL layer. The method comprises the following steps that a new display buffer area is configured; mapping the memory of the new display buffer area to obtain a memory address of a mapping original image; adjusting the size and position of the original image content in the mapped memory area, and writing the processed image data into the memory area corresponding to the new display buffer area; submitting the processed display buffer area to display hardware for sending and displaying; and adjusting according to the size of the equipment display area so as to display the display content in the platform area. According to the image algorithm embedding method based on the MTK Android HAL layer, it can be ensured that the image display effects on different display devices are consistent, and the situation that the images exceed the display range or are not displayed completely is avoided.

Description

Technical Field

[0001] The present invention relates to an image algorithm based on the MTK Android HAL layer, and in particular to a method, device, and storage medium for embedding an image algorithm based on the MTK Android HAL layer. Background Art

[0002] The following problems exist in the image display of existing devices based on the Android system:

[0003] Image display consistency problem: On different display devices (such as tablets, reading and writing platforms, etc.), the image display effect is inconsistent, and the image will exceed the display range or not be fully displayed.

[0004] Low display efficiency: The transmission and processing process of image data in memory is less efficient and has a longer delay. Summary of the Invention

[0005] To solve the above problems, the present invention provides a method for embedding an image algorithm based on the MTK Android HAL layer. The specific technical solution is as follows:

[0006] A method for embedding an image algorithm based on the MTK Android HAL layer includes the following steps:

[0007] Step 100: Configure a new display buffer;

[0008] Step 200: Map the memory of the new display buffer to obtain the memory address of the original mapped image;

[0009] Step 300: Adjust the size and position of the original image content in the mapped memory area, and write the processed image data into the corresponding memory area of the new display buffer;

[0010] Step 400: Submit the processed display buffer to the display hardware for display;

[0011] Step 500: Adjust according to the size of the device display area so that the display content is displayed in the platform area.

[0012] Preferably, when configuring the new display buffer, find the code position for sending the original image in the hwcomposer layer and modify it at this position. When modifying: first create a DisplayBufferQueue queue to manage the display buffer; then, through the queue-dequeueBuffer method in the queue, apply for a new display buffer. The configuration of the new display buffer includes the width, height, and memory space (image format) of the image, and the configuration is consistent with the original Figure 1 one.

[0013] Preferably, the step 200 includes the following steps: First, use mmap to map the memory of the new display buffer to obtain its memory address dest_addr; then, in the display thread OverlayEngine::threadLoop of each frame of the original image, obtain the ion_fd of the original image, and map the memory address of the original image through mmap.

[0014] Further, in the step 300, the original image is scaled and offset using an image algorithm, and the image algorithm includes the following steps:

[0015] Step 310: Execute process_image and pass in a structure parameter with the memory address src_addr of the original image and the memory address dest_addr of the new display buffer;

[0016] Step 320: Initialize the image data, obtain the memory addresses of the source image and the target image, the width and height of the source image, and set the scaling ratio;

[0017] Step 330: Calculate the size of the target image, and calculate the width and height of the target image by multiplying the width and height of the source image by the scaling ratio;

[0018] Step 340: Calculate the horizontal offset and vertical of the target image;

[0019] Step 350: Traverse each pixel of the target image;

[0020] Step 360: Calculate the corresponding pixel position in the source image;

[0021] Step 370: Calculate the memory offset;

[0022] Step 380: Copy the pixel value.

[0023] Further, the initialization of the image data in the step 320 includes the following steps:

[0024] Step 321: Initialize OverlayEngine and execute the constructor;

[0025] Step 322: Initialize the image algorithm;

[0026] Step 323: Configure the width, height, and image format of the image.

[0027] Preferably, the step 400 includes the following steps:

[0028] Step 411: OverlayEngine loops to send the display and execute the function threadLoop;

[0029] Step 412: Determine whether the display queue has been created. If it has been created, proceed to Step 413; otherwise, return to Step 411.

[0030] Step 413: Create a display queue and call getDisplayBufferQueue to create a new DisplayBufferQueue object.

[0031] Step 414: Determine whether the new display queue has been configured. If it has been configured, proceed to Step 413; otherwise, return to Step 415.

[0032] Step 415: Create a display buffer, call configDisplayBufferQueue, configure buffer_param, and set width, height, format, and size.

[0033] Step 416: Determine whether the memory of the new display buffer has been mapped. If it has been mapped, proceed to Step 417; otherwise, return to Step 415.

[0034] Step 417: Map the new display buffer, call mmap, and perform memory mapping to dest_addr based on the newly created display buffer handle out_ion_fd.

[0035] Step 418: Map the original image display buffer, call mmap, and perform memory mapping to src_addr based on the original image display buffer handle ion_fd.

[0036] Step 419: Call the function process_image to perform image scaling and offset operations on the address pointers of src_addr in the original image memory and dest_addr in the newly created memory.

[0037] Step 420: Replace the new out_ion_fd with ion_fd and send it to the display position.

[0038] Step 421: Call munmap to release the memory mapped for the original image.

[0039] Step 422: After sending the display, restore the original image's ion_fd to the display fd and receive the next frame of the image.

[0040] An apparatus for an image algorithm embedding method based on the MTK Android HAL layer, the apparatus comprising: a processor, a memory, and a program; the program is stored in the memory, and the processor calls the program stored in the memory to execute the steps of the image algorithm embedding method based on the MTK Android HAL layer.

[0041] A computer-readable storage medium is configured to store a program, and the program is configured to execute the steps of the method for embedding an image algorithm based on the MTK Android HAL layer.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] The method for embedding an image algorithm based on the MTK Android HAL layer provided by the present invention can ensure consistent image display effects on different display devices, and avoid the situation where the image exceeds the display range or is not fully displayed. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flowchart of the present application;

[0045] Figure 2 is a flowchart of image algorithm processing;

[0046] Figure 3 is a flowchart of initializing an image;

[0047] Figure 4 is a flowchart of sending an image for display. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] The present invention will be further described below in conjunction with the accompanying drawings.

[0049] The original image is corrected and displayed on the display screen according to different display casings to meet diverse display requirements and improve the versatility and user experience of image display. Specifically, the image algorithm can be used to precisely adjust the image for different display devices, including resolution, correction of the display area, etc., so that the image display effect better meets the user's needs. It is particularly suitable for tablet devices. When combined with the use of a reading and writing platform (telescope), automatic correction of the display area can be achieved. In this scenario, the image algorithm will adjust the display area of the original image to a suitable size for further zooming operations on the telescope platform, ensuring that the displayed content presents the best effect within a larger display area, thus avoiding the image exceeding the display platform area.

[0050] As Figures 1 to 4 shown, a method for embedding an image algorithm based on the MTK Android HAL layer includes the following steps:

[0051] Step 100, configure a new display buffer;

[0052] In the hwcomposer layer, find the code location for sending the original image for display (such as overlay.cpp), and make modifications at this location. First, create a new DisplayBufferQueue queue to manage the display buffer, and apply for a new display buffer (disp_buffer.out_ion_fd) through the queue-dequeueBuffer method in the queue; the configuration of the new display buffer includes the width, height, and memory space of the image, and these configurations are required to be the same as the original Figure 1 consistent.

[0053] Step 200: Map the memory of the new display buffer to obtain the memory address of the mapped original image;

[0054] Use mmap to map the memory of the newly applied display buffer to obtain its memory address dest_addr. Then, in the display thread OverlayEngine::threadLoop of each frame of the original image, obtain the ion_fd of the original image, and map the memory address (src_addr) of the original image through mmap. Through this mapping relationship, the original image data can be directly processed.

[0055] Step 300: Use image algorithms to scale, offset, and perform other processing on the original image in the mapped memory area, and adjust parameters such as the size and position of the image content. The processed image data will be written into the memory area corresponding to the new display buffer dest_addr.

[0056] The image algorithm includes the following steps:

[0057] Step 310: Execute process_image and pass in a structure parameter with the memory address src_addr of the original image and the memory address dest_addr of the new display buffer;

[0058] Step 320: Initialize the image data, obtain the memory addresses of the source image and the target image, the width and height of the source image, and set the scaling ratio;

[0059] Step 330: Calculate the size of the target image, and calculate the width and height of the target image by multiplying the width and height of the source image by the scaling ratio;

[0060] Step 340: Calculate the horizontal offset and vertical of the target image;

[0061] Step 350: Traverse each pixel of the target image;

[0062] Step 360: Calculate the corresponding pixel position in the source image;

[0063] Step 370: Calculate the memory offset;

[0064] Step 380, copy pixel values.

[0065] The initialization of the image data in the said step 320 includes the following steps:

[0066] Step 321, initialize OverlayEngine and execute the constructor;

[0067] Step 322, initialize the image algorithm;

[0068] Step 323, configure the width, height, and image format of the image.

[0069] Step 400, submit the processed display buffer (i.e., the disp_buffer.out_ion_fd corresponding to dest_addr) to the display hardware for display. Since it is an asynchronous operation, the correction of image display can be efficiently performed to ensure that the performance of the image on different display devices meets expectations.

[0070] Step 400 includes the following steps:

[0071] Step 411, OverlayEngine loops for display and executes the function threadLoop;

[0072] Step 412, check if the display queue has been created. If it has, go to step 413; otherwise, return to step 411;

[0073] Step 413, create a display queue and call getDisplayBufferQueue, new DisplayBufferQueue object;

[0074] Step 414, check if the new display queue has been configured. If it has, go to step 413; otherwise, return to step 415;

[0075] Step 415, create a display buffer, call configDisplayBufferQueue, configure buffer_param, and set width, height, format, and size;

[0076] Step 416, check if the memory of the new display buffer has been mapped. If it has, go to step 417; otherwise, return to step 415;

[0077] Step 417, map the new display buffer, call mmap, and perform memory mapping to dest_addr according to the newly created display buffer handle out_ion_fd;

[0078] Step 418: Map the original image display buffer. Call mmap to map the memory of the original image display buffer handle ion_fd to src_addr according to the original image;

[0079] Step 419: Call the function process_image to perform operations such as image scaling and offset on the address pointers of src_addr in the original image memory and dest_addr in the newly created memory;

[0080] Step 420: Replace the new out_ion_fd with ion_fd and send it to the display position;

[0081] Step 421: Call munmap to release the memory mapped by the original image;

[0082] Step 422: After sending the display, restore the ion_fd of the original image to the display fd and receive the next frame of image.

[0083] Step 500: Adjust according to the size of the device display area to make the display content appear in the platform area.

[0084] It can adapt to different display casings, ensuring that whether it is a traditional display device or a tablet device integrated with a reading and writing platform, the display correction of the image can be correctly processed. Especially when using the reading and writing platform (zoom lens), the algorithm can adjust according to the size of the device display area to prevent the display content from exceeding the platform area.

[0085] In the scenario of using a tablet device in combination with a zoom lens platform, the image algorithm can dynamically adjust the display area, making the size of the display area adapt to the screen size of the device and achieving the best display on the zoom lens platform. In this way, users can obtain a consistent and comfortable visual experience in different usage scenarios.

[0086] By automatically correcting the display area, it can ensure that the image display effects are consistent on different display devices (such as tablets, reading and writing platforms, etc.), avoiding situations where the image exceeds the display range or is not fully displayed.

[0087] Improve display efficiency: Since the asynchronous processing method is adopted, the transmission and processing process of image data in memory is more efficient, which can reduce latency while ensuring image quality and enhance the user experience.

[0088] Adapt to multiple devices and platforms: This algorithm can be compatible with multiple devices and platforms. Especially when a tablet device and a reading and writing platform (zoom lens) are used in combination, it can flexibly adjust the display area to adapt to different display requirements and ensure the optimization of the display effect.

[0089] By utilizing the underlying image processing interface of the MTK Android HAL, the image display area is accurately calibrated to adapt to different display devices.

[0090] Based on the design of the DisplayBufferQueue queue, asynchronous and efficient display buffer processing is achieved, ensuring the smoothness and real-time performance of image display.

[0091] In the application scenario of tablet devices and the read-write platform (remote lens), the display area is innovatively adjusted to an appropriate size to avoid the image display exceeding the platform area and improve the user's visual experience.

[0092] An apparatus for a method of embedding an image algorithm based on the MTK Android HAL layer, the apparatus comprising: a processor, a memory, and a program; the program is stored in the memory, and the processor calls the program stored in the memory to execute the steps of the method of embedding an image algorithm based on the MTK Android HAL layer.

[0093] The memory and the processor are directly or indirectly electrically connected to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines, such as through a bus connection. The memory stores computer execution instructions for implementing the data access control method, including at least one software function module that can be stored in the memory in the form of software or firmware. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory.

[0094] The memory can be, but is not limited to, a random access memory (Random Access Memory, abbreviated as RAM), a read-only memory (Read Only Memory, abbreviated as ROM), a programmable read-only memory (Programmable Read-Only Memory, abbreviated as PROM), an erasable programmable read-only memory (Erasable Programmable Read-Only Memory, abbreviated as EPROM), an electrically erasable programmable read-only memory (Electric Erasable Programmable Read-Only Memory, abbreviated as EEPROM), etc. Among them, the memory is used to store the program, and the processor executes the program after receiving the execution instruction.

[0095] A processor may be an integrated circuit chip with the ability to process signals. The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0096] A computer-readable storage medium is configured to store a program, and the program is configured to execute the steps of the method for embedding an image algorithm based on the MTK Android HAL layer.

[0097] Embodiments of the present invention are described with reference to the flowcharts of methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing terminal devices to generate a machine, such that the instructions executed by the processors of the computer or other programmable data processing terminal devices generate a device for implementing the functions specified in the flowcharts and / or.

[0098] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that implements the functions specified in the flowchart.

[0099] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, such that a series of operation steps are executed on the computer or other programmable terminal device to generate computer-implemented processing, so that the instructions executed on the computer or other programmable terminal device provide the steps for implementing the functions specified in the flowchart.

[0100] The technical principles of the present invention have been described above in conjunction with specific embodiments. These descriptions are only for explaining the principles of the present invention and cannot be construed in any way as a limitation on the protection scope of the present invention. Based on the explanations herein, those skilled in the art can readily conceive of other specific embodiments of the present invention without creative efforts, and these embodiments will fall within the protection scope of the claims of the present invention.

Claims

1. A method for embedding an image algorithm based on the MTK Android HAL layer, characterized in that It includes the following steps: Step 100, configure a new display buffer; Step 200, map the memory of the new display buffer to obtain the memory address of the original mapped image; Step 300, adjust the size and position of the original image content in the mapped memory area, and write the processed image data into the corresponding memory area of the new display buffer; Step 400, submit the processed display buffer to the display hardware for display; Step 500, adjust according to the size of the device display area so that the display content is displayed in the platform area.

2. The method for embedding an image algorithm based on the MTK Android HAL layer according to claim 1, wherein when configuring the new display buffer, find the code position for sending the original image in the hwcomposer layer and modify it at this position. When modifying: first create a new DisplayBufferQueue queue to manage the display buffer; then apply for a new display buffer through the queue-dequeueBuffer method in the queue. The configuration of the new display buffer includes the width, height, and memory space of the image, and the configuration is the same as that of the original image.

3. The method for embedding an image algorithm based on the MTK Android HAL layer according to claim 1, wherein Step 200 includes the following steps: first use mmap to map the memory of the new display buffer to obtain its memory address dest_addr; then obtain the ion_fd of the original image in the sending thread OverlayEngine::threadLoop of each frame of the original image, and map the memory address of the original image through mmap.

4. The method for embedding an image algorithm based on the MTK Android HAL layer according to claim 3, wherein in Step 300, the original image is scaled and offset using an image algorithm. The image algorithm includes the following steps: Step 310, execute process_image and pass in a structure parameter with the memory address src_addr of the original image and the memory address dest_addr of the new display buffer; Step 320, initialize the image data, obtain the memory addresses of the source image and the target image, the width and height of the source image, and set the scaling ratio; Step 330, calculate the size of the target image, and calculate the width and height of the target image by multiplying the width and height of the source image by the scaling ratio; Step 340, calculate the horizontal offset and vertical of the target image; Step 350, traverse each pixel of the target image; Step 360, calculate the corresponding pixel position in the source image; Step 370, calculate the memory offset; Step 380, copy the pixel value.

5. The method for embedding an image algorithm based on the MTK Android HAL layer according to claim 4, wherein the initialization of the image data in Step 320 includes the following steps: Step 321, initialize OverlayEngine and execute the constructor; Step 322, initialize the image algorithm; Step 323, configure the image width, height, and image format.

6. The method for embedding an image algorithm based on the MTK Android HAL layer according to claim 1, wherein: The step 400 includes the following steps: Step 411, the OverlayEngine sends the display in a loop and executes the function threadLoop; Step 412, determine whether the display queue has been created. If it has been created, go to step 413; otherwise, return to step 411; Step 413, create a display queue, call getDisplayBufferQueue, and create a new DisplayBufferQueue object; Step 414, determine whether the new display queue has been configured. If it has been configured, go to step 413; otherwise, return to step 415; Step 415, create a display buffer, call configDisplayBufferQueue, configure buffer_param, and set width, height, format, and size; Step 416, determine whether the memory of the new display buffer has been mapped. If it has been mapped, go to step 417; otherwise, return to step 415; Step 417, map the new display buffer, call mmap, and perform memory mapping to dest_addr according to the handle out_ion_fd of the newly created display buffer; Step 418, map the original image display buffer, call mmap, and perform memory mapping to src_addr according to the handle ion_fd of the original image display buffer; Step 419, call the function process_image to perform image scaling and offset operations on the address pointers of src_addr in the original image memory and dest_addr in the newly created memory; Step 420, replace the new out_ion_fd with ion_fd at the display position; Step 421, call munmap to release the memory mapped by the original image; Step 422, after sending the display, restore the ion_fd of the original image to the display fd and receive the next frame of the image.

7. An apparatus for an image algorithm embedding method based on the MTK Android HAL layer, characterized in that The device includes: A processor, a memory, and a program; The program is stored in the memory, and the processor calls the program stored in the memory to execute the steps of the method for embedding an image algorithm based on the MTK Android HAL layer according to claim 1.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium is configured to store a program, and the program is configured to execute the steps of the method for embedding an image algorithm based on the MTK Android HAL layer according to claim 1.

Images