OSD image rendering system and method, chip, and electronic device

Abstract

The present application relates to the technical field of image processing, and provides an OSD image rendering system and method, a chip, and an electronic device. The OSD image rendering system comprises: a processor, a first storage module, a display pipeline module, and a second storage module, wherein the first storage module is used for storing OSD rendering data; the second storage module is used for storing data other than the OSD rendering data; the OSD rendering data comprises: a rendering instruction and a rendering resource required for rendering, and a menu image to be transmitted for display which is obtained by rendering; the processor is used for acquiring the rendering instruction and the rendering resource from the first storage module, rendering the rendering resource on the basis of the rendering instruction to obtain said menu image, and storing said menu image in the first storage module; and the display pipeline module is used for acquiring said menu image from the first storage module, compositing said menu image with a main display image, and then displaying same.

Description

OSD image rendering system, method, chip and electronic device Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, provides an OSD image rendering system, method, chip, and electronic device. Background Technology

[0002] Traditional OSD (On-Screen Display) systems include an MCU (Microcontroller Unit) and DDR (Double Data Rate Synchronous Dynamic Random Access Memory). Because DDR is also used by other hardware besides the OSD system, a DDR buffer wall occurs, causing delays when writing and reading data from DDR. Therefore, in current technology, OSD image rendering data suffers from slow storage and slow retrieval, resulting in low OSD image rendering efficiency. Summary of the Invention

[0003] In view of this, this application aims to provide an OSD image rendering system, method, chip, and electronic device to improve the storage and retrieval speed of OSD image rendering data and improve OSD image rendering efficiency.

[0004] In a first aspect, this application provides an OSD image rendering system, comprising: a processor, a first storage module, a display pipeline module, and a second storage module; the first storage module is used to store OSD rendering data; the second storage module is used to store other data besides the OSD rendering data; the OSD rendering data includes: rendering instructions and rendering resources required for rendering, and a rendered menu image to be displayed; the processor is used to obtain the rendering instructions and the rendering resources from the first storage module, and render the rendering resources according to the rendering instructions to obtain the menu image to be displayed, and store the menu image to be displayed in the first storage module; the display pipeline module is used to obtain the menu image to be displayed from the first storage module, and display the menu image to be displayed after mixing it with a main display image.

[0005] In this embodiment, the first storage module stores OSD rendering data, while the second storage module stores other data besides OSD rendering data. Therefore, when the processor reads and writes OSD rendering data from the first storage module, the first storage module does not perform any other read or write operations besides OSD rendering data. This reduces the number of read and write operations required by the first storage module compared to existing methods of storing and writing OSD rendering data via DDR, thus reducing the occurrence of a cache wall. This improves the efficiency of reading and writing OSD rendering data, thereby improving image rendering efficiency.

[0006] In conjunction with the technical solution provided in the first aspect above, in some possible implementations, the processor includes a GPU (Graphics Processing Unit).

[0007] In this embodiment, the menu image to be displayed is rendered by the GPU. Compared with the existing technology that uses the MCU for rendering, this solution can reduce the software and hardware interaction burden of the system, thereby reducing the occupation of system resources, improving the synergy between software and hardware, and rendering a menu image with better display effect and higher flexibility.

[0008] In conjunction with the technical solution provided in the first aspect above, in some possible implementations, the first storage module includes: a processor memory access interface, a pipeline interface module, a storage unit, and an access arbitrator; the processor memory access interface and the pipeline interface are respectively connected to the storage unit through the access arbitrator; the processor memory access interface is used to communicate with the processor so that the processor can obtain rendering instructions and rendering resources required for rendering from the storage unit through the processor memory access interface; the storage unit is used to store the rendering instructions, the rendering resources, and the menu image to be displayed; the pipeline interface module is used to send the menu image to be displayed to the display pipeline module.

[0009] In this embodiment, communication between the processor and the storage unit is achieved through the processor memory access interface, communication between the display pipeline module and the storage unit is achieved through the pipeline interface module, and an access arbitrator is used to allocate access requests between the processor and the display pipeline module. This enables the first storage module to store and retrieve OSD rendering data throughout the entire rendering process.

[0010] In conjunction with the technical solution provided in the first aspect above, in some possible implementations, the first storage module further includes: an external memory access interface, which is connected to the access arbitrator. The external memory access interface is used to receive the rendering instructions and the rendering resources, and to write the rendering instructions and the rendering resources into the storage unit through the access arbitrator.

[0011] In this embodiment of the application, the rendering instructions and rendering resources in the first storage module can be updated and modified through the external memory access interface.

[0012] In conjunction with the technical solution provided in the first aspect above, in some possible implementations, the pipeline interface module includes: a layer control module and at least one direct memory access module; the layer control module is used to send a read instruction to each of the direct memory access modules, the read instruction including a starting address, a data size, and a display position; each of the direct memory access modules is specifically used to respond to the received read instruction, retrieve a menu image to be displayed from the storage unit that matches the data size based on the starting address, and send the display position and the menu image to be displayed to the display pipeline module; the display pipeline module is used to display the menu image to be displayed after mixing it with the main display image based on the display position.

[0013] In this embodiment, the layer control module sends the starting address, data size, and display position to the direct memory read module, enabling the direct memory read module to accurately obtain the corresponding menu image to be displayed. Furthermore, even if multiple memory read modules obtain multiple menu images to be displayed, the display pipeline module can still blend them based on their respective display positions, ensuring that the menu images are correctly displayed in the blended image.

[0014] In conjunction with the technical solution provided in the first aspect above, in some possible implementations, the display pipeline module includes: a layer blending circuit and a data format processing circuit; the data format processing circuit is used to obtain the menu image to be displayed from the first storage module, convert the menu image to be displayed into a specified format, and transmit the menu image to be displayed converted into the specified format to the layer blending circuit; the layer blending circuit is used to blend the menu image to be displayed with the main display image and then display it.

[0015] In this embodiment, the format of the menu image to be displayed can be processed by the data format processing circuit, so that the format-processed menu image to be displayed can meet the subsequent processing requirements.

[0016] In conjunction with the technical solution provided in the first aspect above, in some possible implementations, the display pipeline module includes: a layer blending circuit and a scaling circuit; the scaling circuit is used to obtain the menu image to be displayed from the first storage module, perform a scaling operation on the menu image to be displayed, and transmit the scaled menu image to be displayed to the layer blending circuit; the layer blending circuit is used to blend the menu image to be displayed with the main display image and then display it.

[0017] In this embodiment, a scaling circuit can be used to scale the menu image to be displayed, so that the menu image to be displayed can meet the display requirements and prevent the menu image to be displayed from being too large or too small when displayed.

[0018] In conjunction with the technical solution provided in the first aspect above, in some possible implementations, the first storage module simultaneously outputs N sub-menu images to be displayed, all of which are partial compositions of the same frame of menu images to be displayed. Adjacent sub-menu images among the N sub-menu images have overlapping areas. The display pipeline module includes: a layer blending circuit and an image allocation module. The image allocation module is used to acquire the N sub-menu images to be displayed, and for each pair of sub-menu images with overlapping areas, to cut the overlapping portion of the two sub-menu images so that the boundaries of the two sub-menu images are aligned when displayed. The N sub-menu images after the cutting process are sent to the layer blending circuit. The layer blending circuit is used to blend the N sub-menu images with the main display image before displaying them.

[0019] In this embodiment, since the filter window boundary affects the boundary of the sub-menu images to be displayed, directly stitching them together would result in poor display quality at the stitching point. Therefore, N sub-menu images to be displayed are set to be partial compositions of the same frame of menu images to be displayed, with overlapping areas between adjacent sub-menu images. The overlapping portions of two sub-menu images are then cut off. Since the overlapping portion is necessarily the boundary of the sub-menu images, what is cut off is also necessarily the overlapping portion. This reduces the influence of the filter window boundary on the stitching position of adjacent sub-menu images during subsequent stitching.

[0020] In conjunction with the technical solution provided in the first aspect above, in some possible implementations, the OSD image rendering system further includes: a compression circuit, used to compress the menu image to be displayed output by the processor, and store the compressed menu image to be displayed in the first storage module; correspondingly, the display pipeline module includes: a layer blending circuit and a decompression circuit; the decompression circuit is used to obtain the menu image to be displayed from the first storage module, decompress the menu image to be displayed, and transmit the decompressed menu image to be displayed to the layer blending circuit; the layer blending circuit is used to blend the menu image to be displayed with the main display image and then display it.

[0021] In this embodiment, a compression circuit is used to compress the menu image to be displayed output by the processor, thereby reducing the storage space occupied by the menu image. Correspondingly, a decompression circuit is provided to decompress the compressed menu image to ensure that it can be displayed correctly.

[0022] In conjunction with the technical solution provided in the first aspect above, in some possible implementations, the processor is disposed in a first electronic device, and the first storage module, the display pipeline module, and the second storage module are disposed in a second electronic device.

[0023] In this embodiment, since the processor, the first storage module, the display pipeline module, and the second storage module can be located in different electronic devices, the second electronic device can reuse the processor of the first electronic device to perform image rendering, thereby reducing costs.

[0024] In conjunction with the technical solution provided in the first aspect above, in some possible implementations, the registers in the display pipeline module are all double-buffered registers.

[0025] In this embodiment, by setting all registers in the display pipeline module as double-buffered registers, the display pipeline module can meet the needs of updating the menu images to be displayed at a high frequency.

[0026] Secondly, this application provides an OSD image rendering method, applied to a processor of an OSD image rendering system described in the first aspect and / or in combination with any possible implementation of the first aspect. The method includes: obtaining rendering instructions and rendering resources required for rendering from a first storage module, and completing the rendering of a menu image to be displayed to obtain a menu image to be sent for display; storing the menu image to be sent for display in the first storage module so that the display pipeline module obtains the menu image to be sent for display from the first storage module and performs mixed display with a main display image.

[0027] In this embodiment, the processor reads and writes OSD rendering data from a first storage module, while a second storage module stores other data besides the OSD rendering data. Compared to existing methods of reading and writing OSD rendering data from DDR, this reduces the number of read and write operations required by the first storage module, thereby reducing the occurrence of a cache wall. This improves the efficiency of reading and writing OSD rendering data, and consequently, improves image rendering efficiency.

[0028] Thirdly, this application provides a chip including an OSD image rendering system as described in the first aspect above and / or in combination with any possible implementation of the first aspect above.

[0029] Fourthly, this application provides an electronic device including the chip described in the third aspect above. Attached Figure Description

[0030] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 is a structural block diagram of a first OSD image rendering system provided in an embodiment of this application;

[0032] Figure 2 is a schematic diagram of the space division of a first type of storage unit provided in an embodiment of this application;

[0033] Figure 3 is a schematic diagram of the space division of a second type of storage unit provided in an embodiment of this application;

[0034] Figure 4 is a structural block diagram of a second OSD image rendering system provided in an embodiment of this application;

[0035] Figure 5 is a schematic diagram of the rendering process of the first OSD image rendering system provided in the embodiment of this application;

[0036] Figure 6 is a schematic diagram of the rendering process of the second OSD image rendering system provided in an embodiment of this application;

[0037] Figure 7 is a schematic diagram of the rendering process of the third OSD image rendering system provided in the embodiment of this application;

[0038] Figure 8 is a flowchart illustrating an OSD image rendering method provided in an embodiment of this application. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0040] Please refer to Figure 1, which is a structural block diagram of the first OSD image rendering system shown in the embodiment of this application. It will be described below with reference to Figure 1.

[0041] As shown in Figure 1, the OSD image rendering system 100 includes: a processor 110, a first storage module 120, a display pipeline module 130, and a second storage module 140.

[0042] The first storage module 120 is used to store OSD rendering data; the second storage module 140 is used to store other data besides OSD rendering data; the OSD rendering data includes: rendering instructions and rendering resources required for rendering, as well as the rendered menu image to be displayed.

[0043] The rendering instructions are used to instruct the processor 110 to perform corresponding rendering operations, and the rendering resources provide the necessary data for the rendering operations of the processor 110, such as material pack data and texture pack data.

[0044] The second storage module 140 is the shared storage module in the device or system used by the OSD image rendering system 100, such as DDR, and its specific type is not limited here.

[0045] In one implementation, the first storage module 120 includes: a processor 110 memory access interface, a pipeline interface module, a storage unit, and an access arbitrator.

[0046] Specifically, the processor 110 memory access interface and pipeline interface are connected to the memory unit by accessing the arbitrator.

[0047] The specific types of access to the arbitrator can be set according to actual needs; there are no restrictions on the specific types of access to the arbitrator here.

[0048] The processor 110 memory access interface is used to communicate with the processor 110 so that the processor 110 can obtain rendering instructions and rendering resources required for rendering from the storage unit through the processor 110 memory access interface.

[0049] Optionally, the memory access interface of processor 110 can be any type of interface that enables communication with processor 110.

[0050] Optionally, the processor 110 memory access interface may include a processor 110 read interface and a processor 110 write interface, wherein the processor 110 obtains OSD rendering data from the storage unit through the processor 110 read interface and writes the menu image to be displayed to the storage unit through the processor 110 write interface.

[0051] Optionally, if the processor 110's interface is an AMBA AXI (Advanced Microcontroller Bus Architecture) bus with a bus width of 64 bits, supporting a maximum burst transfer of 64 bytes, and if the memory unit is a 32-bit SRAM (Static Random-Access Memory), then the processor 110's memory access interface needs to segment and buffer the data transmitted by the processor 110 and return the corresponding AXI acknowledge signal.

[0052] The storage unit is used to store rendering instructions, rendering resources, and menu images to be displayed.

[0053] Optionally, the storage unit can be any type of electronic component with storage capabilities, such as a storage array, SRAM memory, etc. The implementation methods of the storage unit include, but are not limited to, the methods exemplified here.

[0054] Optionally, the storage unit may include 16 parallel SRAMs, and a single access can read / write a maximum of 16 SRAMs simultaneously.

[0055] Optionally, the storage unit can also be used to store intermediate data generated by the processor 110 during the rendering process.

[0056] Optionally, the size of the storage unit can be the sum of the storage space required for rendering instructions, rendering resources, menu images to be displayed, and intermediate data.

[0057] When the menu image has high refresh rate requirements, the storage space required for the menu image to be displayed is twice that required for the target resolution menu image to be displayed, in order to meet the ping-pong storage method and achieve high refresh rate display of the menu image.

[0058] For ease of understanding, Figure 2 will be used as an example. As shown in Figure 2, the storage unit can be divided into a rendering instruction space, a rendering resource space, a menu image space to be displayed, and an intermediate data space.

[0059] When the OSD image rendering system 100 needs to execute the ping-pong rendering mode (that is, when the menu image of the current frame is being displayed, the menu image of the next frame has been rendered and needs to be stored in the storage unit), the storage unit also needs to contain two menu image spaces to be sent for display, as shown in Figure 3.

[0060] The pipeline interface module is used to send the menu image to be displayed to the display pipeline module 130.

[0061] In one implementation, the pipeline interface module can be any type of interface that enables communication with the display pipeline module 130.

[0062] Optionally, the pipeline interface module may also include: a layer control module and at least one direct memory read module.

[0063] The layer control module is used to send read commands to each direct memory read module. The read commands include the starting address, data size, and display position.

[0064] The starting address is the starting point for reading data from the storage unit, and the data size specifies the amount of data to be read. The display position represents the final position of the read menu image on the display. It can be determined by the coordinates of the reference point (the coordinates of the reference point in the pixel coordinate system) and its length and width, or by the coordinates of two diagonal points, or by the coordinates of the four corner points. The specific form of the display position is not limited to the examples mentioned above.

[0065] Optionally, when there are two or more direct memory read modules, the layer control module can first check whether the width of the display position of the menu image to be read is greater than a preset threshold before sending read commands to the direct memory read modules. If the width of the display position of any menu image to be read is greater than the preset width threshold, then read commands corresponding to N sub-menu images to be read can be generated. Then, the read command corresponding to each sub-menu image to be read is sent to multiple direct memory read modules, so that each direct memory read module reads the corresponding sub-menu image to be read from the first storage module according to the sub-menu image to be read.

[0066] Among them, each of the N sub-menu images to be displayed is a partial composition of the main menu image, and adjacent sub-menu images have overlapping areas. N is less than or equal to the number of direct memory read modules.

[0067] The width of the display position of the menu image to be displayed refers to the span of the menu image on the X-axis of the pixel coordinate system (the difference between the maximum and minimum horizontal coordinates of the pixels in the menu image to be displayed), which can be determined based on the display position of the menu image to be displayed.

[0068] The preset width threshold can be determined based on the monitor's maximum display width (the difference between the maximum and minimum horizontal coordinates in the pixel coordinate system). For example, the preset width threshold can be half of the monitor's maximum display width, or it can be two-thirds of the monitor's maximum display width.

[0069] The overlapping area between two adjacent sub-menu images to be displayed means that they overlap in the Y-axis direction of the pixel coordinate system. That is, the two adjacent sub-menu images to be displayed have pixels with the same coordinates in the same pixel coordinate system (i.e., the horizontal and vertical coordinates of the pixels are the same).

[0070] Optionally, the method for generating the read instructions corresponding to the N sub-menu images to be displayed can be as follows: First, divide the number of pixels in the first row of the menu image to be displayed into N equal parts, and then determine the initial start address and initial end address of each part of pixels. Then, reduce the start address of each part of pixels (excluding the first part) by the memory size occupied by M pixels to obtain the start address of each part of pixels, that is, obtain N start addresses. The start address will not exceed the start address of the menu image to be displayed, and the start address of the first part of pixels is the start address of the menu image to be displayed.

[0071] Then, determine the data size corresponding to each pixel. This data size is equal to (the number of pixels in the first row of the menu image to be displayed ÷ N + M + K) × the memory size occupied by each pixel × the number of pixels in each column of the menu image to be displayed. Specifically, the data size corresponding to the sub-menu image with the smallest initial starting address is equal to (the number of pixels in the first row of the menu image to be displayed ÷ N + K) × the memory size occupied by each pixel. The data size corresponding to the sub-menu image with the largest initial starting address is equal to (the number of pixels in the first row of the menu image to be displayed ÷ N + M) × the memory size occupied by each pixel. K is an integer greater than or equal to 1. By adding K to the calculation of the data size for all pixels except the last pixel, it is ensured that the overlapping area between two adjacent sub-menu images includes the dividing line (that is, the position where the number of pixels in the first row of the menu image is evenly divided into N parts).

[0072] Then, based on the starting address of each sub-menu image to be displayed, the initial starting address and initial ending address of the sub-menu image to be displayed, the number of pixels in each row of the sub-menu image and the number of pixels in each row of the sub-menu image to be displayed, the display position of each sub-menu image to be displayed is calculated.

[0073] For example, if N=3, M=3, K=2, and the number of pixels per row in the menu image to be displayed is 30, then firstly, the first row of the menu image to be displayed is divided into 3 parts, each with 10 pixels. If the starting address of the menu image to be displayed is A, then the three initial starting addresses are A, A+10B, and A+20B, where B is the memory size occupied by one pixel. Then, the starting address of each pixel is reduced by the memory size occupied by 3 pixels. However, since the starting address will not exceed the starting address of the menu image to be displayed, the first starting address remains A, and the other two starting addresses are A+7B and A+17B, respectively.

[0074] The data size corresponding to the sub-menu image to be displayed starting at address A is (10+2)×B×20, which is 240B. The data size corresponding to the sub-menu image to be displayed starting at address A+7B is (10+2+3)×B×20, which is 300B. The data size corresponding to the sub-menu image to be displayed starting at address A+17B is (10+3)×B×20, which is 260B.

[0075] If the top-left corner coordinate of the image to be displayed is (0, 0), and the length of the image is 30 (horizontal axis length) and the width is 20 (vertical axis length), then the top-left corner coordinate of the sub-menu image starting at address A is (0, 0), with a length of 10 and a width of 20. Therefore, in subsequent image blending, the last two columns of pixels of this sub-menu image need to be discarded. The top-left corner coordinate of the sub-menu image starting at address A+7B is (10, 0), with a length of 10 and a width of 20. Therefore, in subsequent image blending, the first three columns of pixels and the last two columns of pixels of this sub-menu image need to be discarded. The top-left corner coordinate of the sub-menu image starting at address A+17B is (20, 0), with a length of 10 and a width of 20. Therefore, in subsequent image blending, the first three columns of pixels of this sub-menu image need to be discarded.

[0076] Based on this, three read instructions are obtained: Read Instruction 1: Starting address A, data size 240B, display position at top left corner coordinates (0, 0), length 10, width 20. Read Instruction 2: Starting address A+7B, data size 300B, display position at top left corner coordinates (10, 0), length 10, width 20. Read Instruction 3: Starting address A+17B, data size 260B, display position at top left corner coordinates (20, 0), length 10, width 20.

[0077] The examples provided are for illustrative purposes only and should not be construed as limiting the scope of this application.

[0078] The layer control module can be any type of circuit structure that can implement the layer control module, or it can be implemented by a programmable logic array, or it can be implemented by a software module.

[0079] Each direct memory read module is specifically used to respond to a received read instruction, retrieve the menu image to be displayed from the storage unit based on the starting address and matching the data size, and send the display position and the menu image to be displayed to the display pipeline module 130.

[0080] The Direct Memory Access (DMA) module can be implemented using any type of DMA.

[0081] Optionally, the sum of the bandwidths of all memory read modules equals the bandwidth of the main display pipeline. The bandwidth of the main display pipeline is the same as the bandwidth of the pipeline that transmits the main display image.

[0082] For example, if there are two memory read modules and the main display pipeline has a bandwidth of 4 pixels per cycle, then the bandwidth of each memory read module is 2 pixels per cycle. Alternatively, if there are four memory read modules, then the bandwidth of each memory read module is 1 pixel per cycle. These examples are for illustrative purposes only and should not be construed as limiting the scope of this application.

[0083] Accordingly, in this case, the display pipeline module 130 is specifically used to display the menu image to be displayed after mixing it with the main display image based on the display position.

[0084] The specific method of mixing the menu image to be displayed with the main display image is well known to those skilled in the art, and will not be elaborated here for the sake of brevity.

[0085] In one implementation, the first storage module 120 may further include an external memory access interface.

[0086] The external memory access interface is connected to the access arbiter. The external memory access interface is used to receive rendering instructions and rendering resources, and write the rendering instructions and rendering resources to the storage unit through the access arbiter.

[0087] External control circuits such as CPU (Central Processing Unit) and MCU can use this external memory access interface to control whether the OSD image rendering system 100 starts rendering, and to write rendering instructions and rendering resources into the memory unit, or update the rendering instructions and rendering resources in the memory unit.

[0088] The specific details of rendering commands and rendering resources can be set according to actual needs; there are no restrictions on their specific content here.

[0089] Optionally, when the storage unit is SRAM, since the address space of SRAM may be large, the external memory access interface can use the base address plus offset addressing mode for addressing. Therefore, after configuring the base address for the external memory access interface, the SRAM address pointer will point to the base address, and the base address will be automatically incremented after each read / write operation.

[0090] In one implementation, to improve the read and write speeds of the first storage module 120, the bandwidth of the storage unit can be set to a larger value. In this case, a narrow transmission module can be provided, through which the storage unit connects to the access arbitrator.

[0091] When writing data to a storage unit, if the write request is less than the data width of the storage unit, in order to ensure that other data is overwritten by the write operation, the narrow transmission module takes out the data width of one unit of storage unit, merges it with the write data, and then writes the merged data into the storage unit.

[0092] The processor 110 is used to obtain rendering instructions and rendering resources from the first storage module 120, render the rendering resources according to the rendering instructions, obtain the menu image to be displayed, and store the menu image to be displayed in the first storage module 120.

[0093] In one implementation, the processor 110 described above may include a GPU.

[0094] The processor 110 includes a GPU dedicated to rendering OSD images.

[0095] Optionally, the processor 110 may employ a low-power GPU, such as a 2.5D GPU based on Bézier curve drawing (a type of graphics processor designed for MCU / MPU applications that require hardware-accelerated user interface display effects), etc. The specific type of GPU is not limited here.

[0096] The display pipeline module 130 is used to obtain the menu image to be displayed from the first storage module 120 and display it after mixing the menu image to be displayed with the main display image.

[0097] In this implementation, the display pipeline module 130 may include a layer blending circuit. The layer blending circuit retrieves the menu image to be displayed from the first storage module 120 and blends the menu image to be displayed with the main display image before displaying it.

[0098] The layer blending circuit can be any type of circuit structure that can implement the layer blending circuit, or it can be implemented by a programmable logic array.

[0099] In one implementation, the display pipeline module 130 may include a layer blending circuit and a data format processing circuit.

[0100] The data format processing circuit is used to obtain the menu image to be displayed from the first storage module 120, convert the menu image to be displayed into a specified format, and transmit the converted menu image to the layer blending circuit. The layer blending circuit is used to blend the menu image to be displayed with the main display image before displaying it.

[0101] In this implementation, the specified format is the format of the menu image to be displayed, as required by the layer blending circuit.

[0102] The data format processing circuit can be any type of circuit structure that can implement the function of the data format processing circuit, or it can be implemented by a programmable logic array.

[0103] The data format processing circuit can process the format of the menu image to be displayed, so that the format-processed menu image can meet the subsequent processing requirements.

[0104] Optionally, when the first storage module 120 can simultaneously send N menu images to be displayed, the data format processing circuit includes N circuits, each processing one menu image to be displayed. N is a positive integer greater than 1.

[0105] Optionally, if the actual pipeline module 130 includes a data format processing circuit, the read instruction may include the data type in addition to the starting address, data size, and display position.

[0106] The data type indicates the data format of the menu image to be displayed. Different data types of menu images may occupy different amounts of memory and have different data arrangements.

[0107] In this method, when the data format processing circuit receives the menu image to be displayed, it also simultaneously receives the data type of the menu image to be displayed. Then, based on the data type of the menu image to be displayed and the specified format conversion method, it converts the received menu image to be displayed into a menu image of the specified format.

[0108] In one implementation, the display pipeline module 130 may also include a layer blending circuit and a scaling circuit.

[0109] The scaling circuit retrieves the menu image to be displayed from the first storage module 120, scales the menu image, and transmits the scaled menu image to the layer blending circuit. The layer blending circuit blends the menu image to be displayed with the main display image for display.

[0110] The scaling circuit is used to scale the menu image to be displayed so that the menu image meets the display requirements and prevents the menu image from being too large or too small when displayed.

[0111] The scaling circuit can be any type of circuit structure that can implement the scaling circuit function, or it can be implemented by a programmable logic array.

[0112] Optionally, the scaling circuit can be used to enlarge the menu image to be displayed. In this case, the menu image to be displayed that the processor renders and stores can be a small image that occupies less memory, thereby improving the processor's image rendering efficiency and saving storage space.

[0113] Optionally, when the first storage module 120 can simultaneously send N menu images to be displayed, the scaling circuit includes N circuits, each scaling circuit processing one menu image to be displayed. N is a positive integer greater than 1.

[0114] In one embodiment, the display pipeline module 130 may further include: a layer blending circuit and an image allocation module.

[0115] In this implementation, the first storage module 120 simultaneously outputs N sub-menu images to be displayed. These N sub-menu images are partial compositions of the same frame of menu images to be displayed, and adjacent sub-menu images overlap. The image allocation module acquires the N sub-menu images and, for each pair of overlapping sub-menu images, cuts the overlapping portion to ensure that the boundaries of the two cut sub-menu images are aligned during display. The cut N sub-menu images are then sent to the layer blending circuit. The layer blending circuit blends these N sub-menu images with the main display image before displaying them.

[0116] Specifically, the boundary alignment of the two sub-menu images to be displayed means that the boundaries of the two cut sub-menu images to be displayed just touch, so that the display screens of the two sub-menu images to be displayed are stitched together and the resulting display screen is continuous.

[0117] For ease of understanding, let's take N=2 as an example. The menu image to be displayed in the storage unit is divided into two sub-menu images, which are the left and right halves of the main menu image, respectively. The overlapping area of ​​the two sub-menu images is 6 columns of pixels. After the image allocation module obtains these two sub-menu images, it cuts off the rightmost 3 columns of pixels of the left half and the leftmost 3 columns of pixels of the right half, thus eliminating the overlap between the two cut sub-menu images. Then, the two cut sub-menu images are sent to the layer blending circuit.

[0118] Optionally, the image allocation module may receive the display position of each of the N sub-menu images to be sent and displayed simultaneously, and determine whether there is an overlapping area based on the display position.

[0119] Because the filter window boundary affects the boundary of the sub-menu images to be displayed, directly stitching them together would result in poor display quality at the stitching point. Therefore, N sub-menu images to be displayed are set to be partial images composed of the same frame's sub-menu images, with overlapping areas between adjacent sub-menu images. The overlapping areas of two sub-menu images are then cut off. Since the overlapping area is necessarily the boundary of the sub-menu images, what is cut off is also necessarily the overlapping area. This reduces the influence of the filter window boundary on the stitching position of adjacent sub-menu images during subsequent stitching.

[0120] The image allocation module can be any type of circuit structure that can implement the function of the image allocation module, or it can be implemented by a programmable logic array, or it can be implemented by software.

[0121] Optionally, overlapping areas between two adjacent sub-menu images to be displayed refer to overlap along the Y-axis in the pixel coordinate system. That is, the two adjacent sub-menu images to be displayed have pixels with the same coordinates in the same pixel coordinate system (i.e., the horizontal and vertical coordinates of the pixels are the same).

[0122] In one implementation, the OSD image rendering system 100 may further include a compression circuit.

[0123] The compression circuit is used to compress the menu image to be displayed output by the processor 110 and store the compressed menu image to be displayed in the first storage module 120.

[0124] Accordingly, the display pipeline module 130 may include: a layer blending circuit and a decompression circuit.

[0125] The decompression circuit retrieves the menu image to be displayed from the first storage module 120, decompresses the menu image, and transmits the decompressed menu image to the layer blending circuit. The layer blending circuit blends the menu image to be displayed with the main display image for display.

[0126] The menu image to be displayed, output by the processor 110, is compressed by a compression circuit to reduce its footprint in the storage unit. Correspondingly, a decompression circuit is provided to decompress the compressed menu image to ensure it can be displayed correctly.

[0127] The compression circuit and decompression circuit can be any type of circuit structure that can implement the function of the compression circuit or decompression circuit, or they can be implemented separately by a programmable logic array.

[0128] Optionally, when the first storage module 120 can simultaneously send N menu images to be displayed, the decompression circuit includes N circuits, each processing one menu image to be displayed. N is a positive integer greater than 1.

[0129] In one implementation, all registers in the display pipeline module 130 are double-buffered registers. By setting all registers in the display pipeline module 130 to double-buffered registers, the display pipeline module 130 can meet the needs of updating the menu images to be displayed at a relatively high frequency.

[0130] In one embodiment, the display pipeline module 130 may include, in addition to the layer blending circuit, any one or more of the data format processing circuit, scaling circuit, image distribution module, and decompression circuit described above.

[0131] For example, the display pipeline module 130 may include a layer blending circuit, a data format processing circuit, and a scaling circuit; the display pipeline module 130 may include a layer blending circuit, a scaling circuit, and an image allocation module; the display pipeline module 130 may include a layer blending circuit, an image allocation module, and a decompression circuit; the display pipeline module 130 may include a layer blending circuit, a data format processing circuit, and an image allocation module; the display pipeline module 130 may include a layer blending circuit, a data format processing circuit, a scaling circuit, and a decompression circuit; the display pipeline module 130 may include a layer blending circuit, a data format processing circuit, a scaling circuit, and an image allocation module; the display pipeline module 130 may include a layer blending circuit, a data format processing circuit, a scaling circuit, and a decompression circuit; the display pipeline module 130 may include a layer blending circuit, a scaling circuit, an image allocation module, and a decompression circuit; the display pipeline module 130 may include a layer blending circuit, a data format processing circuit, a scaling circuit, an image allocation module, and a decompression circuit.

[0132] In one implementation, the processor of the OSD image rendering system can be located in a first electronic device, and the first storage module, display pipeline module, and second storage module of the OSD image rendering system can be located in a second electronic device.

[0133] The first electronic device can be, for example, a host computer or a server, while the second electronic device can be, for example, a monitor or a television. By reusing the processor in the first electronic device to perform image rendering operations, it is unnecessary to set up an additional processor in the second electronic device to perform image rendering operations, thereby reducing costs.

[0134] To facilitate understanding of the OSD image rendering system described above, we will use the OSD image rendering system shown in Figure 4 as an example for explanation.

[0135] As shown in Figure 4, the OSD image rendering system includes a GPU, a first storage module, a display pipeline module, a second storage module (not shown in Figure 4), and a compression module.

[0136] The first storage module includes a processor memory access interface, a pipeline interface module, storage units, an access arbiter, an external memory access interface, and a narrow transfer module. The pipeline interface module includes a layer control module and two direct memory access modules.

[0137] The display pipeline module includes a layer blending circuit, two data format processing circuits, two scaling circuits, an image allocation module, and two decompression circuits. The two data format processing circuits are connected one-to-one with two direct memory read modules. Each data format processing circuit retrieves the menu image to be displayed from the connected direct memory read module and modifies its format to meet the requirements of the decompression module. After decompressing the menu image, the decompression module sends it to the image allocation module so that the image allocation module can determine whether cropping is necessary. The image allocation module assigns the two menu images to the two scaling circuits respectively. After scaling, the two menu images are then blended with the main display image by the layer blending circuit before being displayed.

[0138] The specific structure and function of each component shown in Figure 4 have been clearly described above, and will not be repeated here for the sake of brevity.

[0139] In one implementation, when a portion of the menu image is updated, the entire menu image can be re-rendered to obtain a new menu image to be displayed. That is, all rendering instructions in the storage unit are updated via an external memory access interface, and the GPU executes all the updated rendering instructions to complete the rendering of the menu image to be displayed.

[0140] Alternatively, only the updated portion of the menu image can be rendered, and then the rendered updated image can overwrite the corresponding position in the previous frame of the menu image to be displayed stored in the storage unit. That is, the external memory access interface updates part of the rendering instructions (corresponding to the updated image area) in the storage unit, the GPU only executes the updated rendering instructions, and then writes the rendered part of the image into the corresponding area in the storage unit, thus completing the update of the previous frame of the menu image to be displayed stored in the storage unit.

[0141] To facilitate understanding of the specific workflow of the OSD image rendering system described above, the OSD image rendering process diagrams shown in Figures 5 to 7 will be used as examples for explanation.

[0142] As shown in Figure 5, in this method, the refresh rate of the OSD layer (i.e., the menu image) is much lower than the refresh rate of the main display image, or it is in a normal working mode where the OSD layer remains unchanged for a long time. When the OSD layer needs to be displayed, the GPU is first notified to draw the OSD layer (i.e., the Nth frame OSD layer) through the external memory access interface. After the OSD layer is drawn, the GPU generates a completion interrupt and writes the rendered OSD layer to be displayed into the storage unit. After receiving the GPU interrupt, the relevant registers in the display pipeline module are configured (OSD configuration shown in Figure 5). Due to the existence of double-buffered registers, if the main display image has not started refreshing after the configuration is completed, the display pipeline module starts working and processes the OSD layer to be displayed (i.e., the Nth frame OSD layer) (format processing, decompression, image allocation, scaling, etc.). If the main display image is already in the active display area (the main display image has been refreshed and displayed), the OSD layer to be displayed will be mixed with the main display image when the main display image is refreshed in the next frame. This working mode can be used when GPU rendering of OSD layers is slow. If the OSD layer needs to be switched or redrawn by the GPU in this working mode, the display pipeline module needs to be turned off first, and the display pipeline module can only be turned on again after the new OSD layer has been rendered and stored.

[0143] Among them, the synchronization area is called Sync, the blanking area is called Blank, and the display area is called Active. Sync, Blank, and Active refer to the areas on the time axis, that is, they all represent time ranges.

[0144] Since the main display image needs to be displayed within its corresponding Active time range, the OSD layer to be displayed needs to be rendered before it becomes Active, and then it can be displayed in the next Active area.

[0145] If the OSD layer to be displayed needs to be shown in the target Active area, then the OSD layer to be displayed needs to be rendered before entering the target Active area. After rendering is complete, if it has not yet entered the target Active area, the data of the OSD layer to be displayed can be transferred outside the layer blending circuit to wait (the layer blending circuit will not receive the data of the OSD layer to be displayed outside the Active area). When the starting time node of the target Active area is reached, the layer blending circuit receives the data of the OSD layer to be displayed and the data of the main display image, and blends the data of the OSD layer to be displayed and the data of the main display image before displaying it.

[0146] Figure 6 illustrates the partial update mode for the OSD layer. This mode is suitable for scenarios where only a small portion of the OSD layer needs modification, while the rest remains unchanged. In this scenario, the drawing and interaction of the OSD layer are consistent with the mode shown in Figure 5. The difference lies in that when the OSD layer undergoes a small change, the rendering instructions in the storage unit can be updated first. The GPU is then notified when the main display image finishes displaying. Subsequently, the GPU executes the update instructions in the storage unit to render the small-scale updated OSD layer, and the updated OSD image overwrites the corresponding area of ​​the original OSD layer in the storage unit, thus achieving a partial update of the OSD layer. This partial update operation needs to be completed before the next frame of the main display image begins displaying. Therefore, the updated area of ​​the OSD layer needs to be completed by the GPU within a sufficiently short time; otherwise, the mode shown in Figure 5 should be used. Alternatively, GPU clock speed can be increased, or a high-performance mode can be enabled to ensure that the GPU can complete the update before the next main pipeline begins displaying.

[0147] The OSD layer update needs to be completed before the target Active region, which is the Active region where the partially updated OSD layer needs to be displayed.

[0148] Figure 7 illustrates the rendering flow of the ping-pong rendering mode. This mode is suitable for scenarios where the OSD layer data is small and needs to be updated frame by frame (to become more realistic as the main display image is updated). In this scenario, as shown in Figure 7, the GPU can begin rendering the next frame of the OSD layer once the corresponding OSD layer has been displayed. The display pipeline module displays the current frame data, while the GPU renders in advance and writes the next frame data into the storage unit. The two pending display menu image spaces in the storage unit are written alternately.

[0149] Since the method shown in Figure 7 requires displaying one OSD image in the Active area corresponding to each main display image, the rendering operation of the Nth frame OSD layer needs to be completed before the Active area corresponding to the Nth frame main display image, so as to display the Nth frame OSD layer in the Active area corresponding to the Nth frame main display image.

[0150] Based on the same technical concept, this application also provides an OSD image rendering method, as shown in Figure 8, which is applied to the processor of the aforementioned OSD image rendering system.

[0151] The OSD image rendering system further includes a first storage module, a display pipeline module, and a second storage module. The first storage module stores OSD rendering data; the second storage module stores other data besides the OSD rendering data. The OSD rendering data includes rendering instructions and rendering resources required for rendering, as well as the rendered menu image to be displayed. The specific structure of the OSD image rendering system has been clearly described above and will not be repeated here for brevity.

[0152] S100: Obtain the rendering instructions and rendering resources required for rendering from the first storage module, and complete the rendering of the menu image to be displayed to obtain the menu image to be sent for display.

[0153] Optionally, after receiving the instruction to start configuration, the rendering instructions and rendering resources required for rendering can be obtained from the first storage module.

[0154] The specific process and principles of rendering based on rendering instructions and rendering resources are well known to those skilled in the art, and will not be elaborated here for the sake of brevity.

[0155] Optionally, intermediate data generated during the rendering of the menu image to be displayed can also be stored in the first storage module.

[0156] S200: Store the menu image to be displayed in the first storage module so that the display pipeline module can obtain the menu image to be displayed from the first storage module and mix it with the main display image.

[0157] The image of the menu to be displayed can be stored in the first storage module through the processor's memory access interface.

[0158] Optionally, when storing the menu image to be displayed in the first storage module, it can be stored in the same storage address each time, overwriting the previous content.

[0159] Alternatively, when it is necessary to store the menu image to be displayed in the first storage module, the menu image to be displayed can be written into the storage unit at the location corresponding to the instruction according to the received instruction including the write address.

[0160] Alternatively, when retrieving the rendering instructions and rendering resources required for rendering from the first storage module, the address of the menu image to be displayed is also obtained. After rendering the menu image to be displayed is completed, the menu image to be displayed is written into the corresponding address area.

[0161] The specific rendering steps and operation process of the processor have been clearly described above, and will not be repeated here for the sake of brevity.

[0162] Based on the same technical concept, this application also provides a chip that may include the OSD image rendering system provided in any of the above embodiments.

[0163] Alternatively, the chip can be a GPU (Graphics Processing Unit), CPU (Central Processing Unit), AI (Artificial Intelligence), ISP (Image Signal Processor), VPU (Video Processing Unit), etc.

[0164] The OSD image rendering system includes processors such as GPU, CPU, AI, ISP, and VPU data processing cores.

[0165] Based on the same inventive concept, this application also provides an electronic device, which may include the chip provided in the above embodiments, and the chip includes an OSD image rendering system.

[0166] In this embodiment, the electronic device can be a computer, server, microcontroller, embedded device, etc., and there are no restrictions.

[0167] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An OSD image rendering system, characterized in that, include: Processor, first storage module, display pipeline module, second storage module; The first storage module is used to store OSD rendering data; The second storage module is used to store data other than OSD rendering data; The OSD rendering data includes: rendering instructions and rendering resources required for rendering, as well as the rendered menu image to be displayed; The processor is configured to obtain the rendering instructions and the rendering resources from the first storage module, render the rendering resources according to the rendering instructions to obtain the menu image to be displayed, and store the menu image to be displayed in the first storage module. The display pipeline module is used to obtain the menu image to be displayed from the first storage module, and then display the menu image to be displayed after mixing it with the main display image.

2. The OSD image rendering system according to claim 1, characterized in that, The first storage module includes: Processor memory access interface, pipeline interface module, memory unit, access arbitrator; The processor memory access interface and pipeline interface are respectively connected to the memory unit through the access arbiter; The processor memory access interface is used to communicate with the processor so that the processor can obtain rendering instructions and rendering resources required for rendering from the storage unit through the processor memory access interface; The storage unit is used to store the rendering instructions, the rendering resources, and the menu image to be displayed. The pipeline interface module is used to send the menu image to be displayed to the display pipeline module.

3. The OSD image rendering system according to claim 2, characterized in that, The first storage module further includes: An external memory access interface is connected to the access arbiter. The external memory access interface is used to receive the rendering instructions and the rendering resources, and write the rendering instructions and the rendering resources into the storage unit through the access arbiter.

4. The OSD image rendering system according to claim 2, characterized in that, The pipeline interface module includes: Layer control module, at least one direct memory read module; The layer control module is used to send a read instruction to each of the direct memory read modules, and the read instruction includes the starting address, data size, and display position. Each of the direct memory read modules is specifically used to respond to a received read instruction, retrieve a menu image to be displayed from the storage unit that matches the data size based on the starting address, and send the display position and the menu image to be displayed to the display pipeline module; The display pipeline module is used to display the menu image to be displayed by mixing it with the main display image based on the display position.

5. The OSD image rendering system according to claim 1, characterized in that, The display pipeline module includes: Layer blending circuit, data format processing circuit; The data format processing circuit is used to obtain the menu image to be displayed from the first storage module, convert the menu image to be displayed into a specified format, and transmit the menu image to be displayed converted into the specified format to the layer mixing circuit. The layer blending circuit is used to blend the menu image to be displayed with the main display image before displaying it.

6. The OSD image rendering system according to claim 1, characterized in that, The display pipeline module includes: Layer blending circuit, scaling circuit; The scaling circuit is used to obtain the menu image to be displayed from the first storage module, scale the menu image to be displayed, and transmit the scaled menu image to the layer blending circuit. The layer blending circuit is used to blend the menu image to be displayed with the main display image before displaying it.

7. The OSD image rendering system according to claim 1, characterized in that, The first storage module simultaneously outputs N sub-menu images to be displayed, each of which is a partial composition of the same frame of menu images to be displayed. Adjacent sub-menu images in the N sub-menu images have overlapping areas. The display pipeline module includes: Layer blending circuit, image assignment module; The image allocation module is used to acquire N sub-menu images to be displayed, and for every two sub-menu images that have overlapping areas, to cut the overlapping parts of the two sub-menu images to be displayed, so that when the two sub-menu images to be displayed are displayed, the boundaries of the two sub-menu images to be displayed are aligned; and to send the N sub-menu images to be displayed after the cutting process to the layer mixing circuit. The layer blending circuit is used to blend the N sub-menu images to be displayed with the main display image before displaying them.

8. The OSD image rendering system according to claim 1, characterized in that, The OSD image rendering system also includes: A compression circuit is used to compress the menu image to be displayed output by the processor and store the compressed menu image to be displayed in the first storage module. The display pipeline module includes: Layer blending circuit, decompression circuit; The decompression circuit is used to obtain the menu image to be displayed from the first storage module, decompress the menu image to be displayed, and transmit the decompressed menu image to the layer blending circuit. The layer blending circuit is used to blend the menu image to be displayed with the main display image before displaying it.

9. The OSD image rendering system according to claim 1, characterized in that, All registers in the display pipeline module are double-buffered registers.

10. The OSD image rendering system according to claim 1, characterized in that, The processor is disposed in the first electronic device, and the first storage module, the display pipeline module, and the second storage module are disposed in the second electronic device.

11. The OSD image rendering system according to any one of claims 1-10, characterized in that, The processor includes a GPU.

12. An OSD image rendering method, characterized in that, The method, applied to a processor of the OSD image rendering system as described in any one of claims 1-11, comprises: The rendering instructions and rendering resources required for rendering are obtained from the first storage module, and the rendering of the menu image to be displayed is completed to obtain the menu image to be sent for display. The image of the menu to be displayed is stored in the first storage module, so that the display pipeline module can obtain the image of the menu to be displayed from the first storage module and mix it with the main display image for display.

13. A chip, characterized in that, Includes the OSD image rendering system according to any one of claims 1-11.

14. An electronic device, characterized in that, Including the chip as described in claim 13.

Images