Pixel matrix controller

Abstract

Pixel matrix controller (10, 10', 10'', 10''') for controlling at least three pixel groups (22a, 22b, 22c), comprising: an image memory (14) for storing image data associated with the at least three pixel groups (22a, 22b, 22c); and a controller (12, 12') which is configured to read out the image data from the image memory (14) group-by-group and to control the at least three pixel groups correspondingly group-by-group, wherein the controller (12, 12') is configured to control the at least three pixel groups in a first and in a second order.

Description

[0001] Pixel matrix controller

[0002] Description

[0003] Embodiments of the present invention relate to a pixel matrix controller and to a display, in particular a microdisplay, a communication element and a corresponding method. Preferred embodiments relate to a programmable scanning device, in particular for ultra-flexible highspeed scanning control of a microdisplay. Further embodiments describe a special control of a row decoder of a display matrix. In general, embodiments are in the field of controlling a display, in particular a row decoder of a display matrix or pixel matrix in general, as also used in communication applications.

[0004] According to prior art, it is common for the rows of a display or microdisplay to be programmed sequentially row by row in a scanning process, i.e. from top to bottom or vice versa.

[0005] Microdisplays refer to displays with a typical display diagonal of 0.1" to typically 1.5", with some selected displays up to 2.5". Such microdisplays are usually stacked, i.e. the control circuit is located directly below (therefore also referred to as the backplane) the elements to be controlled (also referred to collectively below as the frontplane). Depending on the application and optical use of the microdisplay, these elements to be controlled can be a layer system for the realization of an organic LED, an inorganic LED or an LC material. Accordingly, we speak of an OLED microdisplay, a micro-LED microdisplay (also commonly referred to as uLED or pLED) or an LCOS (liquid crystal on silicon) microdisplay. The control circuit is adapted to the special properties of the element to be controlled. Examples include: precise electrical control in the typical voltage range of 3 V or higher, good current injection into the layer and measures for ageing compensation in OLED microdisplays, high current densities of > 1 A / cm2for uLED microdisplays or no static current flow into the element to be controlled in LCOS microdisplays. To realize an active matrix display, the backplane contains electrical circuit components such as transistors, capacitors or resistors. The same can be configured in different technologies. Examples include thin-film transistors (TFT such as IGZO, a-Si or LTPS) or the realization in single-crystal silicon. In the following, the further explanation will be based without restriction on single-crystal silicon with CMOS circuit technology (others are conceivable). Due to the ongoing progress of TFT miniaturization, however, a partial realization on TFT is also conceivable in the future. According to the prior art, microdisplays based on a silicon backplane are mainly realized in the technology nodes 0.25um...90nm. The vast majority of microdisplays are driven by a video stream with a constant refresh rate, i.e. for example with a video data stream at 30 Hz, 60 Hz, 90 Hz, 120 Hz or 240 Hz. In the following, this type of architecture is referred to as "refresh rate-based architecture". There are few control schemes that deviate therefrom - one example is the so-called "memory-based architecture", an architecture for the partial update of pixels (always describes the smallest element that can be controlled independently of the other). What all these architectures have in common so far is that, in addition to the control circuit for the optical elements, they contain as further circuit blocks a row and column decoder, possibly a DAC functionality for "translating" the digital pixel data into current and voltage values for the pixel and a block for realizing a display data interface (parallel interface, LVDS, SPI, etc.) depending on the above-mentioned control principle. This dependency results from the required data bandwidth: For example, a "refresh rate-based architecture" requires a much higher data rate, as all display pixels are constantly overwritten - even if this is not needed, for example, as the value of the pixel has not changed at all. A parallel interface (i.e. parallel digital, single-ended rows, which together transmit the data value of the pixel or several pixels, as well as additional control rows, also known as digital RGB) or serial transmissions (e.g. LVDS low-voltage differential signaling) are usually used here. Examples of this are for LCOS and for an OLED microdisplay. In such systems, quite complex, external video control electronics are required to realize a conversion to the mentioned interfaces. This is usually done using a separate integrated circuit or an FPGA. This increases the required power consumption as well as the installation space (and is therefore disadvantageous for the ergonomics of a system) - both factors that are of great importance in numerous mobile ("wearable") applications. For the "memory-based architecture", on the other hand, the transmission of the changing pixels is sufficient. Depending on the application, this considerably reduces the necessary transmission bandwidth, so that less complex interfaces, such as SPI, can be considered. In each of these cases, an adapted control circuit is required.

[0006] Typical control of microdisplay realizations uses a scanning method for row programming, i.e. the rows are programmed sequentially in ascending or descending direction. For this type of control, it is necessary for all rows to be rewritten and, above all, transmitted. The renewed control and data transmission costs resources such as energy and computing power, which has a negative impact on the overall efficiency of the microdisplay and, above all, the efficiency of the control. Energy efficiency is a crucial issue for microdisplays in particular. In other applications of a pixel matrix, the refresh rate plays a decisive role. There is therefore a need for an improved approach. The object of the present invention is to provide a concept for controlling a pixel matrix, e.g. a display, which offers an improved trade-off between efficiency, flexibility and display options, in particular a sufficient refresh rate.

[0007] The object is solved by the subject matter of the independent patent claims.

[0008] Embodiments of the present invention provide a pixel matrix controller for controlling at least three pixel groups, e.g. rows, of a display, in particular a microdisplay, or a pixel matrix in general. The pixel matrix controller includes an image memory for storing image data associated with the at least three pixel groups (rows) and a controller configured to read out the image data from the image memory group-by-group and to control the at least three pixel groups correspondingly group-by-group. The controller is configured to read out and control the at least three pixel groups (rows) in a first and a second order (e.g. 123 and 132, i.e. in a skipped order) or configured to selectively control the at least three pixel groups (image group), i.e. in a freely programmable order. According to embodiments, it is still conceivable that at least three or more orders, i.e. first, second and third orders (e.g. group 123 or 132 or 231 or 321) are used by the controller.

[0009] If three pixel groups are assumed, which primarily also rows (alternatively columns or freeform area), the first order can include control of the second of the at least three pixel groups directly after the first pixel group, while the second order comprises control of the third pixel group directly after the first. With at least four or more pixel groups, there are considerably more variation variants for the order, e.g. 1243, 1324, 3214, etc. Also, a pixel group can be omitted from a pass or a pixel group can be repeated in a pass. According to embodiments, it would also be conceivable that the control of the pixel groups is repeated. This free programmability can used in combination with shortening of the sequence runtime, e.g. by interleaving the control. The advantage of this control with the flexible order independent of the absolute number of pixel groups is that the free programmability of the scanning order of all rows created in this way enables a very high refresh rate from the integrated image memory, which reduces optical ripples of the pixels (in display applications). This increased refresh rate is particularly important for (optical) modulators for telecommunications or LIDAR, for example.

[0010] The embodiments of the invention describe an architecture of a freely programmable row decoder control, wherein the order of programming is determined by a sequence stored in a memory present in the system, but preferably integrated directly on the backplane, and can be individually adapted for the respective application. Embodiments of the present invention are thus based on the finding that by integrating a flexibly controllable program memory, e.g. in a backplane (or display), the flexibility, image reproduction quality and also energy efficiency or general efficiency can be considerably increased. Flexibly controllable means that a flexible scanning order of the rows of the pixel matrix is made possible during readout and control. The flexible scanning order is achieved by the option to selectively read out image data from an image memory and thus control individual pixel groups, i.e. individual rows group-by-group, for example. The result of this flexible control can be seen in the abandonment of a fixed order, i.e. in the possibility of at least two or more different (control) orders.

[0011] According to embodiments, the image memory is integrated directly into the backplane, e.g. the microdisplay, which means that the display matrix can be connected with a high bandwidth. Due to the almost arbitrary order during readout, it is possible to freely program the scanning order for all rows, which leads to a very high refresh rate from the integrated image memory, which reduces the optical ripple of the pixels (crucial in displays, but also in (light) modulators for telecommunications or LIDAR, for example). If, for example, at least three pixel groups, i.e. at least three rows, are assumed, the order can, for example, be defined in such a way that in one pass, in the first order, the two of the at least three pixel groups are controlled directly after the first of the at least three pixel groups, wherein the second order is defined by the fact that the third of the at least three pixel groups is controlled directly after the first of the at least three pixel groups. Of course, any other order is also conceivable; for example, one of the at least three pixel groups, e.g. the second, could be controlled directly one after the other, i.e. twice (e.g. row 1223), or one of the at least three pixel groups could not be controlled in one pass. If more than three pixel groups are assumed, e.g. four pixel groups or rows, the number of possible permutations (with and without repetition) for the order increases arbitrarily. Four or more pixel groups provide an embodiment in which the scalability of the approach becomes clear, i.e. several display rows are programmable or can be programmed independently of each other. According to embodiments, the controller is then configured to select further possible or all further possible orders. According to embodiments, the controller is configured to control the pixel groups repeatedly, e.g. with the same order as the previous order or with a new order when repeated. For example, if three pixel groups are assumed in the simplest embodiment, the second order can be used for the first repetition and the first order for the second repetition, so that the order varies from iteration to iteration. One or of the pixel groups can also be controlled twice or more often in each iteration or over several iterations. Advantageously, this enables previously unattainable frame rates for microdisplays in the kilohertz range.

[0012] According to embodiments, the controller is configured to transmit individual bits or image data associated with individual pixels sequentially to the respective one of the at least three pixel groups in order to control the respective one of the at least three pixel groups. In addition or alternatively, individual bits associated with individual pixels of the respective of at least three pixel groups can be transmitted individually to the respective of the at least three pixel groups in order to control the respective of the at least three pixel groups pixel by pixel (e.g. half row or only individual pixels). According to embodiments, the controller can also be configured to control at least two of the at least three pixel groups or at least two regions of the pixel matrix, to which the at least three pixel groups belong, with different resolutions or different resolutions of a pixel intensity or different refresh rates. The background to this is that the flexible control allows pixels, e.g. in the edge area, to be clustered (combined) and thus controlled together ("binning"). Consequently, the resolution at these positions is different, e.g. reduced. This means, for example, that a lower local resolution can be achieved at the edge regions than in the center. Similarly, individual pixels can be controlled with a different depth (4 bit vs. 10 bit in the brightness or gray value, for example). These two approaches have an advantageous effect on efficiency.

[0013] Further, according to embodiments, the controller can be configured to dynamically adapt the order of the pixel groups to be controlled at runtime, i.e. dynamic adaptation of the control with regard to local resolution, frequency and / or greylevel resolution.

[0014] According to embodiments, the pixel matrix controller can also comprise a command memory. Even if it is assumed in the above embodiments that the image memory and / or command memory is integrated in the backplane, it can also be arranged externally, i.e. locally as an external part of the pixel matrix controller.

[0015] For the control according to the flexible modes explained above, the pixel matrix controller can be configured to receive one or more commands from a group of commands comprising the following:

[0016] Programming command with selection of the pixel group;

[0017] Programming command with selection of the pixel group and with selection of the image memory address; - Programming command with selection of the pixel group with selection of the bit significance;

[0018] - Programming command with selection of the pixel group and selection of the pixel subgroup;

[0019] - Wait command;

[0020] - Program flow control

[0021] - Programming command for instruction-based control and / or for programmable control and / or for control in any order of the at least three pixel groups and / or for addressing control.

[0022] Based on this, the pixel controller then executes the commands, i.e. processes the same in order to control the image memory and / or the at least three pixel groups of the display accordingly.

[0023] According to a further embodiment, the controller can also be configured to control at least two of the at least three pixel groups simultaneously. This increases its complexity, but also makes it possible to further increase the frame rates of the microdisplay, which offers a significant advantage. This feature also contributes to free programmability.

[0024] In the above embodiments, it was primarily assumed that the pixel groups represent rows, as the scanning order is fixed in prior art applications. According to embodiments, exactly this scanning order for the rows is flexible. According to further embodiments, the pixel matrix can also be controlled column by column instead of row by row. In this embodiment, the pixel group would then be a column. Generally speaking, the at least three pixel groups can represent different regions.

[0025] According to a further embodiment, the control mode can also vary. For example, depending on the selected mode or the selected region, an amplitude-modulated conversion, a time- modulated conversion or a combined conversion of the image data can take place.

[0026] According to further embodiments, the pixel matrix controller comprises a frame controller configured to control all of the at least three pixel groups. According to embodiments, this frame controller can be configured to perform global brightness adjustment for all of the at least three pixel groups and / or to perform blanking and / or unblanking and / or to insert black images and / or to insert inverted images. This has the advantage that optical effects (e.g. motion thickness, motion blur) can be reduced, wherein no intervention in the external data pipeline is required, contrary to classic scanning architecture.

[0027] According to embodiments, an integrated, programmable solution is provided that enables flexible adaptation of the architecture to the frontplane and the application. This represents a significant competitive advantage, as this architecture typically requires a very small CMOS processor node, which is accompanied by significant NRE. Flexibility of architecture expands the market that can be reached with these products.

[0028] A further embodiment provides a method for controlling at least three pixel groups, e.g. at least three rows of a display, in particular a microdisplay. The method includes the steps of reading out image data associated with the at least three pixel groups from an image memory, wherein reading out takes place group-by-group; and controlling the at least three pixels group-by-group according to the image data, wherein the at least three pixel groups can be controlled in a first and in a second order.

[0029] According to further embodiments, the method can be computer-implemented. That is, a computer program is provided to perform the method when the program runs on a processor or controller as explained above.

[0030] Embodiments of the present invention will be explained with reference to the accompanying drawings. They show:

[0031] Fig. 1a a schematic block diagram of a pixel matrix controller according to a basic embodiment;

[0032] Fig. 1b a schematic block diagram of a pixel matrix controller (basic architecture) according to a further embodiment;

[0033] Fig. 1c a schematic flow diagram of a method for control according to an embodiment;

[0034] Fig. 2 a schematic representation of different commands for processing in a pixel matrix controller according to embodiments;

[0035] Fig. 3 a schematic representation of a pixel matrix controller according to extended embodiments (multiple simultaneous control of rows or pixel groups); Fig. 4a a schematic block diagram of a pixel matrix controller according to extended embodiments with a frame controller; and

[0036] Fig. 4b a schematic block diagram of a pixel matrix controller according to extended embodiments with a frame controller (multiple simultaneous control of rows or pixel groups); and

[0037] Fig. 5 a schematic table comparing different types of modulation to explain the advantages of embodiments.

[0038] Before embodiments of the present invention will be explained below with reference to the accompanying drawings, it should be noted that equal elements and structures are provided with the same reference numbers, so that the description of the same is interapplicable or interchangeable.

[0039] Fig. 1a shows a pixel matrix controller 10 including the two central components controller 12 and memory 14. In this embodiment, the controller 12 is a so-called row controller, which is configured to access the data memory or frame buffer 14. Both units 12 and 14 are connected to the display matrix 20. The display matrix here is, for example, an n column and m row matrix with n x m pixels. It is assumed that at least three pixel groups are provided here, here in the form of at least three rows 22a, 22b and 22c, which are controlled via the controller 12. The controller 12 is configured to retrieve image data associated with the at least three pixel groups from the image memory, i.e. to read them out, and to control the three rows 22a, 22b and 22c accordingly with the image data group-by-group. Typically, in the prior art, control is performed according to a predefined order, namely in a descending direction, i.e. 22a before 22b before 22c, or in an ascending direction, i.e. 22c before 22b before 22a. This means that the at least three pixel groups or pixel rows 22a-22c are displayed one after the other, i.e. successively, and then displayed again in the next iteration, wherein updating is possible. In the event display rows do not change, the same row is displayed again or further, wherein the data volume is nevertheless increased accordingly, as the same information is loaded again.

[0040] According to embodiments, it is now possible for the controller 12 to selectively access the memory 14 and thus select and read out image data for the at least three pixel groups (image groups) or one of the at least three pixel groups (image groups) and use these to control the associated one of the at least three pixel groups (image groups), e.g. 22a or 22b or 22c. This selectivity changes the order in which the pixel groups or 22a, 22b and 22c are controlled.

[0041] With the three rows shown here as an example, there is at least the possibility of control with at least two orders, i.e. with at least a first and a second order that differ from each other. It should be noted at this point that the order is not to be regarded as rigid, but can vary from iteration to iteration. Individual controls of pixel groups can also be omitted or repeated several times during an iteration. It is therefore possible to individually control each of the at least 3 pixel groups and thus only update the pixel groups for which an update is necessary. Vice versa, this means that rows that do not change do not necessarily have to be reloaded and controlled, but can simply continue to be displayed. For other rows or pixel groups 22a-22c, the free resource can be used to increase the refresh rate. The result is a flexible scanning order of rows 22a-22c of the microdisplay 20 shown here with the advantage of high frame rates, e.g. in the kilohertz range. The high refresh rates can reduce optical ripples in the pixels.

[0042] Fig. 1c shows the underlying method 100. The method 100 comprises the two basic steps 110 and 120. In the first step 110, for example, the one or more image data are read out from the memory 14 by the controller 12 in order to then control the at least three pixel groups 22a-22c group-by-group. As explained above, the control is carried out in a freely programmable manner, so that the controller is configured to control at least two different orders of the control of the at least three pixel groups with the rows 22a-22c as a result.

[0043] With reference to Fig. 1b, an extended variation of the controller 10, namely the controller 10', is explained below. Compared to the controller 10, the same also comprises a program memory 13, which is directly connected to the controller 12. Before the configuration of the variant 10' shown here will be explained in detail, the mode of operation of the individual elements 12, 13, 14 and 20 will be discussed. The frame buffer 14 can be realized in the microdisplay-integrated memory and includes the image data of all or some of the pixels. The data is written to the memory via an external interface (e.g. HDMI, DP, MIPI, USB, LVDS, etc.) or created on the backplane itself (e.g. by rendering a GPU).

[0044] One embodiment defines a display or microdisplay with a corresponding controller 10 (12+14), wherein the display comprises the matrix 20 and is controlled by the controller 12. A further embodiment relates to the transmission of these data from the image memory to the pixel matrix.

[0045] The controller, in this case row controller 12, comprises s a row controller program memory

[0046] 13. The program memory 13 contains the instructions that describe the programming (or display) order. The row controller 12 evaluates the instructions accordingly from the program memory and controls both the frame buffer (i.e. selection of the image data of a row or part of a row requested by the instruction) and the selection of the pixel row to be programmed. It also ensures the consistency of the data from the frame buffer to the selected pixel row (for example, by requesting the data from the image memory in an offset manner, reading out should only be possible via one data pipeline).

[0047] As can be seen here, the row controller 12 directly controls the display matrix 20, wherein the data from the frame buffer 14 are output directly to the display matrix 20. For this purpose, a control connection is provided between the row controller 12 and the frame buffer

[0048] 14, which directly addresses the data stored in the memory 14 for transmission to the matrix 20. Based thereon, the frame buffer transmits the data to the display matrix, wherein the display matrix writes the data to the corresponding row of the display matrix by selecting the respective row using the row controller 12.

[0049] The pixel matrix is preferably the pixel matrix of a microdisplay. Typically realized pixel sizes are in the range of 4.3 pm x 4.3 pm4or 6.3 pm x 2.1 pm5or 9.3 pm x 3.1 pm6, i.e. generally in the range of 2 pm to 9.5 pm or 1 pm to 10 pm or 1 pm to 15 pm pixel dimension. The second pixel dimension can be identical or slightly different, i.e. square and rectangular.

[0050] The aim of the arrangement is to program the pixels of the pixel matrix with a freely programmable control and therefore variable control order. This includes writing the pixels with the entire data word of the pixel values to be displayed as well as parts thereof. In the most consequent case, the latter only includes the programming of a single bit of the data word, wherein the individual bits are programmed into the pixel sequentially and possibly with different time weightings (and the weighting can be programmed dynamically) and the mapping of the value results as an average value over time. Mixed forms then contain several bits, wherein these can then also be amplitude-modulated, wherein only the individual parts of the real data value are time-modulated.

[0051] According to embodiments, all four of the aforementioned components 12, 13, 14 and 20 are integrated on the microdisplay backplane (for microdisplays, this is usually a circuit realized in CMOS) and can thus be combined in one system in an extremely compact and energy-saving manner.

[0052] According to embodiments, the image memory and the controller can be realized as a common backplane of a pixel matrix and / or the image memory, the controller and the pixel matrix can be realized as connected components.

[0053] According to embodiments, free programmability of the transfer of the image content from the image memory 14 to the pixel matrix 20 is created. This is realized by a program memory 13, which can contain different instructions that are interpreted and executed by the row controller 12. The basic structure of such an instruction consists of an instruction code and a data region, which implements different functions depending on the instruction. Some embodiments of these instructions and their inventive function / effect are described below (cf. also Fig. 2):

[0054] Individual exemplary commands are explained below with reference to Fig. 2. Fig. 2 shows a collection of six instruction commands (exemplary structure) as they are stored in the row control program memory; these are numbered with reference numbers 31 , 32, 33, 34, 35, 36. The general structure provides for the instruction to begin with a command code, which allows different instructions to be distinguished (examples of possible row instructions are given below). The commands can comprise several parts, as is made clear by the addition of the reference number supplements ...a, ...b, ...c.

[0055] • Scan programming command of a row 32: In addition to the coding of the command 32a itself, the instruction only contains the row 32b of the pixel matrix to be programmed. The row controller 12 implements this instruction by requesting the corresponding data in the image data memory. As soon as the same are applied to the pixel matrix, the row decoder is also addressed by the row controller 12 with the corresponding address, wherein the pixel row matching the data is selected and the data are transferred to the pixels. The entire row can be programmed with a single programming operation or, if the bandwidth of the output side of the image memory is limited, only parts of a row can be programmed. In the latter case, the programming of the entire row is realized by sequential programming of the individual sections. In addition, the architecture enables a row-specific repetition rate via the programming, i.e. individual regions of the pixel matrix up to individual rows can provided with an individual repetition rate. Scan programming command of a row with selection of the image memory address 33: In this embodiment, in addition to the matrix row of instruction 33a, the row address 33b and the data address 33c of the associated data are added in the image memory. This enables a location-dependent influencing of the image content such as a combination of pixels, i.e. a local resolution reduction or generally a local resolution influencing or a location-dependent influencing of the resolution of the intensity gradation (possibly in combination with the instruction described below specifying the bit significance of the data word) of the pixel value or a combination of both. These possibilities can be used to advantage for so-called "foveated rendering" (reduced rendering effort due to a reduction in image quality in the peripheral vision area, as the human eye is less sensitive to such effects in this area).

[0056] Scan programming command of a row with selection of bit significance 34: In addition to programming the complete data of an image row, it can also be necessary to transfer only individual data bits from the image memory to the pixel matrix. In this embodiment, apart from the instruction 34a, the row address 34b and the desired bit significance (bit plane) 34c for addressing the frame buffer are added to the command. This is particularly advantageous if the pixels in the pixel matrix can only receive a single bit or parts (for example, when realizing particularly small pixels) and the total pixel value is realized via the time weighting of the individual bits (also referred to as bitplanes in their entirety) in the pixel. In this case, the instruction is extended by specifying the bitplane number, i.e. the respective bitplane significance (e.g. for a data value of 8 bits, this means 8 bitplanes and thus a bitplane number from 1 to 8 or 0 to 7). In this embodiment, it is also possible to program the entire row or parts of a row. In addition, this command, in combination with the possibility of individual row-specific repetition rates implemented by the architecture according to the invention, can be used to realize different areas of the pixel matrix with different time-averaged bit depths.

[0057] Scan programming command of a row with selection of the pixel group 35: In some applications, different pixels are assigned different functions. The simplest example of this is the embodiment of a red, green and blue pixel (further example YU V or CYM , generalized pixel groups). If the same are to be transmitted sequentially and / or independently of the pixel matrix or the selected row, the selection of the desired pixel group is also part of instructions 35a, 35b, 35c. • Wait command 36: The program memory is processed, e.g., by continuously increasing the address. Depending on the desired programming of the pixel matrix, a time weighting can require that the time until the desired time weighting is reached must first be waited for. This requires the instruction wait command 36, which includes the number of cycles to be waited for as a parameter. Non-continuous address processing would also be possible, namely through program flow control.

[0058] • Program flow control: In addition to a wait command, other complex instructions for program flow control are also conceivable, such as program end, jump, loop, wait for external event, jump to external event or similar.

[0059] It is obvious that further application-specific complex instructions result from the above examples, such as the selection of the bitplane of a certain color or the selection of a certain row for a certain time. Further expansion stages of this architecture include an adapted CPU-like implementation of the row controller or an architecture based on RISC (reduced instruction set computer), CISC (complex instruction set computer) or VLIW (very large instruction word).

[0060] According to further embodiments, it would be conceivable that the controller and the image memory are configured to control two or more groups 22a-22c, that is, e.g. two rows, simultaneously. The background to this is that in small CMOS process nodes, the arrangement described above can be highly parallelized (i.e. output of a complete pixel row from the integrated frame buffer) and implemented with high frequency. Depending on the embodiment, the refresh rate that can be realized from the image memory to the pixels can be increased many times over and thus allows very high modulation frequencies (for example 240 Hz, 480 Hz, 1 kHz, 5 kHz, 10 kHz, 20 kHz, 50 kHz, 100 kHz and more), which are far higher than those of conventional video microdisplays (typically 60 Hz, 120 Hz or 240 Hz). This is made possible in particular by the free programmability of the row scanning order and the connection of the pixel matrix to the frame buffer with a high bandwidth, as updating of the individual rows can be interleaved, the number of waiting cycles required can be minimized and thus the refresh rate can be increased.

[0061] According to further embodiments, the architecture allows further scaling. If the writing of a single pixel row requires too long a row time for the desired refresh rate, the architecture can be extended to include the option of programming several rows simultaneously (implementation of parallelization, cf. Fig. 3).

[0062] Fig. 3 shows a controller 10" of a scaled architecture for several (simultaneously) scanned rows. The controller 10" is essentially comparable to the controller 10', i.e. the same comprises the components 13, 14 and 20. In contrast to the controller 10', however, the controller 10" comprises the controller 12', which is configured to select several of the m rows and to control the display matrix 20 simultaneously with the same. In this case, several rows are encoded in the instruction or several adjacent rows can be combined to form a common pixel group, which is then programmed together. In addition, different parallel programmed row groups at different positions of the pixel matrix are conceivable as a further expansion stage. Combining several rows into a pixel group shortens the instruction length (for example, if 4 consecutive rows are always combined), as only the base address of the group of 4 has to be coded in the instruction and not the individual addresses of all 4 rows. These expansion stages can reduce the effective number of required row programming operations and thus further increase the refresh rate. According to the invention, this further expansion stage can be combined with the instruction types explained above.

[0063] The previous prior art embodiments only include the cyclical processing of the instructions of the row controller program memory. However, if a global weighting of the row programming is to be realized, an implementation at the level of the row program memory is quite complex. The arrangement according to the embodiments can therefore be extended by a higher-level frame controller, which can be synchronized with the row control via the "frame" synchronization signal. The sync signal is provided with the reference number 29.

[0064] Fig. 4a shows an expanded architecture extended by an (optionally) programmable frame controller 27, which enables global modulation of the signals. The higher-level frame controller 27 is implemented as part of the controller 10'". The controller 10'" in turn comprises the individual controller 12, in this case the row controller 12, as well as the memory 14 for controlling the display matrix 20. In addition, the row controller memory 13 and a frame controller memory 28 are also provided in this embodiment. The frame controller 28 is arranged between the row controller 12 or memory 14 and the display matrix 20.

[0065] Unlike the row controller 12, the frame controller 28 does not modulate individual rows 22a- 22c, but complete images. For example, a global brightness setting can be realized by blanking entire frames. In addition, blanking can also be used to reduce visually perceived effects such as motion blurring. In current displays, this blanking has to be realized by external data provision, which requires a higher external data rate, including the associated disadvantages of increased power consumption due to recharging higher power capacities in the system electronics, increased circuit complexity in the electronics, lower maximum line length and sensitivity to interference. This disadvantage can be avoided by the implementation as embodiment of the discussed architecture, as the frame controller uses the information, e.g., on blanking (data is read once and kept constant; this enables lower energy consumption). In the most consistent embodiment, reading out the data from the memory can be replaced by constant data to further reduce the power consumption in the case of blanking. The frame controller 28 can also perform other functions such as image inversion or unblanking. The frame controller realizes a time-controlled modification of the row selection and / or the pixel data of the frame buffer. If complex sequences of the functions of the frame controller 27 are to be implemented, the same can be provided with a program memory 28 in the same way as the row controller 12. This enables instruction-based, application-specific programming in a similar way to the row controller program memory 13.

[0066] Fig. 4b shows a further embodiment of the architecture extended by an (optionally) programmable frame controller 27, wherein the same is configured for the simultaneous control of several rows (cf. control between control 12 and controller 27 or between memory 14 and controller 27 as well as control between controller 27 and matrix 20). Otherwise, the embodiment in Fig. 4b corresponds to the embodiment in Fig. 4a.

[0067] A further embodiment provides for the implementation of both a separate row controller programming memory 13 and a frame controller programming memory 28 for individual groups of pixels in order to realize several individually programmable pixel groups.

[0068] According to embodiments, this means that a pixel matrix can be divided into groups of pixels. For example, the pixel matrix can comprise 2, 3 or 4 or any number >1 groups, each of which comprises its own controller 12 with optional programming memory 13 and frame memory 14. Above that, each of these groups can also comprise its own frame controller 27 with optional programming memory 28 or a frame controller 27 shared between the groups. In addition to the local structuring in pixel groups in the sense of rows or columns, the pixel matrix can also have several differently colored elements (e.g. RGB) per pixel. This means that the pixel matrix controller can be configured multiple times for sub-areas and sub-groups It should be noted that embodiments can also include an interchange of rows and columns.

[0069] This free programmability can be used in combination with shortening of the sequence runtime, e.g. by interleaving the control. The background is explained with reference to Fig. 5.

[0070] Fig. 5 shows a table comparing different modulation schemes with different tradeoffs between display accuracy and sequence runtime or refresh rate to explain the possible shortening of the sequence runtime and the associated increase in refresh rates.

[0071] The internal refresh rate is mainly defined by the length of the scanning sequence, i.e. the number of commands and wait states in the command program memory. To emphasize this effect together with the capabilities of the backplane to drive different schemes based on the flexible command memory architecture, three typical scanning schemes for display applications are explained:

[0072] PWM: Pulse width modulation is a well-known approach to approximating an analog value by switching between two digital states. There are many different types of PWM, but the binary weighted bit plane scheme (one for each bit of the tone scale resolution, from MSB to LSB) is the most common in display applications. It goes hand in hand with a very good representation of the digital data value. However, the entire sequence has to be relatively long to avoid pixel row programming conflicts at the LSB bit level. In order to maintain correct values, many wait states have to be introduced, especially in the MSB bit level. This considerably reduces the update rate that can be realized. This is the motivation for the following two schemes / approaches.

[0073] Shortest scan: (Yuan Ji et al. "Optimal scan strategy for mega-pixel and kilo-gray- level OLED-on-silicon microdisplay". Applied Optics 51.17 (June 2012), p. 3731. doi: 10.1364 / ao.51.003731): This scanning scheme aims for the shortest possible sequence by using only consecutive programming commands. Wait states are only implemented to fill the pattern. In this sense, the optimal scanning approach follows the idea of the shortest possible pattern and thus a maximum internal update rate, although considerable errors in the mapping of the values have to be accepted.

[0074] Hopping scan: (Mutsumi Kimura et al. "Time-Ratio Grayscale and Hopping Scan with Current Uniformization for Thin-Film Transistor Driven Organic Light-Emitting Diode Displays". In: Japanese Journal of Applied Physics 45.5B (2006), pp. 4407-4412. doi: 10.1143 / JJAP.45.4407): This approach attempts to strike a balance between the two previous schemes by accepting a slightly longer sequence, but also errors are kept within reasonable limits.

[0075] Further details on the scanning length and error parameters can be found in the table in Fig. 5.

[0076] One embodiment provides a display, e.g. microdisplay with one of the (integrated) pixel matrix controllers explained above. Possible technical realization here is OLED.

[0077] A further embodiment relates to a communication element for transmitting data using a pixel matrix and one of the (integrated) pixel matrix controllers explained above. The communication element can include, for example, the pixel matrix and the pixel matrix controller. Possible examples of a communication element are satellite communication stations or satellite communication base stations, an optical multiplexer. Possible technical implementation here is LCD or microLED.

[0078] Another embodiment relates to an optical multiplexer or LIDAR element with pixel matrix and the pixel matrix controller. The communication element or multiplexer and the LIDAR element benefit in particular from the high "image" refresh rate.

[0079] Features or optional features discussed in connection with the microdisplay are also transferable to the applications just explained: communication element, multiplexer or LIDAR element, which also use pixel matrices.

[0080] According to a further embodiment, a microdisplay, such as the display of a wearable, a microdisplay for use in augmented reality (AR), for use in virtual reality (VR), for use in mixed reality (MR) is used. According to a further embodiment, a light modulator is provided for the purposes of holography, telecommunications, optical data transmission, LIDAR technology or space technology, etc.

[0081] According to embodiments, the microdisplay architecture is characterized in that the pixel data is read out from an integrated image memory, wherein the programming order and mode of the individual rows or pixel groups is determined by flexibly programmable instructions from a likewise integrated row controller program memory or pixel group controller program memory and is possible in any order.

[0082] According to embodiments, individual bits of the pixel data of the individual pixels can be transferred to the pixel matrix sequentially (in any programmable order of rows).

[0083] It should be noted that, according to embodiments, several bits (or all bits) of the pixel data can be transferred simultaneously to the corresponding pixel of the pixel matrix sequentially (in any programmable order of rows).

[0084] According to embodiments, two, more or all pixels of all rows can be programmed with a single or all bits.

[0085] According to an embodiment, the architecture is extended so that not only a single row, but several rows can be programmed simultaneously and individual, several or all pixels of these selected rows are programmed.

[0086] According to an embodiment, the programming is configured so that different areas of the pixel matrix are programmed with a different local resolution or / and a different resolution of the pixel intensity value

[0087] According to an embodiment, the programming is configured in such a way that different areas of the pixel matrix are programmed with different refresh rates.

[0088] According to embodiments, the programming of the different local resolutions and / or different resolutions of the pixel intensities can be adapted at runtime. According to embodiments, it would be conceivable that parts of the data are implemented amplitude- modulated and parts of the data are implemented time-modulated via the programmable row program memory in the pixel.

[0089] It should be noted at this point that, according to embodiments, the pixel matrix controller is comparable to a CPU (computer processing unit), GPU (graphics processing unit) or DSP (digital signal processing unit) in terms of complexity. The complexity can be assessed, for example, on the basis of the number of transistors, the number of computing operations that can be carried out in parallel or the scope of the instruction set. The pixel matrix controller explained here according to embodiments is more similar to a CPU, GPU or DSP, such as an ARM core, than to a purely instruction-based controller, as is otherwise typical for pixel matrix elements. According to embodiments, the pixel matrix controller can be configured to execute at least two or more processes simultaneously and / or to process image data. The GPU can also be configured to decide which of the at least three pixel groups are controlled at a high repetition rate (high refresh rate or update rate) and which are controlled at a comparably slower rate. This can happen depending on information or states. The control can also be generally selected depending on this information or states. This means that, according to embodiments, it is conceivable that the pixel matrix controller determines which of the at least three pixel groups is controlled based on further information or states, such as temperature states. According to embodiments, the determination is made at runtime. Further, the pixel matrix controller can also determine at runtime which row is updated and which is not.

[0090] According to an embodiment, a frame controller can also be provided, which is configured to insert dark images, light images or inverted images into the modulation. According to an embodiment, the modulation can be programmed differently for individual pixel groups by implementing individual cell controller programming memories and / or image controller programming memories in the architecture.

[0091] Although some aspects have been described in the context of an apparatus, it is obvious that these aspects also represent a description of the corresponding method, such that a block or device of an apparatus also corresponds to a respective method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or detail or feature of a corresponding apparatus. Some or all of the method steps may be performed by a hardware apparatus (or using a hardware apparatus), such as a microprocessor, a programmable computer or an electronic circuit. In some embodiments, some or several of the most important method steps may be performed by such an apparatus.

[0092] A signal encoded according to the invention, such as an audio signal or a video signal or a transport stream signal, can be stored on a digital storage medium or can be transmitted on a transmission medium, such as a wireless transmission medium or a wired transmission medium, e.g. the Internet.

[0093] Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a Blu-Ray disc, a CD, an ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, a hard drive or another magnetic or optical memory having electronically readable control signals stored thereon, which cooperate or are capable of cooperating with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable.

[0094] Some embodiments according to the invention include a data carrier comprising electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0095] Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer.

[0096] The program code may, for example, be stored on a machine readable carrier.

[0097] Other embodiments comprise the computer program for performing one of the methods described herein, wherein the computer program is stored on a machine readable carrier. In other words, an embodiment of the inventive method is, therefore, a computer program comprising a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0098] A further embodiment of the inventive method is, therefore, a data carrier (or a digital storage medium or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier, the digital storage medium, or the computer-readable medium are typically tangible or nonvolatile.

[0099] A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may, for example, be configured to be transferred via a data communication connection, for example via the Internet.

[0100] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0101] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0102] A further embodiment in accordance with the invention includes an apparatus or a system configured to transmit a computer program for performing at least one of the methods described herein to a receiver. The transmission may be electronic or optical, for example. The receiver may be a computer, a mobile device, a memory device or a similar device, for example. The apparatus or the system may include a file server for transmitting the computer program to the receiver, for example.

[0103] In some embodiments, a programmable logic device (for example a field programmable gate array, FPGA) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware apparatus. This can be a universally applicable hardware, such as a computer processor (CPU) or hardware specific for the method, such as ASIC.

[0104] The above described embodiments are merely illustrative for the principles of the present invention. It is understood that modifications and variations of the arrangements and the details described herein will be apparent to others skilled in the art. It is the intent, therefore, that the invention is limited only by the scope of the appended claims and not by the specific details presented by way of description and explanation of the embodiments herein.

Claims

Claims1. Pixel matrix controller (10, 10', 10", 10"') for controlling at least three pixel groups (22a, 22b, 22c) of a pixel matrix (20), in particular a pixel matrix (20) of a display or a microdisplay, comprising: an image memory (14) for storing image data associated with the at least three pixel groups (22a, 22b, 22c); and a controller (12, 12') configured to read out the image data from the image memory (14) group-by-group and to control the at least three pixel groups correspondingly group-by-group, the controller (12, 12') being configured to control the at least three pixel groups in a first and in a second order.

2. Pixel matrix controller (10, 10', 10", 10'") according to claim 1 , wherein the controller (12, 12') is configured to control the at least three pixel groups in a third order; and / or wherein the first order defines that the second of the at least three pixel groups (22a, 22b, 22c) is controlled directly after the first of the at least three pixel groups (22a, 22b, 22c), wherein the second order defines that the third of the at least three pixel groups (22a, 22b, 22c) is controlled directly after the first of the at least three pixel groups (22a, 22b, 22c).

3. Pixel matrix controller (10, 10', 10", 10'") according to any one of the preceding claims, wherein the pixel matrix controller (10, 10', 10", 10'") is configured to control at least four pixel groups (22a, 22b, 22c).

4. Pixel matrix controller (10, 10', 10", 10'") according to any one of the previous claims, wherein the controller (12, 12') is configured to sequentially transmit individual bits of the image data associated with individual pixels to the respective one of the at least three pixel groups (22a, 22b, 22c) to control the respective one of the at least three pixel groups (22a, 22b, 22c), or to transmit individual bits associated with individual pixels of the respective one of at least three pixel groups (22a, 22b, 22c) individually to the respective one of the at least three pixel groups (22a, 22b, 22c) in order to control the respective one of the at least three pixel groups (22a, 22b, 22c) pixel by pixel.

5. Pixel matrix controller (10, 10', 10", 10"') according to any one of the preceding claims, wherein the controller (12, 12') is configured to control at least two of the at least three pixel groups (22a, 22b, 22c) or at least two regions of the pixel matrix (20) with different local resolution and / or different resolution of a pixel intensity value and / or different update rate.

6. Pixel matrix controller (10, 10', 10", 10'") according to any one of the preceding claims, wherein the controller (12, 12') is configured to dynamically adapt the order of the pixel groups to be controlled at runtime.

7. Pixel matrix controller (10, 10', 10", 10'") according to any one of the previous claims, wherein the pixel matrix controller (10, 10', 10", 10'") is configured to receive one or more commands from a group of commands, and to process the same in such a way as to control the image memory (14) and / or the at least three pixel groups (22a, 22b, 22c) of the display accordingly, wherein the group of commands comprises the following:Programming command with selection of the pixel group (22a, 22b, 22c);Programming command with selection of the pixel group (22a, 22b, 22c) and with selection of the image memory address;Programming command with selection of the pixel group (22a, 22b, 22c) with selection of the bit significance;Programming command with selection of the pixel group (22a, 22b, 22c) and selection of the pixel subgroup;Wait command;Program flow control;Programming command for instruction-based control and / or for programmable control and / or for control in any order of the at least three pixel groups and / or for addressing control.

8. Pixel matrix controller (10, 10', 10", 10"') according to any one of the previous claims, wherein the pixel matrix controller (10, 10', 10", 10'") is configured to have a CPU / GPU / DSP comparable complexity or to have a number of transistors or functional elements or implemented commands comparable to a GPU, CPU, DSP; and / or wherein the pixel matrix controller (10, 10', 10", 10'") is configured to perform several processes in parallel and / or to process image data and / or is configured to update individual ones of the at least three pixel groups (22a, 22b, 22c) at a higher rate than other pixel groups and / or performs the control in dependence on further information and / or in dependence on further states and / or is configured to determine at runtime which of the at least three pixel groups (22a, 22b, 22c) is updated.

9. Pixel matrix controller (10, 10', 10", 10'") according to any one of the preceding claims, wherein the image memory (14) and the controller (12, 12') are realized as a common backplane of a pixel matrix or the image memory (14), the controller (12, 12') and the pixel matrix are realized as a connected component.

10. Pixel matrix controller (10, 10', 10", 10'") according to any one of the previous claims, wherein the controller (12, 12') is configured to control at least two of the at least three pixel groups (22a, 22b, 22c) simultaneously.

11. Pixel matrix controller (10, 10', 10", 10'") according to any one of the previous claims, wherein the at least three pixel groups (22a, 22b, 22c) define regions of the pixel matrix (20); and / or wherein the at least three pixel groups (22a, 22b, 22c) comprise rows of the pixel matrix (20) or columns of the pixel matrix (20); wherein the at least three pixel groups (22a, 22b, 22c) comprise several regions, columns, rows or several consecutive regions, columns, rows.

12. Pixel matrix controller (10, 10', 10", 10'") according to any one of the preceding claims, wherein the controller (12, 12') is configured to convert the image data in an amplitude-modulated and / or time-modulated manner depending on the selectedmode and depending on the region.

13. Pixel matrix controller (10, 10', 10", 10"') according to any one of the preceding claims, further comprising a frame controller (27) configured to control all of the at least three pixel groups (22a, 22b, 22c).

14. Pixel matrix controller (10, 10', 10", 10'") according to claim 13, wherein the frame controller (27) is configured to perform global brightness adjustment for all of the at least three pixel groups (22a, 22b, 22c) and / or to perform blanking and / or to perform unblanking and / or to insert black images and / or to insert inverted images.

15. Pixel matrix controller (10, 10', 10", 10'") according to claim 13 or 14, wherein the frame controller (27) is connected to a frame controller programming memory (28) or comprises a frame controller programming memory (28).

16. Pixel matrix controller (10, 10', 10", 10'") according to any one of the preceding claims, wherein the pixel matrix controller (10, 10', 10", 10'") is configured multiple times for partial regions and partial groups.

17. Communication element, optical multiplexer or lidar element comprising a pixel matrix controller (10, 10', 10", 10'") according to any one of the preceding claims.

18. Display, in particular microdisplay with a pixel matrix controller (10, 10', 10", 10'") according to any one of claims 1-16.

19. Method for controlling at least three pixel groups (22a, 22b, 22c) of a pixel matrix (20), in particular a pixel matrix (20) of a display or a microdisplay, comprising: reading out image data associated with the at least three pixel groups (22a, 22b, 22c) from an image memory (14), the reading out being performed group-by-group; and controlling the at least three pixel groups (22a, 22b, 22c) group-by-group according to the image data, wherein the at least three pixel groups (22a, 22b, 22c) can be controlled in a first and in a second order.

20. Computer program for carrying out the method according to claim 19 when the program runs on a processor and / or a controller (12, 12').

Images