Method and apparatus for secure display of asil-related data on a motor vehicle display device

Abstract

The invention relates to a method for displaying ASIL-dependent data on a motor vehicle display device, the motor vehicle comprising a transmitting unit, a receiving unit and a display unit for displaying an image generated by the receiving unit. According to the invention, images having a higher safety classification are displayed on the display unit in a manner modified less in the course of image improvement than images having a lower safety classification. In the method, information about the safety classification of the image to be displayed is added to the data stream by a binary coding of the classification in the color bit information of at least one pixel of the image to be displayed via a pixel matrix by means of the transmitting unit in order to control the change of the image for images of each safety classification in the course of an image improver assigned to the receiving unit for outputting image data.

Description

Technical Field

[0001] This invention relates to a method for safely displaying ASIL-related data on a motor vehicle display device, wherein display content with a higher safety rating is displayed by the display unit in a manner that requires less modification through an image enhancement process compared to display content with a lower safety rating, and the invention also relates to an apparatus for performing the method. Background Technology

[0002] DE 10 2019 205 237 A1 discloses a method for displaying ASIL-D information using a low-security device. Here, data with different security classifications (none or ASIL A, B, C, D) are processed differently, i.e., compared to display content with lower classifications, display content with higher classifications is displayed by the display unit with minimal or no modification (brightening, color optimization, etc.).

[0003] DE 10 2015 200 292 A1 describes a method for safely displaying ASIL-related data on a motor vehicle display device. Safety-critical data and non-safety-critical data are processed separately and independently to produce non-safety-critical display elements and safety-critical display elements. The safety-critical display elements are displayed graphically overlaid with the non-safety-critical display elements in a display state using a display device. Safety-critical image data and non-safety-critical image data can be read separately and additionally or alternatively independently. Furthermore, the safety-critical display elements and non-safety-critical display elements can be output separately and additionally or alternatively independently. Summary of the Invention

[0004] The purpose of this invention is to provide a method and apparatus for safely displaying ASIL-related data on a motor vehicle display device, the ASIL-related data providing more accurate and detailed information that allows for the inference of the accurate display content of the image to be displayed.

[0005] The features of this invention are derived from the independent claims. Advantageous modifications and designs are the subject of the dependent claims. Other features, applications, and advantages of this invention are derived from the following description and explanation of the embodiments of the invention shown in the figures.

[0006] This objective is achieved using the method explained at the beginning: information regarding the security classification of the image to be displayed is added to the data stream through binary encoding in the color bit information of at least one pixel of the image to be displayed via the pixel matrix. This controls, i.e., minimizes or completely prohibits, changes to the security-related display content of the image during the image enhancement process. Binary encoding provides information beyond the color information at the pixel level. From this, the type or form of each transformed pixel can be inferred, allowing for the accurate retrieval of the image's displayed content. Pixel-level metadata information is also added to the bit-level image information without distorting the original image signal. The pixel-by-pixel parsed image data is then displayed to the user according to its initial structure.

[0007] Advantageously, the image data generation unit adds binary encoding to the color bit information and transmits the color bit information data stream to the image data output unit. The image data is rendered in the rendering unit, and classification binary encoding is added corresponding to the security classification of the image data. By classifying the image data beforehand within the rendering unit when creating the image, misprocessing of the image data by the image data output unit and its associated image improver is avoided. An image improver refers to a software module or device with such a software module known from the prior art, which optimizes an image using image improvement techniques, such as regarding color, contrast, brightness, sharpness, resolution, and noise, before displaying the image on a display device. The term "image" in the sense of this disclosure includes all types of display content on a display device, i.e., symbols, images, text, or animations.

[0008] In one design, before transmitting the image to be displayed to the display unit, at least one pixel containing a classification binary code is read from the data stream received by the unit that generates and outputs image data. Based on the classification binary code of the at least one pixel, a determination regarding an image improvement implementation is made and implemented accordingly before transmission to the display unit. By separating information about the image data from the binary code, precise evaluation of the binary code is possible, allowing for accurate identification of the security classification of the image to be displayed before display.

[0009] In one variation, the read classification binary code of at least one pixel is replaced by the original color value bits stored in the pixel before the binary encoding, wherein the image to be displayed is modified during the image improvement process according to its classification binary code. The image improver, having received instructions on how to perform the image improvement process after reading the classification binary code, subsequently replaces the classification binary code with the original color value bits stored in the pixel before the addition of the binary encoding, before implementing the image improvement process. Advantageously, the color values ​​of the image provided to the image improver correspond exactly to the color values ​​of the image before the addition of the classification binary code. Color pixels classified as safety-related "ASIL" meet the requirements for the specified safety classification without quality loss, and pixels classified as non-critical can be fully optimized during the image improvement process.

[0010] In one implementation, the color values ​​of the image to be displayed, output after the image enhancement process, are checked to see if they correspond to a predetermined security level defined by the associated binary encoding. For this purpose, the image generated by the rendering unit before binary encoding is added is compared with the image to be displayed. In particular, this comparison reliably identifies errors or deviations for images with high ASIL ratings (images whose images before binary encoding are necessarily identical to those sent from the receiving unit to the display unit), and may output a security alarm or initiate auxiliary measures. This provides a redundancy through which the security level of the image to be displayed is checked again.

[0011] In another design, a bit is reserved in a predetermined color of the RGB information for binary encoding. This results in two classification states.

[0012] In an alternative approach, one pixel for each color of the RGB information is reserved for binary encoding. This increases the number of classification states.

[0013] The number of classification states can be further increased by reserving one bit alternately in the color RGB bit information for each new data frame for binary encoding. Furthermore, if the classification bit fails to alternate between the color value bit information, the image data output unit identifies an image freeze.

[0014] Alternatively, for binary encoding, additional LSB information is provided in a separate image with a predetermined color depth. This allows for the addition of any number of pixels for each color, thus enabling a wide variety of classification possibilities.

[0015] In another design, supplementary channels for color encoding are used as metadata in the binary encoding for ASIL classification. These supplementary channels are, for example, one or more bits predetermined for transparency in RGBA color encoding, or one or more bits predetermined for the white component in RGBW color encoding. The supplementary channel comprises multiple bits or one byte. The supplementary A channel (α) and W channel for each corresponding color encoding are, in principle, supplemented with a fourth byte for each of the three color bytes used for R, G, and B, in which the classification binary encoding is added. RGB data is transmitted in a compressed format to reduce the amount of data transmitted in the data channel from a transmitting unit, for example, designed as a rendering unit, to a receiving unit for processing and outputting the image. Advantageously, the bytes for the supplementary channels, i.e., also for the A or W channels, are not compressed, thus eliminating the possibility of erroneous classification binary encoding output due to compression or decompression errors.

[0016] Another aspect of the invention relates to an apparatus for displaying ASIL-related data on a vehicle display device. The apparatus includes an image data generation unit and an image data output unit communicating via a data stream, wherein the image data output unit is connected to a display unit. The image data generation unit adds additional information regarding the security classification of the image data to be displayed to the data stream by binary encoding a classification in the color bit information of at least one pixel of the image data, and thus classifies the image data to be displayed regarding the security classification. This allows the image data output unit to control changes to the security-related display content of the image during the image enhancement process by minimizing or completely prohibiting the image enhancement process. Attached Figure Description

[0017] Other advantages, features, and details arise from the following description—with reference to the figures where necessary—detailed in the description of at least one embodiment. The described and / or illustrated features may form the subject matter of the invention individually or in any meaningful combination, and may be independent of the claims, and in particular, additionally, the subject matter of one or more separate applications. Identical, similar, and / or functionally identical components are given the same reference numerals.

[0018] in:

[0019] Figure 1 A schematic view of the device of the present invention is shown.

[0020] Figure 2 Showing the use of according to Figure 1 The apparatus implements an embodiment of the method according to the invention, which has a two-level functional safety process in the receiving unit.

[0021] Figure 3 An embodiment showing a flowchart in the receiving unit is provided.

[0022] Figure 4 An embodiment for processing ASIL-classified pixels in a receiving unit is shown.

[0023] Figure 5 An embodiment for processing non-ASIL classified pixels in a receiving unit is shown.

[0024] Figure 6 An example is shown where a classification bit is added and subsequently replaced by bits from the original image data. Detailed Implementation

[0025] Figure 1 A schematic diagram of a device according to the present invention is shown. The device 100 includes a transmitting unit 110 and a receiving unit 120, both of which process image data, which is displayed in a display unit 130, which is part of the receiving unit 120. The transmitting unit 110 is a rendering unit for generating an image from the original data and includes a first layer 140, within which non-critical pixel layers are added to the image. In a second layer 150 designed as a protected unit, image elements classified as safety-critical are added to the image. The criticality of the pixel layers used to generate the image depends, for example, whether the image presents display content for ASIL-related systems, such as autonomous driving systems, or for unprotected systems, such as entertainment systems. The image content, rendered entirely at the transmitting unit 110 and classified at the pixel level, forms a compressed data stream in a serializer 160. This compressed data stream is transmitted to the receiving unit 120 via a video transmission path 170 and decompressed and converted into image information by means of a deserializer 180. The receiving unit 120 receives the transmitted image information by means of a controller 190. Whether the pixels of the image information are security-critical (ASIL-related) or non-critical (which is determined by pixel-content mapping implemented according to security classification), the image to be displayed is modulated or not modulated in the image improver (125), that is, during the image improvement process, and is transmitted to the display unit 130 of the receiving unit 120.

[0026] Figure 2 Showing the use of according to Figure 1The apparatus 100 implements an embodiment of the method according to the invention, in which a two-level functional safety process is performed in the receiving unit 120. In function 1, visualization-related internal and external information, preferably vehicle information, is gathered in the transmitting unit 110. Visualization-related parameters are associated with image information (here, pixel-color content) for image composition. Next, in function 2, the image information is summarized with metadata containing binary codes that classify the image information into security levels, i.e., rendered into a single image information to achieve pixel-level formatting. After the rendered image information is transmitted from the transmitting unit 110 (function 3) to the receiving unit 120, in function 4, the data stream of the image to be displayed is divided into display information (color and position) and metadata (classification and position) according to pixels. The metadata here provides a description of the ASIL-related classification according to the security level in a pixel-level manner, while the display information determines the pixel color. A classification check for each pixel is performed in the first functional safety level 5. In response to the identified ASIL-related metadata, the system provides corresponding feedback, including specifications for the image improvement and display processes based on predetermined parameters, such as those derived from a lookup table. These specifications are provided to the image improvement process and display unit 130. Based on these specifications and the results of Functional Safety Classification 5, the image improvement process for each pixel is processed in function 6 according to applicable specifications categorized by pixel. Function 7 checks whether compliance with applicable specifications regarding pixel-based modifications or optimizations for predetermined display content is achieved. Here, the values ​​received in the receiving unit are compared pixel-by-pixel with the values ​​to be output to the display unit, for example, using a hash algorithm. If the values ​​match, the image information is transferred to a driver for visualization in the display unit in function 8.

[0027] As described, for each individual image transmitted along the video transmission path 170 between the transmitting unit 110 and the receiving unit 120, information about the pixel type is added to the image pixels on the transmitting unit 110 side. This is done via an RGB pixel-classification mapping. Classification at the RGB pixel level depends on the number of bits provided and is done in the corresponding LSB data (Last Significant Bit-Daten) of the RGB matrix. The classification probability should be represented by combining 8 bits of RGB pixels:

[0028]

[0029] The following examples illustrate several possible behaviors of RGB-based pixel formatting schemes:

[0030] 1. One LSB pixel is reserved in blue B, thus there are two classification states.

[0031] R=8,G=8,B=8→R=8,G=8,B=7+x

[0032] RRRRRRRR GGGGGGGG BBBBBBBB→RRRRRRRR GGGGGGGG BBBBBBBx

[0033] 2. Reserve 1 slot for each color, resulting in 8 possible category states.

[0034] R=8,G=8,B=8→R=7+x,G=7+x,B=7+x

[0035] RRRRRRRR GGGGGGGG BBBBBBBB→RRRRRRRRx GGGGGGGx BBBBBBBx

[0036] 3. One bit for each color is reserved for each data frame, resulting in eight classification states across three data frames. This is frame-based iterative pixel classification without significant image loss. With each frame, the classification bit alternates between red, green, and blue bit codes.

[0037] Data Frame 1:

[0038] R=8,G=8,B=8→R=7+x,G=8,B=8

[0039] RRRRRRRR GGGGGGGG BBBBBBBB→RRRRRRRRx GGGGGGGG BBBBBBBB

[0040] Data Frame 2:

[0041] R=8,G=8,B=8→R=8,G=7+x,B=8

[0042] RRRRRRRR GGGGGGGG BBBBBBBB→RRRRRRRR GGGGGGGx BBBBBBBB

[0043] Data Frame 3:

[0044] R=8,G=8,B=8→R=8,G=8,B=7+x

[0045] RRRRRRRR GGGGGGGG BBBBBBBB→RRRRRRRR GGGGGGGG BBBBBBBx

[0046] 4. LSB information is added to a separate image with 10-bit color depth, thus providing classification states 1 to N.

[0047] R=10, G=10, B=10→R=10-n+n*x, G=10-n+n*x, B=10-n+n*x;

[0048] For n=3,

[0049] RRRRRRRRRR GGGGGGGGGG BBBBBBBBBB→RRRRRRRRxxx GGGGGGGGxxx BBBBBBBBxxx

[0050] 5. Meta-information is provided by encoding in an additional channel (α and white).

[0051] R=8,G=8,B=8,A=8→R=8,G=8,B=8,A=(A+x)=8

[0052] RRRRRRRR GGGGGGGG BBBBBBBB aaaaaaaa→RRRRRRRR GGGGGGGG BBBBBBBBaaaaxxxx

[0053] In all the classification cases shown, x represents a reserved bit for LSB formatting.

[0054] Figure 3 This diagram illustrates one embodiment of a flowchart within the controller 190 of the receiving unit 120. In box 200, the overall image, presented as matrix pixel data, is read in. Individual pixels are extracted within box 210, and these individual pixels are read into the inspection process in box 220. Subsequently, in box 230, the pixel type of the individual pixel is checked. If security-critical pixel content is identified using binary encoding based on classification, the process proceeds to box 240, where restrictive criteria for pixel tampering are determined. These restrictive criteria could, for example, be anti-tampering protection. The process then continues to box 250, where pixel data, whose non-critical content was identified in box 230, is also forwarded. When the inspected pixel contains non-critical pixel content, image enhancement of the image data is performed in box 250. If a pixel with security-critical content is identified in box 230, the image data is processed only within the permissible restrictive criteria. The process moves from box 250 to box 260, where it is checked whether the applicable inspection criteria, perhaps determined in box 240, for pixels with security-critical content, are complied with. Within box 270, pixel categories are removed from the image data, and these pixel categories are replaced by the initial individual pixel data within box 280.

[0055] Figure 4An embodiment for processing ASIL-classified pixels in receiving unit 120 is shown. In function 3, image data is transmitted from transmitting unit 110 to receiving unit 120 as already explained, and this image data is converted into a total image pixel matrix in receiving unit 120. In function 4, the total image is segmented by pixel. Furthermore, consider now, for example, the 8th bit (B8) of the blue value, which contains display information (color 00000000 00000000 00000001) and metadata (classification 1). A classification check of bit B8 is performed in the first functional safety level 5. The evaluation result indicates that classification level 1 = ASIL pixel. In functional step 6, the classification bit is replaced with a temporary bit belonging to the original image information; that is, in this case, bit B8 is replaced, i.e., the bit is reset from 1 to the image value 0. Further, in function 6, an image improvement process is performed to process the RGB pixels of the image information according to applicable criteria related to the parameters given by the ASIL pixel classification. In this regard, binary code 1 means that pixel-based image enhancement is not allowed on the image belonging to bit B8 (first functional safety level 5). Function 7 checks whether the pixel-based modification or optimization criteria set in first functional safety level 5 are met. If the result is positive, function 8 transfers the image data to the driver for visualization within display unit 130. If the result is negative, emergency measures are taken, such as shutting down the image enhancer function, generating an alternative value, or outputting an indication of the erroneous function to the vehicle user.

[0056] Figure 5 An embodiment for processing non-ASIL classified pixels in the receiving unit 120 is shown. Here, functions 3 and 4 correspond to the combination Figure 4 The functions explained. In this case, it is not necessary to replace the classification bit with the color bit of the initial image, because both have a value of 0. However, classification bit B8 contains 0 as classification bit information in this case, which indicates non-safety-critical content. Therefore, changes to the image belonging to bit B8 are allowed during the image improvement process, which is also performed in function 6. Here, the standard set in the first functional safety level 5 is also checked using function 7. Because the image containing pixel B8 has non-critical content, there is no additional verification standard, so the pixel value is output to the display unit 130 in the form changed in function 6, that is, in the form of image improvement.

[0057] like Figure 6 As illustrated in detail, the metadata is replaced by the original color information of the previous or earlier image, which is achieved, in particular, by temporarily storing multiple potentially changing color value bits in the controller 190 for each pixel. In this example, the temporary storage is explained according to the pixel formatting scheme described above in item 3, in which classification bits are alternately written into the bit information of the RGB color values. In the blue LSB bits, according to the...Figure 2 As explained in function 2 or the corresponding method step, ASIL classification bit 1 is encoded as a binary code for classification. The LSB color value bits applicable to the original blue pixels before encoding are temporarily stored along with the red and green color value bits, as indicated by the arrows, see 6a. The classification bit is read out and used to classify... Figure 4 Function 5 is used to define the ASIL relevance of the pixel or image belonging to that bit. In function 6, the classification bit is used to overwrite the temporary bit of the color value again; that is, the classification bit is like... Figure 6 b is replaced by a value of 0. Therefore, the original pixel or the original image belonging to that pixel that existed before encoding is processed in the image improver according to the read classification bits, that is, it is output without improvement in the current case. Correspondingly, the classification bits encoded in red or blue are replaced by the temporary LSB bits of the original image before being transmitted to the image improver.

Claims

1. A method for safely displaying ASIL-related data on a display device in a motor vehicle, the motor vehicle comprising a transmitting unit (110), a receiving unit (120), and a display unit (130) for displaying an image output by the receiving unit (120), characterized in that, Compared to images with lower security ratings, images with higher security ratings are displayed on the display unit (130) in a manner that requires less modification during the image improvement process. Information about the security rating of the image to be displayed is added to the data stream via the transmitting unit (110) by means of binary encoding in the color bit information of at least one pixel of the image to be displayed via the pixel matrix, so that changes to the image can be controlled for each security rating within the image improver (125) associated with the receiving unit (120) for outputting image data during the image improvement process.

2. The method according to claim 1, characterized in that, Before transmitting the image to be displayed, which contains pixels, to the display unit (130), the classification binary code of the at least one pixel is read from the data stream received by the receiving unit (120) for outputting image data, wherein a determination is made on the implementation of image improvement based on the binary code of the at least one pixel.

3. The method according to claim 2, characterized in that, Before implementing the image improvement process, the read binary code of at least one pixel is replaced with the original color value bits stored in the pixel before binary encoding. Subsequently, during the image improvement process, the image to be displayed is modified using binary encoding according to its category.

4. The method according to claim 1 or 2, characterized in that, The color values ​​of the image to be displayed after the image enhancement process are checked to see if the image to be output is output in accordance with the security level set by the binary code.

5. The method according to claim 1 or 2, characterized in that, One bit is reserved in the color value bits of the predetermined color in the RGB information for the binary encoding.

6. The method according to claim 1 or 2, characterized in that, For the binary encoding, one bit is reserved for each color of the RGB information.

7. The method according to claim 1 or 2, characterized in that, For the binary encoding, a bit of a predetermined color is alternately reserved in each data frame.

8. The method according to claim 1 or 2, characterized in that, For the binary encoding, additional LSB least significant bits are provided in a separate image with a predetermined color depth, or several bits are reserved in a supplementary channel for color encoding.

9. The method according to claim 8, characterized in that, The color encoding is either RGBA encoding or RGBW encoding.

10. An apparatus for safely displaying ASIL-related data on a display device in a motor vehicle, the motor vehicle including a transmitting unit (110) for generating image data and a receiving unit (120) for outputting image data, the two communicating via a data stream, the motor vehicle further including a display unit (130) for displaying the image output by the receiving unit (120), characterized in that, Compared to images with lower security ratings, images with higher security ratings are displayed on the display unit (130) in a manner that requires less modification during the image improvement process. The transmitting unit (110) for generating image data is configured to, during the image improvement process, in the image improver (125) associated with the receiving unit (120) for outputting image data, add additional information about the security rating of the image data to be displayed as a classification binary code to the pixels of the image data for each security rating.

Images