A system and method for verifying the integrity of safety images output to a digital display

Abstract

The present invention is a system for verifying the integrity of a safety image output to a digital display for a vehicle, which is placed inside an integrated controller for In-Vehicle Infotainment (IVI) and Advanced Driver Assistance System (ADAS) of a vehicle, and includes a safety image rendering unit that generates a first safety image and a first marker corresponding to a safety event of the vehicle, and generates a second safety image in which the first marker is inserted into pixel data at a predetermined location of the first safety image; a display device that outputs the second safety image generated by the safety image rendering unit; and a safety image output verification unit that extracts a second marker of the second safety image from the screen of the display device and verifies the integrity of the second safety image based on the extracted second marker.

Description

Technology Field

[0001] The present invention relates to a system for verifying the integrity of a safety image and a method for verifying its integrity, and more specifically, to a system for verifying the integrity of a safety image output to a digital display for a vehicle and a method for verifying its integrity. Background Technology

[0002] Recently, in zone-based automotive architectures, high-performance computing systems such as domain controllers are evolving toward integrating various functions based on a single system. To achieve this, controllers capable of supporting heterogeneous functions based on multi-core SoCs are required. In particular, safety mechanisms are becoming increasingly important in SoCs where diverse functions are integrated.

[0003] For example, a multi-core SoC-based heterogeneous function support controller may include a cockpit domain controller, and the cockpit domain controller must simultaneously support AVN (Audio Video Navigation), digital display, climate control system, driver monitoring system, and ADAS (Advanced Driver Assistance System).

[0004] Here, the vehicle digital display includes a cluster, a head-up display (HUD), an infotainment display (IVI), etc., and includes a screen that delivers the vehicle's status, important warning information that may occur during driving, and interactive video to the driver in real time.

[0005] Vehicle digital displays include warning lights indicating abnormalities in key vehicle systems such as the engine, tire pressure, brakes, and oil pressure, as well as notifications for driver assistance systems like lane departure, collision warning, and blind spot warning. In addition, basic driving information such as speed, fuel level, temperature, seatbelt status, airbag status, door status, and battery condition is provided, delivering important visual information that enables the driver to respond immediately.

[0006] Therefore, vehicle display devices must visually convey safety information to the driver in real time, and in particular, functional safety standards such as ISO 26262 require technical means to verify whether such information is accurately output to the display.

[0007] Conventionally, to verify whether the relevant information is accurately output to the display, CRC or hash-based verification methods can be used to detect transmission errors by calculating a CRC value based on image or frame buffer data and comparing whether the values ​​on the transmitting and receiving sides match. However, since this method relies solely on image data at the time of transmission or rendering, it cannot directly verify whether the data is actually being output to the display. Furthermore, because there is no standard to verify the accuracy of the screen capture timing, there is a risk of timing errors or confusion with other images. Additionally, since CRC or hash comparisons at the entire frame level determine integrity based on the entire display, it cannot separately identify whether a specific safety image (e.g., a warning light) within the screen is actually being output. This may fail to meet the individual functional unit verification requirements demanded by functional safety standards such as ISO 26262.

[0008] Additionally, a frame number watermarking method can be used to detect whether a frame is frozen by inserting a frame number into the LSB region of a pixel and determining whether the screen is being updated normally. However, while this method is useful for verifying whether a frame is frozen, it may not be able to distinguish the exact content or type of the image. For example, there is a possibility that the image may be mistaken for normal simply because the frame number increases, even if a warning light has disappeared or changed.

[0009] Additionally, bitmap image comparison or direct GPU framebuffer reading methods can be used to detect rendering errors by comparing the output of the original image with the original image at the whole-image level. However, these methods are computationally intensive, make it difficult to verify display integrity in real-time, and can cause rendering delays or system overload. Furthermore, some watermarking methods can degrade display quality by directly affecting pixel colors.

[0010] Figure 1 is a figure showing a case where an original safety image is superimposed on another image within a digital display.

[0011] As illustrated in FIG. 1, when the original safety image (100) is displayed on a digital display (110), the original safety image (100) may be displayed overlaid with other UI elements or decoration images (120). Additionally, there may be cases where the original safety image (100) is not actually displayed on the digital display.

[0012] Figure 2 is a figure showing a conventional software processing method for verifying safety image integrity.

[0013] As illustrated in FIG. 2, the CRC calculation processing unit (220) calculates the Cyclic Redundancy Check (CRC) value for each pixel of the original safety image (100) transmitted by the safety image output unit (200) and the pre-set rectangular region of interest (ROI) image (130) of FIG. 1, and transmits it to the error detection and notification unit (230). The error detection and notification unit (230) checks whether the region of interest (ROI) image (130) and the original safety image (100) match based on the difference between the two CRC values, and if they do not match, provides a notification signal indicating that there is an error in the safety image output to the actual digital display.

[0014] However, conventional verification of the integrity of safety images can be slow because it is performed on every pixel of the ROI area image.

[0015] In addition, the process of calculating the CRC value of every pixel requires significant computational resources, making it difficult to perform quickly in real-time.

[0016] In particular, accurate verification becomes difficult if other UI elements or images are included within the configured region of interest. For instance, if a decorative image overlaps the area where the original safety image is displayed, the safety image being verified cannot be accurately recognized. Ultimately, this makes it impossible to reliably verify the integrity of safety images displayed on vehicle digital displays—that is, to confirm whether the information in the safety image matches the actual warning information.

[0017] Furthermore, since verification is performed without distinguishing between the original safety image and the background image, clear boundaries are often not established. If background elements or other UI elements other than the original safety image are included in the verification target, it becomes difficult to detect subtle changes or errors in the safety image, which can lower the sensitivity of error detection. Consequently, there is a possibility that minor changes to the safety image caused by interference from other UI elements may be overlooked during the verification process. In particular, confusion with the background can lead to situations where errors go undetected even if they occur, potentially compromising the reliability of the verification.

[0018] This issue hinders the accurate integrity verification of safety images output on automotive digital displays and can be a particularly significant constraint in the modern automotive digital display environment, where design elements are becoming increasingly complex. Prior art literature

[0019] Republic of Korea Registered Patent 2443751 (Method for validating image data, Robert Bosch GmbH) Republic of Korea Registered Patent 2541501 (Method and apparatus for detecting image errors, Telechips) The problem to be solved

[0020] Embodiments of the present invention provide a technology for quickly and accurately verifying whether an original safety image is actually output to a digital display for a vehicle.

[0021] However, the problem to be solved by the present invention is not limited thereto, and may be extended in various ways within an environment that does not deviate from the spirit and scope of the present invention. means of solving the problem

[0022] In order to achieve the above objectives, as one aspect, a system for verifying the integrity of a safety image output to a digital display for a vehicle, which is placed inside an integrated controller for In-Vehicle Infotainment (IVI) and Advanced Driver Assistance System (ADAS) of a vehicle, may include a safety image rendering unit that generates a first safety image and a first marker corresponding to a safety event of the vehicle, and generates a second safety image in which the first marker is inserted into pixel data at a predetermined location of the first safety image; a display device that outputs the second safety image generated by the safety image rendering unit; and a safety image output verification unit that extracts a second marker of the second safety image from the screen of the display device and verifies the integrity of the second safety image based on the extracted second marker.

[0023] The first marker and the second marker may include at least a portion of the identifier (ID) and frame number (Frame Counter) inserted by the safety image rendering unit when the first safety image is generated.

[0024] The above identifier (ID) and frame number (Frame Counter) are uniquely assigned according to the type of the first safety image, wherein the type may be distinguished by the content of the safety event and the time of creation of the safety image or the time of occurrence of the safety event.

[0025] The above-mentioned predetermined location may include four vertices or corners of the first safety image.

[0026] The first marker above may be composed of multiple bit fragments of a predetermined number of bits.

[0027] The second marker may include the original and a copy of the first marker.

[0028] The above safety image rendering unit can separate the original of the first marker into a first plurality of bit fragments with a predetermined number of bits, and distribute and arrange the first plurality of bit fragments in a first area among the predetermined locations of the first safety image.

[0029] The above safety image rendering unit can separate the replica of the first marker into a second plurality of bit fragments with a predetermined number of bits, and distribute and place the second plurality of bit fragments in a second area among the predetermined locations of the first safety image.

[0030] The first region and the second region do not overlap, and the safety image output verification unit can extract an error correction bit, an identifier (ID), and a frame number (Frame Counter) for a safety image using a plurality of bit fragments extracted from the first region and the second region.

[0031] The above safety image output verification unit can verify the integrity of the second safety image by comparing the identifier (ID) and frame number (Frame Counter) included in the first marker with the identifier (ID) and frame number (Frame Counter) included in the second marker, respectively.

[0032] Each of the first plurality of bit fragments and the second plurality of bit fragments of the first marker of the first safety image can be inserted into the LSB (Least Significant Bit) of each RGB channel at the predetermined position of the first safety image.

[0033] Each of the above safety image rendering unit and the above safety image output verification unit includes a CRC (Cyclic Redundancy Check) calculation processing unit, and the CRC calculation processing unit can verify the integrity of the second safety image by comparing the CRC calculated value of the first safety image and the CRC calculated value of the second safety image, depending on whether the two CRC values ​​match.

[0034] The above CRC calculation processing unit can perform CRC calculation only when the identifier (ID) and frame number (Frame Counter) extracted from the same region of interest (ROI) of the first safety image and the second safety image are identical.

[0035] The above safety image rendering unit and the above safety image output verification unit can operate independently in parallel.

[0036] The above safety image output verification unit may include periodically capturing the screen of the display device.

[0037] The above safety image output verification unit may include capturing the screen of the display device when it receives a safety event transmitted periodically or from an upper system.

[0038] The safety image output verification unit may perform at least one of the following operations: log storage, vehicle warning light illumination, and triggering fail-safe logic within the vehicle, if the second marker is not extracted, or if at least one of the CRC calculated value, identifier, and frame number extracted and compared from each of the first marker and the second marker is different from each other. Effects of the invention

[0039] The disclosed technology may have the following effects. However, this does not mean that a specific embodiment must include all of the following effects or only the following effects; therefore, the scope of the rights of the disclosed technology should not be understood as being limited by this.

[0040] The integrity verification system and integrity verification method for a safety image output on a digital display according to embodiments of the present invention select verification targets based on markers, thereby enabling the determination of when the safety image was output and what its content should be even in a complex display environment, and thus clearly selecting the timing and targets for verification of the safety image output on the digital display.

[0041] An integrity verification system and method for a safety image output to a digital display according to embodiments of the present invention can prevent unnecessary computation by performing CRC verification only when the restored information matches the current rendering target.

[0042] The marker used in the integrity verification system and integrity verification method for a safety image output to a digital display according to embodiments of the present invention can maintain verification reliability even when pixel damage or noise occurs in the display device through a Hamming error correction function and a double insertion structure.

[0043] The marker used in the integrity verification system and integrity verification method for a safety image output to a digital display according to embodiments of the present invention is hiddenly inserted into the LSB of each RGB channel and causes only minute changes that are barely perceptible to the eye, so it does not have a substantial effect on the display quality of the output image.

[0044] The integrity verification system and integrity verification method for a safety image output to a digital display according to embodiments of the present invention can be utilized as an integrity verification means to achieve a functional safety level of ASIL-B or higher as required by ISO 26262, etc.

[0045] The integrity verification system and integrity verification method for safety images output to a digital display according to embodiments of the present invention can distinguish and collectively verify each safety image even when multiple images are displayed simultaneously, because the region of interest (ROI) and marker insertion location for each safety image are predefined.

[0046] The integrity verification system and method for a safety image output to a digital display according to embodiments of the present invention perform only region of interest (ROI)-based marker detection and CRC comparison, so the amount of computation is smaller compared to the full frame comparison method, and it is suitable for real-time processing.

[0047] The integrity verification system and integrity verification method for a safety image output to a digital display according to embodiments of the present invention can detect various error situations, such as rendering failure, overlay collision, and frame freezing, by processing rendering and verification in parallel. Brief explanation of the drawing

[0048] Figure 1 is a figure showing a case where the original safety image is overlaid on another image within the cluster. Figure 2 is a figure showing a conventional software processing method for verifying safety image integrity. FIG. 3 is a block diagram of a system for verifying the integrity of a safety image output to a digital display according to one embodiment of the present invention. FIG. 4 is a figure showing a method of placing markers at predefined corner pixels of each image in a vehicle digital display according to one embodiment of the present invention. FIG. 5 is a figure showing the configuration of corner pixels of an image in a vehicle digital display according to one embodiment of the present invention. FIG. 6 is a block diagram of a system for verifying the integrity of a safety image output to a digital display according to another embodiment of the present invention. FIG. 7 is a block diagram showing an IVI (In-Vehicle Infotainment) / ADAS (Advanced Driver Assistance System) integrated controller that performs embodiments of the present invention. Specific details for implementing the invention

[0049] The present invention is capable of various modifications and may have various embodiments; specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and it should be understood that it includes all modifications, equivalents, and substitutions that fall within the technical spirit and scope of the present invention. In describing the present invention, detailed descriptions of related prior art are omitted if it is determined that such detailed descriptions may obscure the essence of the present invention.

[0050] Terms such as "first," "second," etc., may be used to describe various components, but the components are not limited by these terms. The terms may be used solely for the purpose of distinguishing one component from another.

[0051] The terms used in this invention are used merely to describe specific embodiments and are not intended to limit the invention. While the terms used in this invention have been selected to be as widely used as possible in consideration of their functions within the invention, they may vary depending on the intent of those skilled in the art, case law, or the emergence of new technologies. Furthermore, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this invention should be defined not merely by their names, but based on their meanings and the overall content of the invention.

[0052] A singular expression may include a plural expression unless the context clearly indicates otherwise. In the present invention, terms such as "comprising" or "having" are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0053] FIG. 3 is a block diagram of a system for verifying the integrity of a safety image output to a digital display according to one embodiment of the present invention.

[0054] The safety image integrity verification system (300) of the present invention of FIG. 3 may include a marker generation unit (310), a marker insertion unit (320), a safety image output unit (330), a display device (340), a display screen capture unit (350), a marker extraction unit (360), an integrity verification unit (370), and a CRC calculation processing unit (380).

[0055] The marker generation unit (310) generates an identifier (ID) and a frame number (Frame Counter) for the safety image (301). The identifier (ID) is uniquely assigned for each type of safety image (e.g., safety event content, time of safety event occurrence, or time of safety image output, etc.) and is predefined during system initialization.

[0056] The frame counter is counted whenever the safety image (301) is rendered. For example, the identifier (ID) can be composed of 3 bits and the frame counter of 5 bits, and the ratio can be adjusted according to system requirements.

[0057] The marker generation unit (310) combines an identifier and a frame number to form 8-bit data, and then encodes it with a Hamming (12, 8) code to generate a 12-bit marker. The Hamming (12, 8) code can provide 1-bit error correction and 2-bit error detection functions by adding 4-bit parity to the 8-bit data.

[0058] The final marker generated by the marker generation unit (310) can be configured with the following structure.

[0059] 1) Data field bit count: 8 bits (Identifier (3 bits) + Frame number (5 bits))

[0060] 2) Number of parity field bits: 4 bits (Hamming Parity)

[0061] 3) Total number of marker bits = 12 bits

[0062] Here, parity bits are inserted according to Hamming position rules, and as a result, markers can be arranged in the format [P1 P2 D1 P4 D2 D3 D4 P8 D5 D6 D7 D8].

[0063] The final marker generated by the marker generation unit (310) is prepared to be redundantly distributed and inserted into the LSB (Least Significant Bit) space of RGB 3 channels x 4 pixels, and can be transmitted to the marker insertion unit (320).

[0064] The marker insertion unit (320) can conceally insert markers that are redundantly distributed (i.e., including the original and duplicate of the marker) at the four vertex (or corner) pixels of the region of interest (ROI) within the screen at the time of rendering, i.e., the area where the safety image (301) to be verified is displayed. The insertion method can be configured so as not to affect the display quality by inserting into the LSB of each RGB channel.

[0065] Here, by inserting two duplicated markers into the top and bottom corner pixels respectively, it is possible to configure the system so that even if one marker cannot be restored due to noise or pixel damage, alternative verification can be performed using the other marker.

[0066] The CRC calculation processing unit (380) can calculate a first CRC value for the region of interest (ROI) of the original safety image (301) before the marker is inserted immediately before rendering the safety image, and store it as a reference value. Subsequently, it can recalculate a second CRC value for the same region of interest (ROI) of the safety image actually output to the display device (340), and verify the integrity of the safety image actually output by comparing the reference value and the second CRC value.

[0067] The safety image output unit (330) outputs a safety image (301) with a marker inserted and a calculated CRC value to a display device (340), and the safety image output to the display device (340) becomes a screen capture target.

[0068] The display device (340) is a display device that is visually exposed to the actual driver, such as an in-vehicle cluster, HUD, or IVI, and outputs an image that is subject to integrity verification.

[0069] The display screen capture unit (350) can capture a screen output from the display device (340) at a predetermined period or at the time of an event occurrence. The captured screen can be input to the marker extraction unit (360) and the CRC calculation processing unit (380).

[0070] The marker extraction unit (360) can extract and combine the LSB data of each RGB channel of the corner pixel area of ​​a predefined image from the captured screen to restore each of the two redundantly inserted markers.

[0071] The marker extraction unit (360) can be configured to recover identifiers and frame numbers through Hamming (12, 8) decoding from each of the two redundantly inserted markers, and to correct even 1-bit errors. Here, if at least one of the two recovered markers is valid, the identifier (ID) and frame number (Frame Counter) included in the marker can be used to determine whether it is subject to integrity verification. Specifically, the identifier (ID) is unique information that distinguishes which safe image the marker corresponds to, and through this, the validity of the marker can be verified. In addition, the frame number (Frame Counter) can be used as a criterion to compare whether the image at the time of rendering and the image actually output on the display correspond to the same frame.

[0072] Therefore, when a marker is restored, a CRC comparison is performed only if the identifier and frame number match the information stored at the time of rendering, thereby allowing for the verification of the integrity of the safety image. Here, the verification of the integrity of the safety image may include verifying whether the safety image currently being output to the display device (340) is being output with the correct content at the correct time. On the other hand, if the restoration of both markers fails, the safety image may be treated as a verification failure.

[0073] The integrity verification unit (370) can determine integrity based on whether the CRC values ​​calculated for the region of interest (ROI) of the safe image (301) at the time of rendering and the image captured by the display screen capture unit (350) match through the CRC calculation processing unit (380). That is, if the two CRC values ​​match, the integrity is considered verified, and if they do not match, it is determined that an error occurred in the output of the safe image. Consequently, the integrity verification unit (370) does not perform integrity verification on the safe image captured by the display screen capture unit (350).

[0074] FIG. 4 is a figure showing a method of placing markers at predefined corner pixels of each image in a vehicle digital display according to one embodiment of the present invention.

[0075] The illustrated vehicle digital display (400) of FIG. 4 may include a plurality of information or a plurality of safety images (401 to 408, 410) displayed at predetermined locations. In particular, the safety image 410 may be predefined to display information and images related to ADAS (Advanced Driver Assistance System).

[0076] According to one embodiment of the present invention, the marker insertion unit (320) of FIG. 3 duplicates (i.e., duplicates) the 12-bit marker generated by the marker generation unit (310) and can distribute and insert the original marker and the duplicate marker into pixels of at least one of four vertices (or corners) among a plurality of images (401 to 408, 410) each displayed at predetermined locations within the vehicle digital display (400) according to a predetermined rule.

[0077] As illustrated in FIG. 4, the safety image 410 shows four markers (M1o_1, M1o_2, M1c_1, M1c_2) placed at each of the four vertices of the safety image 410. Specifically, the original marker (M1o) placed in the safety image 410 can be separated into markers M1o_1 and M1o_2 and placed at the two upper vertices. In the same way, the duplicate marker (M1c) of the original marker (M1o) placed in the safety image 410 can be separated into markers M1c_1 and M1c_2 and placed at the two lower vertices.

[0078] That is, by distributing and inserting the original marker (M1o) into two upper vertices as M1o_1 and M1o_2, and inserting duplicate markers (M1c) with the same content into two lower vertices as M1c_1 and M1c_2, a total of two marker sets can be provided. Each of the sets of M1o and M1o_2 distributed and inserted into two upper vertices and the sets of M1c_1 and M1c_2 distributed and inserted into two lower vertices contains bit fragments divided into RGB LSB units to restore the entire 12-bit marker, and by combining M1o_1 and M1o_2 or combining M1c_1 and M1c_2, a bit sequence identical to the original marker (M1o) can be reconstructed.

[0079] Therefore, even if one of the marker sets M1o_1, M1o_2 and M1c_1, M1c_2 cannot be restored due to display noise, pixel damage, or capture error, the same information can be restored through another marker set, thereby significantly reducing the possibility of verification failure and improving the reliability of integrity verification.

[0080] Meanwhile, safety image 401 shows four markers (M2o_1, M2o_2, M2c_1, M2c_2) placed at each of the four vertices of safety image 401.

[0081] Specifically, the original marker (M2o) placed on the safety image 401 can be separated into markers M2o_1 and M2o_2 and placed at the upper two vertices. In the same way, the duplicate marker (M2c) of the original marker (M2o) placed on the safety image 401 can be separated into markers M2c_1 and M2c_2 and placed at the lower two vertices.

[0082] On the one hand, the original marker and the duplicate marker can be placed separately on the left or right side, respectively. On the other hand, M1o_1 can be placed on the upper left and M1o_2 on the lower right, and M1c_1 can be placed on the upper right and M1c_2 on the lower left.

[0083] FIG. 5 is a figure showing the configuration of corner pixels of an image in a vehicle digital display according to one embodiment of the present invention.

[0084] FIG. 5 is a figure showing the configuration of corner pixels of safety image 410 and safety images 401 to 408 of FIG. 4. Each of the four corner pixels of safety image 410 may include an R channel, a G channel, and a B channel.

[0085] The original marker (M1o) corresponding to the safety image 410 can be composed of 12 bits, and the original marker (M1o) can be separated into M1o_1 and M1o_2 and placed on the upper right and upper left sides, respectively. Additionally, the replica marker (M1c) can be separated into M1c_1 and M1c_2 and placed on the lower right and lower left sides, respectively.

[0086] M1o_1, which is a part of the original marker (M1o), can be separated into three 2-bits (501 to 503) and hiddenly inserted into the LSBs of the upper left R channel, G channel, and B channel, respectively.

[0087] M1o_2, which is another part of the original marker bit (M1o), can be separated into three 2-bits (504 to 506) and hiddenly inserted into the LSBs of the upper right R channel, G channel, and B channel, respectively.

[0088] M1c_1, which is part of the replica marker (M1c), can be separated into three 2-bits (510 to 512) and hiddenly inserted into the LSBs of the R channel, G channel, and B channel on the lower left side.

[0089] M1c_2, which is another part of the replica marker bit (M1c), can be separated into three 2-bits (513 to 515) and hiddenly inserted into the LSB of each of the R channel, G channel, and B channel on the lower right side.

[0090] FIG. 6 is a block diagram of a system for verifying the integrity of a safety image output to a digital display according to another embodiment of the present invention.

[0091] According to another embodiment of the present invention, a system (600) for verifying the integrity of a safety image of FIG. 6 shows that two subsystems (610, 620) operate in parallel to verify the integrity of a safety image output to a digital display.

[0092] The system (600) for verifying the integrity of the safety image of FIG. 6 may include a safety image rendering unit (610), a safety image output verification unit (620), and a display device (630).

[0093] The safety image rendering unit (610) can render a safety image and output the rendered safety image to a display device (630).

[0094] The safety image output verification unit (620) can verify the integrity of the safety image output to the display device (630).

[0095] The safety image rendering unit (610) and the safety image output verification unit (620) can operate in parallel independently of each other while the cluster system (city omitted) is running.

[0096] In particular, the safety image output verification unit (620) can independently determine whether the correct safety image is being displayed correctly at the intended time by using the identifier (ID) and frame number included in the marker extracted from the screen capture unit (621), without synchronizing with the safety image rendering unit (610).

[0097] The safety image rendering unit (610) is requested to output a safety image when the cluster system is executed, when the upper system (e.g., an electronic control unit (ECU) in the vehicle) boots up or when a safety event occurs.

[0098] Subsequently, the safety image generation unit (611) generates a safety image defined by the user or the upper system. Here, the safety image to be rendered can be determined according to the application or cluster internal logic.

[0099] The CRC calculation processing unit (612) can pre-calculate a CRC value based on the original safety image to be output. The pre-calculated CRC value can be used as a verification standard value for the integrity verification unit (625) of the safety image output verification unit (620).

[0100] The marker generation unit (613) can perform marker generation and Hamming encoding. That is, the marker generation unit (613) can generate a 12-bit marker through Hamming encoding based on the identifier and frame number of the rendered safe image. Here, the number of bits is adjustable.

[0101] The marker insertion unit (614) can insert an encoded 12-bit marker by hiding it in the LSB of the corner pixel RGB value of the rendered safety image. Specifically, by utilizing the lower 2 bits of each existing RGB channel (8 bits) to insert the marker data, information can be hidden at a level that is imperceptible to human vision. For example, the pixel value (255, 127, 64) of the original RGB (8, 16, 24) is changed to (252, 124, 62) after hiding, but the difference in the changed value △RGB = (3, 3, 2) may be visually indistinguishable.

[0102] The encoded 12-bit markers can be duplicated and inserted by distributing them at the top and bottom (see FIGS. 4 and 5).

[0103] In general digital image systems, including automotive digital displays, noise tends to occur primarily in the horizontal direction, which can be attributed to technical factors such as the display's raster scan order (proceeding from left to right and top to bottom), digital signal interference, electromagnetic interference (EMI), and line synchronization errors.

[0104] Afterwards, the safety image with the inserted marker can be rendered and output to the display device (630).

[0105] The safety image output verification unit (620) can periodically capture an image corresponding to a safety event from a screen displayed on a display device (630). Alternatively, if information received from a higher system (e.g., an electronic control unit (ECU) in the vehicle, a central control unit, a zone control unit, etc.) corresponds to a predetermined safety event (e.g., lane departure, detection of a front obstacle, etc.), the safety image can be captured from a screen displayed on a display device (630).

[0106] The marker extraction unit (622) can extract RGB LSB from the corner pixel area of ​​the captured safety image. Specifically, error correction, identifier, and frame number can be restored through Hamming decoding. If the marker is not extracted, the user can be notified of the marker extraction error (626).

[0107] Subsequently, the restored marker information (e.g., identifier and frame number) is compared with a verification queue containing the identifier and frame number stored at the time the safe image was rendered.

[0108] The CRC calculation processing unit (624) can recalculate the CRC value based on the region of interest (ROI) of the captured image only when there is an item that matches the identifier and frame number stored in advance in the verification waiting queue (Que).

[0109] If there is no matching item in the verification waiting queue for the restored information, the marker is considered invalid and is treated as a marker invalid failure (627).

[0110] The integrity verification unit (625) compares CRCs and determines integrity. It can determine integrity by comparing whether the CRC value of the original image before rendering matches the CRC value of the safe image captured by the screen capture unit (621). If the CRC value of the safe image before rendering does not match the CRC value of the captured safe image, the user can be notified of the integrity verification failure (628).

[0111] Additionally, the results of marker extraction error (626), marker validity failure (627), and integrity verification failure (628) may be stored as a log in internal memory if necessary, or may trigger a vehicle warning light or fail-safe logic within the vehicle.

[0112] FIG. 7 is a block diagram showing an IVI (In-Vehicle Infotainment) and / or ADAS (Advanced Driver Assistance System) integrated controller performing embodiments of the present invention.

[0113] The IVI and / or ADAS integrated controller of FIG. 7 may include a main cluster controller (710) and a sub-cluster controller (730).

[0114] The main cluster controller (710) must control the AVN, head-up display (HDU), and HVAC unit (720) of safety grade ASIL-QM, and the sub cluster controller (730) must also control the vision, cluster, and DMS (740) of safety grade ASIL-B.

[0115] As such, the safety image integrity verification system of the present invention can be placed inside an IVI and / or ADAS integrated controller or cluster.

[0116] Accordingly, an IVI and / or ADAS integrated controller (710) or cluster performing the embodiments of the present invention can output a corresponding safety image at an accurate time and with accurate content to a display device, i.e., a cluster or head-up display, when a safety event occurs in the vehicle, and at the same time can achieve a functional safety rating of ASIL-B or higher as required by ISO 26262, etc. Explanation of the symbols

[0117] 100: Original safe image 110: Digital Display 120: UI Design Components 130: ROI area image 200, 330, 615: Safety image output section 310, 613: Marker generation section 320, 614: Marker insertion part 340, 630: Display device 210, 350, 621: Display screen capture unit 360, 622: Marker extraction section 220, 380, 612, 624: CRC calculation processing unit 230: Error Detection and Notification Section 301, 410, 401 to 408: Safety images 300, 600: Safe Image Integrity Verification System 700: IVI (In-Vehicle Infotainment) / ADAS (Advanced Driver Assistance System) Integrated Controller

Claims

Claim 1 A system for verifying the integrity of a safety image output to a digital display for a vehicle, which is positioned within an IVI (In-Vehicle Infotainment) and / or ADAS (Advanced Driver Assistance System) integrated controller or cluster of a vehicle, comprising: a safety image rendering unit that generates a first safety image and a first marker corresponding to a safety event of the vehicle, wherein the first marker is generated by encoding data combining an identifier (ID) and a frame number (Frame Counter) into a Hamming code to enable 1-bit error correction and 2-bit error detection, and generates a second safety image in which the first marker is inserted into pixel data at a predetermined location of the first safety image; and a display device that outputs the second safety image generated by the safety image rendering unit. The safety image output verification unit includes a second marker for the second safety image on the screen of the display device and verifies the integrity of the second safety image based on the extracted second marker, wherein the second marker includes an original and a copy of the first marker, and the safety image rendering unit separates the original of the first marker into a first plurality of bit fragments of a predetermined number of bits and distributes the first plurality of bit fragments in a first area among the predetermined locations of the first safety image, and separates the copy of the first marker into a second plurality of bit fragments of a predetermined number of bits and distributes the second plurality of bit fragments in a second area among the predetermined locations of the first safety image that does not overlap with the first area. The above safety image rendering unit includes a verification waiting queue that stores the identifier and frame number of the corresponding safety image at the time the first safety image is rendered, and secretly inserts each of the first plurality of bit fragments and the second plurality of bit fragments into the LSB (Least Significant Bit) of each of the RGB channels at a predetermined position; the safety image output verification unit operates in parallel independently of the safety image rendering unit without synchronization, captures a screen output to the display device periodically or upon receiving a safety event from an upper system, extracts the second marker from the LSB of each of the RGB channels of the captured screen, and restores the identifier and frame number by Hamming decoding; each of the safety image rendering unit and the safety image output verification unit includes a CRC (Cyclic Redundancy Check) calculation processing unit, and the CRC calculation processing unit compares the CRC calculated value of the first safety image and the CRC calculated value of the second safety image, and verifies the integrity of the second safety image based on whether the two CRC values ​​match, wherein the CRC calculation processing unit compares the first safety image and the second A system for verifying the integrity of a safety image that performs CRC calculation only when the identifier (ID) and frame number (Frame Counter) extracted from the same region of interest (ROI) of the safety image are identical. Claim 2 A safety image integrity verification system according to claim 1, wherein the first marker and the second marker include at least a portion of an identifier (ID) and a frame number (Frame Counter) inserted by the safety image rendering unit when the first safety image is generated. Claim 3 In paragraph 2, the identifier (ID) and frame number (Frame Counter) are uniquely assigned according to the type of the first safety image, wherein the type is distinguished by the content of the safety event and the time of creation of the safety image or the time of occurrence of the safety event, a safety image integrity verification system. Claim 4 A safety image integrity verification system according to claim 1, wherein the predetermined position includes four vertices or corners of the first safety image. Claim 5 A security image integrity verification system according to claim 1, wherein the first marker is composed of a plurality of bit fragments of a predetermined number of bits. Claim 6 delete Claim 7 delete Claim 8 delete Claim 9 In claim 1, the safety image output verification unit extracts an error correction bit, an identifier (ID), and a frame number (Frame Counter) for the safety image using a plurality of bit fragments extracted from the first region and the second region, a safety image integrity verification system. Claim 10 In claim 9, the safety image output verification unit verifies the integrity of the second safety image by comparing the identifier (ID) and frame number (Frame Counter) included in the first marker with the identifier (ID) and frame number (Frame Counter) included in the second marker, respectively. Claim 11 delete Claim 12 delete Claim 13 delete Claim 14 delete Claim 15 delete Claim 16 delete Claim 17 A safety image integrity verification system according to claim 1, wherein the safety image output verification unit performs at least one of the operations of log storage, vehicle warning light illumination, and triggering fail-safe logic in the vehicle when the second marker is not extracted from the second safety image, or when at least one of the CRC calculated value, identifier, and frame number extracted and compared from each of the first marker and the second marker is different from each other.

Images