System and method for verifying video overlay buffer integrity

Abstract

A camera display system is provided for a vehicle including an image sensor configured to capture video images including a plurality of image frames and a display system. The display system includes an overlay buffer, an image processor, and a display for displaying the images received from the image processor. The image processor configured to: (a) build and store an overlay image in the overlay buffer, (b) receive an image frame from the image sensor, (c) read the overlay image from the overlay buffer, (d) perform an integrity check on the read overlay image to ensure it has not become corrupted, (e) when the integrity check is passed, superimpose the overlay image onto the image frame and repeat steps (b)-(d); and (f) when the integrity check is failed, repeat steps (a)-(d). The display system further includes a display for displaying the images received from the image processor.

Description

SYSTEM AND METHOD FOR VERIFYING VIDEO OVERLAY BUFFER INTEGRITYCROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims priority to and the benefit under 35 U.S.C. § 119(e) of U.S. Provisional Patent Application No. 63 / 663,261, filed on June 24, 2024, entitled "SYSTEM AND METHOD FOR VERIFYING VIDEO OVERLAY BUFFER INTEGRITY," by Andrew J. Oosterhouse, the entire disclosure of which is incorporated herein by reference.TECHNICAL FIELD

[0002] The present disclosure generally relates to a camera display system for a vehicle and more particularly to a rear camera display (RCD) and / or full display mirror (FDM) system providing a rearward view to a display.SUMMARY

[0003] According to one aspect of the present disclosure, a camera display system is provided for a vehicle, the camera display system including: an image sensor configured to capture video images including a plurality of sequential image frames and a display system. The display system including: a first overlay buffer in which a first overlay image is built and stored, and an image processor configured to: (a) build the first overlay image in the first overlay buffer, (b) receive an image frame from the image sensor, (c) read the first overlay image from the first overlay buffer, (d) perform an integrity check on the first overlay image each time it is read from the first overlay buffer to ensure the first overlay image has not become corrupted, (e) when the integrity check is passed, superimpose the first overlay image onto the image frame and repeat steps (b)-(d); and (f) when the integrity check is failed, repeat steps (a)-(d). The display system further includes a display coupled to the image processor for displaying the images received from the image processor.

[0004] According to another aspect of the present disclosure, a method for verifying overlay buffer integrity is provided, the method includes: (a) building and storing a first overlay image into a first overlay buffer; (b) reading the first overlay image from the first overlay buffer each time a video image frame is received from an image sensor; (c) performing anintegrity check of the first overlay image each time the first overlay image is read from the first overlay buffer; (d) when the integrity check is passed, superimposing the first overlay image onto a video image frame and repeating steps (b) and (c); and (e) when the integrity check is failed, repeating steps (a)-(c).

[0005] According to another aspect of the present disclosure, a camera display system is provided for a vehicle, the camera display system including: an image sensor configured to capture video images including a plurality of sequential image frames; and a display system including: a first overlay buffer in which a first overlay image is built and stored; a second overlay buffer in which a second overlay image is built and stored; an image processor; and a display coupled to the image processor for displaying the images received from the image processor. The image processor configured to: (a) build the first overlay image in the first overlay buffer, (b) receive an image frame from the image sensor, (c) read the first overlay image from the first overlay buffer, (d) perform an integrity check on the first overlay image each time it is read from the first overlay buffer to ensure the first overlay image has not become corrupted, (e) when the integrity check is passed, superimpose the first overlay image onto the image frame and repeat steps (b)-(d), (f) when the integrity check is failed, repeat steps (a)-(d), (g) when a second overlay image different from the first overlay image is required, building and storing the second overlay image in the second overlay buffer, (h) receive an image frame from the image sensor, (i) read the second overlay image from the second overlay buffer, (j) perform the integrity check of the second overlay image each time the second overlay image is read from the second overlay buffer, (k) when the integrity check is passed, superimpose the second overlay image onto the image frame and repeat steps (h)-(j), and (I) when the integrity check is failed, repeat steps (g)-(j).

[0006] These and other features, advantages, and objects of the present disclosure will be further understood and appreciated by those skilled in the art by reference to the following specification, claims, and appended drawings.BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The present disclosure will become more fully understood from the detailed description and the accompanying drawings, wherein:

[0008] FIG. 1 is an electrical circuit diagram in block form of a camera display system for a vehicle;

[0009] FIG. 2A is an illustration of the integrity check process on a frame 0 as performed by the system shown in FIG. 1;

[0010] FIG. 2B is an illustration of the integrity check process on a frame 1 as performed by the system shown in FIG. 1;

[0011] FIG. 2C is an illustration of the integrity check process on a frame 2 as performed by the system shown in FIG. 1;

[0012] FIG. 2D is an illustration of the integrity check process on a frame 3 as performed by the system shown in FIG. 1;

[0013] FIG. 2E is an illustration of the integrity check process on a frame 4 as performed by the system shown in FIG. 1;

[0014] FIGs 3 and 3A are a flowchart illustrating a method for verifying overlay buffer integrity that may be implemented using the system shown in FIG. 1; and

[0015] FIG. 4 is a flowchart showing an example of an integrity check subroutine that may be performed in the method illustrated in FIGS. 3 and 3A.DETAILED DESCRIPTION

[0016] The present illustrated embodiments reside primarily in combinations of method steps and apparatus components related to a vehicle rear camera display (RCD) system and method thereof. Accordingly, the apparatus components and method steps have been represented, where appropriate, by conventional symbols in the drawings, showing only those specific details that are pertinent to understanding the embodiments of the present disclosure so as not to obscure the disclosure with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein. Further, like numerals in the description and drawings represent like elements.

[0017] In this document, relational terms, such as first and second, top and bottom, and the like, are used solely to distinguish one entity or action from another entity or action, without necessarily requiring or implying any actual such relationship or order between such entities or actions. The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0018] Vision systems, such as RCD systems, have become commonplace within vehicles. RCD systems typically include a rearward facing camera, an image processor, and a display. RCD systems cause the rearward image from the camera to be displayed whenever the vehicle is in reverse gear. Graphic overlays may be superimposed that show the projected rearward path of the vehicle based on the steering angle of the vehicle. Other graphic or textual overlays on the display may include, for example, compass heading, menus, status indicators, or button locations (if the display is a touch screen). Such overlay images are built and stored in overlay buffers. The overlays are then read out of the buffer and superimposed on each of the image frames from the camera. One problem that can arise is that the overlay images may become corrupted and therefore display something other than what is intended. One approach to address this problem is to update the overlay buffers continually to try to assure data integrity. The solution described below provides an integrity check of the overlay image data without constant re-writes to the overlay buffers.

[0019] Referring to FIG. 1, camera display system 10 is shown equipped with an image sensor 12 and a display system 14. The image sensor 12 may, for example, be configured to capture images of a rearward field of view.

[0020] The display system 14 may include an image processor 20, a first overlay buffer 30 in which a first overlay image is built and stored, and a display 50 coupled to the image processor 20 for displaying the images received from the image processor 20. The first overlay buffer 30 may be part of the image processor 20 or separate from the image processor 20. The image processor 20 may be configured to: (a) build the first overlay imagein the first overlay buffer 30, (b) receive an image frame from the image sensor 12, (c) read the first overlay image from the first overlay buffer 30, (d) perform an integrity check on the first overlay image each time it is read from the first overlay buffer 30 to ensure the first overlay image has not become corrupted, (e) when the integrity check is passed, superimpose the first overlay image onto the image frame and repeat steps (b) and (d); and (f) when the integrity check is failed, repeat steps (a)-(d).

[0021] The display system 14 may further include a second overlay buffer 32 in which a second overlay image is built and stored. The image processor 20 may be further configured to: (g) when a second overlay image different from the first overlay image is required, building and storing the second overlay image in the second overlay buffer 32; (h) receive an image frame from the image sensor 12; (i) read the second overlay image from the second overlay buffer 32; (j) perform the integrity check of the second overlay image each time the second overlay image is read from the second overlay buffer 32; (k) when the integrity check is passed, superimpose the second overlay image onto the image frame and repeat steps (h)- (j); and (I) when the integrity check is failed, repeat steps (g)-(j).

[0022] The image processor 20 may include DDR memory 23 and a field programmable gate array (FPGA) 28 for receiving the images from the image sensor 12 and / or the DDR memory 23 and the first and second overlay buffers 30 and 32, which may reside in DDR memory 23. The DDR memory 23 may buffer the video from the image sensor 12 before being blended with the overlay images.

[0023] The display 50 may be positioned in a rearview mirror assembly, a front console, or in any other location viewable by the driver. The image processor 20 may be located in the same relative location as the display 50 or may be located proximate the rear image sensor 12 or in an intermediate location.

[0024] The image processor 20 may also include an overlay render engine 34 that builds the overlay images in the first and second overlay buffers 30 and 32 under direction of a display controller 36. The display controller 36 directs the overlay render engine to generate a selected overlay image in a selected one of the first and second overlay buffers 30 and 32.The display controller 36 also generates a buffer pointer that identifies from which of the first and second overlay buffers 30 and 32 the FPGA 28 is to read an overlay image.

[0025] The display controller 36 may determine what overlay images to generate in response to various inputs such as inputs received from a user interface 38 and / or from a local or vehicle bus 39.

[0026] As described above, the image processor 20 performs the integrity check of the overlay images. This task may be delegated to the FPGA 28. As will be described in more detail, the integrity check may be performed by: calculating a cyclic redundancy check (CRC) for a first overlay image the first time it is read from the first overlay buffer and storing the calculated CRC; for each time thereafter that the first overlay image is read from the first overlay buffer, calculating a CRC for the overlay image and comparing the calculated CRC to the stored CRC; when the calculated CRC is the same as the stored CRC, the integrity check is passed; and when the calculated CRC is not the same as the stored CRC, the integrity check is failed. The integrity check on the second overlay image may be performed in the same manner. The method performing an integrity check using the CRC is only one example, the integrity check may be performed in other ways.

[0027] FIGS. 2A-2E show a sequence of examples of the integrity check process according to the example above using CRC. FIG. 2A shows a first example for an image frame 0 in which the first overlay buffer 30 is selected and the first overlay image stored therein is read for the first time. Here, the FPGA 28 may include a CRC calculation block 41 that outputs a calculated CRC value 42 from the first overlay image data. Because this is the first time the first overlay image is read, the calculated CRC value is stored in the stored CRC block 44. The FPGA 28 may also include a CRC comparison block 43 whose function is described below.

[0028] FIG. 2B shows an example for a subsequent image frame 1 in which the first overlay buffer 30 is selected and the first overlay image stored therein is read for the second time. In this case, the CRC calculation block 41 outputs a calculated CRC value 42 from the first overlay image data as read the second time. Because this is the second time the first overlay image is read, the CRC comparison block 43 compares the calculated CRC value to the stored CRC value in the stored CRC block 44. In this example, the CRC values are the same, so noerror condition arises, and the integrity check is passed. The calculated CRC value is not stored in the CRC block 44. Also, in this example, the display controller 36 determines that a second new overlay image is required and directs the overlay render engine 34 to build and store the second overlay image in the second overlay buffer 32.

[0029] FIG. 2C shows an example for an image frame 2 in which the second overlay buffer 30 is selected and the second overlay image stored therein is read for the first time. In this case, the CRC calculation block 41 outputs a calculated CRC value 42 from the second overlay image data. Because this is the first time the second overlay image is read, the calculated CRC value is stored in the stored CRC block 44. If there is a new overlay image required, the display controller 36 directs the overlay render engine 34 to build and store the first overlay image in the first overlay buffer 30.

[0030] FIG. 2D shows an example for a subsequent image frame 3 in which the second overlay buffer 30 is selected and the second overlay image stored therein is read for the second time. In this case, the CRC calculation block 41 outputs a calculated CRC value 42 from the second overlay image data as read the second time. Because this is the second time the second overlay image is read, the CRC comparison block 43 compares the calculated CRC value to the stored CRC value in the stored CRC block 44. In this example, the CRC values are the same, so no error condition arises and the integrity check is passed. The calculated CRC value is not stored in the CRC block 44.

[0031] FIG. 2E shows an example for a subsequent image frame 4 in which the second overlay buffer 30 is selected and the second overlay image stored therein is read for the third time. In this case, the CRC calculation block 41 outputs a calculated CRC value 42 from the second overlay image data as read the third time. Because this is not the first time the second overlay image is read, the CRC comparison block 43 compares the calculated CRC value to the stored CRC value in the stored CRC block 44. In this example, the CRC values are not the same, so an error condition arises, and the integrity check is failed. The calculated CRC value is not stored in the CRC block 44. In the event of an error condition, the second overlay image is attempted to be re-built in the second overlay buffer 32 and then that rebuilt second overlay image is subjected to the integrity check. If the re-built second overlayimage passes the integrity check, it is used as if no prior error condition occurred. However, if the re-built second overlay image fails the integrity check, any one or more of the following may occur: the second overlay image is no longer superimposed on the current image frame, an error message is generated, and / or the video and overlays are turned off in the case of an FDM with the FDM reverting to a mirror only mode.

[0032] It should be noted that the image processor 20 is configured to operate in the same manner as in FIG. 2E if there is an error condition relative to the first overlay image read from the first overlay buffer 30.

[0033] As also shown in FIG. 2E, if there is a new overlay image required, the display controller 36 directs the overlay render engine 34 to build and store the first overlay image in the first overlay buffer 30.

[0034] The display system 12 may be implemented on a full display mirror (FDM) platform where a second FDM camera is used having a higher rearward field of view to provide a full- time display of the rearward scene so as to replace the function of the mirror of the rearview mirror assembly.

[0035] FIG. 3 shows a method 100 for verifying overlay buffer integrity is thus provided where the method includes: (a) building and storing a first overlay image into a first overlay buffer (step 102); (b) reading the first overlay image from the first overlay buffer each time a video image frame is received from an image sensor (step 104); (c) performing an integrity check of the first overlay image each time the first overlay image is read from the first overlay buffer (steps 106 and 108); (d) when the integrity check is passed, superimposing the first overlay image onto a video image frame (step 110) and repeating steps (b) and (c) (steps 104-108); and (e) when the integrity check is failed, repeating steps (a)-(c) (steps 102- 108).

[0036] As shown in FIGS. 3 and 3A, if a second overlay image that is different from the first overlay image not required (step 112) steps (b) and (c) (steps 104-108) are repeated. If a different second overlay image is required (step 112), the method 100 may further include: (f) building and storing the second overlay image into a second overlay buffer (step 114); (g) reading the second overlay image from the second overlay buffer each time a video imageframe is received from the image sensor (step 116); (h) performing the integrity check of the second overlay image each time the second overlay image is read from the second overlay buffer (steps 118 and 120); (i) when the integrity check is passed, superimposing the second overlay image onto a video image frame (step 122) and repeating steps (g) and (h) (steps 116-120); and (j) when the integrity check is failed, repeating steps (f)-(h) (steps 114-120).

[0037] An example of an integrity check subroutine 108, 120 that may be used in method 100 is shown in FIG. 4. In the method, the integrity check 108, 120 is performed by: calculating a CRC for the overlay image; determining if it is the first time the overlay image is read from the buffer (step 130); the first time it is read from the overlay buffer storing the calculated CRC (step 132) and the subroutines returns to step 104 or 116; for each time thereafter that the first overlay image is read from the first overlay buffer, calculating a CRC for the overlay image and comparing the calculated CRC to the stored CRC (step 134); when the calculated CRC is the same as the stored CRC, the integrity check is passed (step 138); and when the calculated CRC is not the same as the stored CRC, the integrity check is failed step (136).

[0038] It will be appreciated that embodiments of the disclosure described herein may be comprised of one or more conventional processors and unique stored program instructions that control one or more processors to implement, in conjunction with certain nonprocessor circuits, some, most, or all of the functions of an image sensor system and method thereof, as described herein. The non-processor circuits may include, but are not limited to signal drivers, clock circuits, power source circuits, and / or user input devices. Alternatively, some or all functions could be implemented by a state machine that has no stored program instructions, or in one or more application specific integrated circuits (ASICs), in which each function or some combinations of the functions are implemented as custom logic. Of course, a combination of the two approaches could be used. Thus, the methods and means for these functions have been described herein. Further, it is expected that one of ordinary skill, notwithstanding possibly significant effort and many design choices motivated by, for example, available time, current technology, and economic considerations, when guided bythe concepts and principles disclosed herein will be readily capable of generating such software instructions and programs and ICs with minimal experimentation.

[0039] It should be appreciated by those skilled in the art that the above-described components may be combined in additional or alternative ways not explicitly described herein. Modifications of the various implementations of the disclosure will occur to those skilled in the art and to those who apply the teachings of the disclosure. Therefore, it is understood that the embodiments shown in the drawings and described above are merely for illustrative purposes and not intended to limit the scope of the disclosure, which is defined by the following claims as interpreted according to the principles of patent law, including the doctrine of equivalents.

Claims

CLAIMSWhat is claimed is:

1. A camera display system for a vehicle, the camera display system comprising: an image sensor configured to capture video images including a plurality of sequential image frames; and a display system comprising: a first overlay buffer in which a first overlay image is built and stored, an image processor configured to:(a) build the first overlay image in the first overlay buffer,(b) receive an image frame from the image sensor,(c) read the first overlay image from the first overlay buffer,(d) perform an integrity check on the first overlay image each time it is read from the first overlay buffer to ensure the first overlay image has not become corrupted,(e) when the integrity check is passed, superimpose the first overlay image onto the image frame and repeat steps (b)-(d), and(f) when the integrity check is failed, repeat steps (a)-(d), and a display coupled to the image processor for displaying the images received from the image processor.

2. The camera display system of claim 1, wherein the display system further comprises a second overlay buffer in which a second overlay image is built and stored, wherein the image processor is further configured to:(g) when a second overlay image different from the first overlay image is required, building and storing the second overlay image in the second overlay buffer;(h) receive an image frame from the image sensor;(i) read the second overlay image from the second overlay buffer;(j) perform the integrity check of the second overlay image each time the second overlay image is read from the second overlay buffer;(k) when the integrity check is passed, superimpose the second overlay image onto the image frame and repeat steps (h)-(j); and(l) when the integrity check is failed, repeat steps (g)-(j).

3. The camera display system of any one of claims 1 and 2, wherein the image processor performs the integrity check by: calculating a cyclic redundancy check (CRC) for first overlay image the first time it is read from the first overlay buffer and storing the calculated CRC; for each time thereafter that the first overlay image is read from the first overlay buffer, calculating a CRC for the overlay image and comparing the calculated CRC to the stored CRC; when the calculated CRC is the same as the stored CRC, the integrity check is passed; and when the calculated CRC is not the same as the stored CRC, the integrity check is failed.

4. The camera display system of any one of claims 2 and 3 and further comprising a user interface for receiving user inputs to change the overlay image5. The camera display system of claim 4, wherein the image processor comprises: an overlay render engine for building the first overlay image in the first overlay buffer and the second overlay image in the second overlay buffer; and a display controller coupled to the user interface, the overlay render engine, the first overlay buffer, and the second overlay buffer, the display controller is configured to determine what overlay images to generate in response to inputs received from one of the user interface and a vehicle bus.

6. The camera display system of claim 1 and further comprising a user interface for receiving user inputs to change the first overlay image7. The camera display system of claim 6, wherein the image processor comprises: an overlay render engine for building the first overlay image in the first overlay buffer; and a display controller coupled to the user interface, the overlay render engine, and the first overlay buffer, wherein the display controller is configured to determine what overlay images to generate in response to inputs received from one of the user interface and a vehicle bus.

8. The camera display system of any one of claims 1-7, wherein the image processor comprises a memory for receiving images from the image sensor.

9. The camera display system of claim 8, wherein the image processor comprises a field programmable gate array (FPGA) coupled to the image sensor, the memory, the first overlay buffer, and the display controller, wherein the FPGA performs the integrity check.

10. The camera display system of any one of claims 1-9, wherein if the integrity check is failed a second time after repeating steps (a)-(d), one of the following occurs: the first overlay image is no longer superimposed on received images, and an error message is generated.

11. A camera display system for a vehicle, the camera display system comprising: an image sensor configured to capture video images including a plurality of sequential image frames; and a display system comprising: a first overlay buffer in which a first overlay image is built and stored; a second overlay buffer in which a second overlay image is built and stored;an image processor configured to:(a) build the first overlay image in the first overlay buffer,(b) receive an image frame from the image sensor,(c) read the first overlay image from the first overlay buffer,(d) perform an integrity check on the first overlay image each time it is read from the first overlay buffer to ensure the first overlay image has not become corrupted,(e) when the integrity check is passed, superimpose the first overlay image onto the image frame and repeat steps (b)-(d),(f) when the integrity check is failed, repeat steps (a)-(d),(g) when a second overlay image different from the first overlay image is required, building and storing the second overlay image in the second overlay buffer,(h) receive an image frame from the image sensor,(i) read the second overlay image from the second overlay buffer,(j) perform the integrity check of the second overlay image each time the second overlay image is read from the second overlay buffer,(k) when the integrity check is passed, superimpose the second overlay image onto the image frame and repeat steps (h)-(j), and(l) when the integrity check is failed, repeat steps (g)-(j); and a display coupled to the image processor for displaying the images received from the image processor.

12. The camera display system of claim 11, wherein the image processor performs the integrity check by: calculating a cyclic redundancy check (CRC) for the first or second overlay image the first time it is read from the respective first or second overlay buffer and storing the calculated CRC;for each time thereafter that the first or second overlay image is read from the respective first or second overlay buffer, calculating a CRC for the overlay image and comparing the calculated CRC to the stored CRC; when the calculated CRC is the same as the stored CRC, the integrity check is passed; and when the calculated CRC is not the same as the stored CRC, the integrity check is failed.

13. The camera display system of any one of claims 11 and 12 and further comprising a user interface for receiving user inputs to change the overlay image14. The camera display system of claim 13, wherein the image processor comprises: an overlay render engine for building the first overlay image in the first overlay buffer and the second overlay image in the second overlay buffer; and a display controller coupled to the user interface, the overlay render engine, the first overlay buffer, and the second overlay buffer, the display controller is configured to determine what overlay images to generate in response to inputs received from one of the user interface and a vehicle bus.

15. The camera display system of any one of claims 11-14, wherein the image processor comprises a memory for receiving images from the image sensor.

16. The camera display system of claim 15, wherein the image processor comprises a field programmable gate array (FPGA) coupled to the image sensor, the memory, the first overlay buffer, the second overlay buffer, and the display controller, wherein the FPGA performs the integrity check.

17. The camera display system of any one of claims 11-16, wherein if the integrity check is failed a second time after repeating steps (a)-(d) or a second time after repeating steps (g)-(j), one of the following occurs: the first overlay image is no longer superimposed on received images, and an error message is generated.

18. A method for verifying overlay buffer integrity, the method comprising:(a) building and storing a first overlay image into a first overlay buffer;(b) reading the first overlay image from the first overlay buffer each time a video image frame is received from an image sensor;(c) performing an integrity check of the first overlay image each time the first overlay image is read from the first overlay buffer;(d) when the integrity check is passed, superimpose the first overlay image onto a video image frame and repeat steps (b) and (c); and(e) when the integrity check is failed, repeat steps (a)-(c).

19. The method of claim 18, further comprising:(f) when a second overlay image different from the first overlay image is required, building and storing the second overlay image into a second overlay buffer;(g) reading the second overlay image from the second overlay buffer each time a video image frame is received from the image sensor;(h) performing the integrity check of the second overlay image each time the second overlay image is read from the second overlay buffer;(i) when the integrity check is passed, superimposing the second overlay image onto a video image frame and repeating steps (g) and (h); and(j) when the integrity check is failed, repeating steps (f)-(h).

20. The method of any one of claims 18 and 19, wherein the integrity check is performed by: calculating a cyclic redundancy check (CRC) for first overlay image the first time it is read from the first overlay buffer and storing the calculated CRC;for each time thereafter that the first overlay image is read from the first overlay buffer, calculating a CRC for the overlay image and comparing the calculated CRC to the stored CRC; when the calculated CRC is the same as the stored CRC, the integrity check is passed; and when the calculated CRC is not the same as the stored CRC, the integrity check is failed.

Images