Detecting Outdated Videos

Abstract

An apparatus and corresponding system are disclosed. The apparatus may include a display. Additionally, the apparatus and / or system may be operable to detect stale images. Further, stale images may be detected in apparatus and / or systems where incoming images are received at a frame rate different than a frame rate of a display, such as from an imager. The controller may detect stale images by assigning a hash value to each frame of image displayed by the display, storing the new hash value in memory, and maintaining a counter. The counter may be maintained such that repeatedly assigning a hash value increments the counter by one and assigning a new hash value resets the counter to zero. Further, a stale image may be determined based at least in part on the counter value.

Description

[Technical Field]

[0001] [CROSS-REFERENCE TO RELATED APPLICATIONS] This application claims the benefit of and priority to U.S. Provisional Patent Application No. 63 / 215,582, entitled "Stale Video Detection," filed June 28, 2021, the disclosure of which is incorporated herein by reference in its entirety.

[0002] The present invention relates generally to video systems, and more particularly to video systems operable to detect outdated images. [Background technology]

[0003] In some cases, video systems may erroneously present "stale video." Stale video generally refers to old video that has not been updated with new frames. This may be particularly problematic for displays integrated into a vehicle's rearview mirror assembly. Specifically, presenting stale video to vehicle occupants may pose a significant safety risk. However, ensuring that stale video is not presented by a vehicle video system presents complications because the incoming frame rate from a camera, such as a rear or backup camera, is often variable, while the display frame rate remains constant. Therefore, the display and camera may have different frame rates at any given time. As a result, if the display frame rate is greater than the camera frame rate, some repeated frames can be expected due to the display advancing to a new frame before a new image is received from the camera. Summary of the Invention

[0004] In accordance with the present disclosure, problems associated with detecting stale video in systems where the camera and display may have different frame rates are substantially reduced or eliminated.

[0005] According to one aspect, a system is disclosed. The system may include a camera, a buffer, a display, and / or a controller. The camera may be operable to capture a series of images at a first frame rate. In some embodiments, the first frame rate may be variable. The buffer may have a first number of memory storage slots. Additionally, the buffer may be operable to continuously and cyclically store the most recent image received from the camera at each time interval into the memory storage slot at the frequency of the time interval. The display may be operable to continuously and cyclically present images from the memory storage slot at a second frame rate. In some embodiments, the second frame rate may be greater than the first frame rate. Consequently, the first frame rate may be less than the second frame rate. In some embodiments, the first number may be determined based at least in part on the highest expected ratio between the first frame rate and the second frame rate. The controller may be operable to assign a hash value to each frame of image displayed by the display, store the new hash value in memory, maintain a counter, and determine whether one or more images in the buffer are stale. In some embodiments, the controller may be further operable to avoid storing repeated hash values ​​in the memory. In some embodiments, the memory may retain only a second number of stored new hash values, thereby retaining only the most recently stored new hash value. The second number may be greater than or equal to the first number. A counter may be maintained such that assigning a repeated hash value increments the counter by one and assigning a new hash value resets the counter to zero. Further, a stale image may be determined based at least in part on the counter value. In some embodiments, a determination that one or more images in the buffer are stale may further be based at least in part on the counter value exceeding the first number. In some embodiments, the controller may be further operable to prevent stale images from being displayed by the display.Additionally or alternatively, the display may be operable to display a notice related to the determination of the buffer having one or more stale images.

[0006] According to another aspect of the present disclosure, an apparatus is disclosed. In some embodiments, the apparatus may be a rearview assembly for a vehicle. The apparatus may further include a display and a controller. The display may be operable to display images at a first frame rate. The controller may be operable to receive images at a second frame rate. In some embodiments, the images may be operatively received from a camera having a rearward field of view relative to a vehicle in which the apparatus may be disposed. In some embodiments, the first frame rate and the second frame rate may be different. Additionally, the second frame rate may be variable. Additionally, the controller may be further operable to continuously and cyclically store the most recently received images at a time interval frequency in a buffer having a first number of memory storage slots. Additionally, the display may be further operable to continuously and cyclically display the images from the memory storage slots. Furthermore, the controller may be operable to assign a hash value to each frame of image displayed by the display, store the new hash value in memory, maintain a counter, and determine whether one or more images in the buffer are stale. In some embodiments, the controller may be further operable to avoid repeatedly storing hash values ​​in memory. In some embodiments, the memory may retain only a second number of stored new hash values, thereby retaining only the most recently stored new hash value. The second number may be greater than or equal to the first number. A counter may be maintained such that assigning a repeated hash value increments the counter by one and assigning a new hash value resets the counter to zero. Further, a stale image may be determined based at least in part on the counter value. In some embodiments, a determination that one or more images in the buffer are stale may further be based at least in part on the counter value exceeding the first number. In some embodiments, the controller may be further operable to prevent stale images from being displayed by the display.Additionally or alternatively, the display may be operable to display a notice related to the determination of the buffer having one or more stale images.

[0007] According to another aspect, a system is disclosed. The system may include a camera, a buffer, a video recorder, and / or a controller. The camera may be operable to capture a series of images at a first frame rate. In some embodiments, the first frame rate may be variable. The buffer may have a first number of memory storage slots. Additionally, the buffer may be operable to continuously and cyclically store the most recent image received from the camera at each time interval into the memory storage slot at the frequency of the time interval. The video recorder may be operable to continuously and cyclically record images from the memory storage slot at a second frame rate. In some embodiments, the second frame rate may be greater than the first frame rate. Consequently, the first frame rate may be less than the second frame rate. In some embodiments, the first number may be determined based at least in part on the highest expected ratio between the first frame rate and the second frame rate. The controller may be operable to assign a hash value to each frame of image recorded by the video recorder, store the new hash value in memory, maintain a counter, and determine whether one or more images in the buffer are stale. In some embodiments, the controller may be further operable to avoid storing repeated hash values ​​in the memory. In some embodiments, the memory may retain only a second number of stored new hash values, thereby retaining only the most recently stored new hash value. The second number may be greater than or equal to the first number. A counter may be maintained such that assigning a repeated hash value increments the counter by one and assigning a new hash value resets the counter to zero. Further, stale images may be determined based at least in part on the counter value. In some embodiments, the determination that one or more images in the buffer are stale may further be based at least in part on the counter value exceeding the first number. In some embodiments, the controller may be further operable to prevent stale images from being recorded by the video recorder.Additionally or alternatively, the video recorder may be operable to record a notification relating to the determination of the buffer having one or more stale images.

[0008] According to yet another aspect of the present disclosure, an apparatus is disclosed. In some embodiments, the apparatus may be a rearview assembly for a vehicle. Further, the apparatus may include a video recorder and a controller. The video recorder may be operable to record images at a first frame rate. The controller may be operable to receive images at a second frame rate. In some embodiments, the images may be operatively received from a camera having a rearward field of view relative to a vehicle in which the apparatus may be disposed. In some embodiments, the first frame rate and the second frame rate may be different. Additionally, the second frame rate may be variable. Additionally, the controller may be further operable to continuously and cyclically store the most recently received images at a time interval frequency in a buffer having a first number of memory storage slots. Additionally, the video recorder may be further operable to continuously and cyclically record images from the memory storage slots. Furthermore, the controller may be operable to assign a hash value to each frame of image recorded by the video recorder, store the new hash value in memory, maintain a counter, and determine whether one or more images in the buffer are stale. In some embodiments, the controller may be further operable to avoid storing repeated hash values ​​in the memory. In some embodiments, the memory may retain only a second number of stored new hash values, thereby retaining only the most recently stored new hash value. The second number may be greater than or equal to the first number. A counter may be maintained such that assigning a repeated hash value increments the counter by one and assigning a new hash value resets the counter to zero. Further, stale images may be determined based at least in part on the counter value. In some embodiments, the determination that one or more images in the buffer are stale may further be based at least in part on the counter value exceeding the first number. In some embodiments, the controller may be further operable to prevent stale images from being recorded by the video recorder.Additionally or alternatively, the video recorder may be operable to record a notification relating to the determination of the buffer having one or more stale images.

[0009] These and other aspects, objects, and features of the present disclosure will be understood and appreciated by those skilled in the art upon review of the following specification, claims, and accompanying drawings. It will also be understood that features of each embodiment disclosed herein may be used in conjunction with, or in place of, features in other embodiments. [Brief explanation of the drawings]

[0010]

[0011] [Figure 1] FIG. 1 is a schematic diagram of a video system. DETAILED DESCRIPTION OF THE INVENTION

[0012] For purposes of description herein, it is to be understood that the specific devices and processes illustrated in the accompanying drawings, and described in the following specification, are merely exemplary embodiments of the inventive concepts defined in the appended claims. Hence, specific dimensions and other physical characteristics related to the embodiments disclosed herein are not to be considered limiting, unless the claims expressly state otherwise.

[0013] This disclosure is directed to a video system 100. Accordingly, Figure 1 illustrates a schematic diagram of one embodiment of video system 100. Video system 100 may include an imager 110, a buffer 120, a display 130, a controller 140, and / or a video recorder 150. In some embodiments, video system 100 may be incorporated into a vehicle.

[0014] Imager 110 may be any device configured and / or operable to capture light and generate a plurality of corresponding images. For example, imager 110 may be a camera. As a result, imager 110 may be a pixel sensor using semiconductor charge-coupled device (CCD) or complementary metal-oxide semiconductor (CMOS) technology. In some embodiments, images may be captured continuously as a video stream. Therefore, the images may be captured according to a first frame rate. The first frame rate may be approximately 30 frames per second. In some embodiments, the first frame rate may be substantially constant. In other embodiments, the first frame rate may be variable. In some such embodiments, the first frame rate may vary based at least in part on the imaging conditions. In some embodiments, the images captured by imager 110 may have a field of view corresponding to a scene external to the vehicle. As a result, the field of view may correspond to a field of view traditionally associated with an interior rearview assembly, a driver's side exterior rearview assembly, a passenger's side exterior rearview assembly, or a backup camera. Therefore, the view may be a rearward and / or sideward view relative to the vehicle.

[0015] The buffer 120 comprises a first number of memory storage slots 121. Accordingly, the buffer 120 may be stored in computer memory, such as random access memory (RAM). Furthermore, the computer memory may be a non-transitory computer-readable medium (CRM). Additionally, the buffer 120 may be communicatively connected to the imager 110. As used herein, the phrase "communicatively connected" may mean directly connected or indirectly connected through one or more electrical components. Accordingly, the buffer 120 may be configured and / or operable to receive images from the imager 110. Consequently, the buffer 120 may be configured and / or operable to continuously and cyclically store the most recent complete image received from the imager 110 in one of the memory storage slots 121 at a time interval. The time interval may have a frequency. In some embodiments, the frequency may be substantially constant. Thus, at each time interval, the most recent image is stored in the storage slot 121. For example, at a first time interval, the most recent image received from the imager 110 may be the first image. Thus, the first image may be stored in the first storage slot 121-1. Thereafter, at a second time interval, the most recent image received from imager 110 may then be stored in the second storage slot 121-2. In some cases, this image may again be the first image if buffer 120 has not yet completely received the second image from imager 110. Alternatively, this image may be the second image if the second image has been completely received from imager 110. The second image may be an image recorded subsequently to the first image. This process may be repeated through subsequent time intervals and storage slots (e.g., 121-3, 121-4, 121-5, ... 121-n) until an image is stored in the last storage slot 121-n of the first number of storage slots 121. Once an image has been stored in the last storage slot 121-n, the buffer 120 may, at the next time interval, cycle back to the beginning and store the most recent image in the first storage slot 121-1, i.e., overwriting the first image.

[0016] Display 130 may be any device configured and / or operable to present one or more digital images for viewing by a user or passenger. In some embodiments, display 130 may be configured and / or operable to retrieve images directly from buffer 120 for presentation. Consequently, display element 130 may be LCD, LED, OLED, microLED, plasma, DLP, or another technology. In some embodiments, display element 130 may be disposed within a rearview assembly, such as an interior rearview assembly of a vehicle. Images presented by display 130 may be presented at a second frame rate. Thus, images presented by display 130 may be a video stream. Additionally, the second frame rate may be substantially constant. In some embodiments, the second frame rate may be approximately 60 frames per second. Consequently, the second frame rate may be substantially greater than the first frame rate. In some embodiments, the second frame rate may be substantially equal to the time interval frequency. In some embodiments, the first number may be determined at least in part based on the highest expected ratio between the first frame rate and the second frame rate. Further, display 130 may be communicatively connected to buffer 120. As a result, frames presented by display 130 may be retrieved continuously and cyclically from storage slots 121. For example, an image stored in first storage slot 121-1 may be presented in the first frame, followed by an image stored in second storage slot 121-2 in the second frame. This may continue until the image stored in nth storage slot 121-n is presented in the nth frame. Furthermore, after presenting the image in nth storage slot 121-n, display 130 may cyclically return to first storage slot 121-1 and present the image stored therein as frame n+1 of the video stream presented by display 130, and the process may proceed again through storage slots 121, and so on.

[0017] Controller 140 may be communicatively coupled to imager 110, buffer 120, and / or display 130. In some embodiments, controller 140 may be configured and / or operative to receive one or more images from imager 110. In some such embodiments, controller 140 may be configured and / or operative to continuously and cyclically store the most recent complete image received from imager 110 in memory storage slot 121 at time intervals. In some embodiments, controller 140 may be configured and / or operative to provide one or more images to display 130 for presentation. In some such embodiments, controller 140 may be configured and / or operative to continuously and cyclically retrieve one or more images from memory storage slot 121 and relay them to display 130. Further, controller 140 may comprise memory 141 and processor 142.

[0018] Memory 141 may be a non-transitory computer-readable medium (CRM). As a result, memory 141 may be configured and / or operable to store one or more instructions, such as one or more algorithms, to provide configuration and operation of controller 140. Examples of memory 141 include a conventional hard disk, solid-state memory, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electronically erasable programmable read-only memory (EEPROM), optical or magnetic disk, and dynamic random access memory (DRAM). In some embodiments, memory 141 may include buffer 120. In other embodiments, memory 141 and buffer 120 may be separate entities. Furthermore, memory 141 may store one or more algorithms. The algorithms may be configured and / or operable to detect stale images. To detect stale images, the algorithm may be configured and / or operable to assign a hash value to images added to memory storage slots 121 of buffer 120 and / or to each frame of images presented by display 130. In some embodiments, the hash may be a cyclic redundancy check (CRC) hash.

[0019] The assigned hash value may be stored in memory 141. In some embodiments, controller 140 may only store new hash values ​​in memory 141. Thus, in such an embodiment, controller 140 may not store repeated or duplicate hash values ​​in memory 141. Additionally, in some embodiments, controller 140 may retain only a second number of stored hash values. The second number of retained stored hash values ​​may be only the most recent stored hash value. In some embodiments, the second number may be greater than or equal to the first number. Consequently, in some such embodiments, the second number may be equal to the first number.

[0020] Additionally, the algorithm may be configured and / or operable to maintain a counter. An assigned hash value that matches any previously assigned hash value stored in memory 141 increments the counter by one. Conversely, an assigned hash value that does not match any previously assigned hash value resets the counter to zero. Further, the algorithm may be configured and / or operable to determine that one or more images in the buffer are stale based at least in part on the counter value exceeding the number of frames stored in the buffer.

[0021] A processor 142 may be communicatively coupled to the memory 141. Furthermore, the processor 142 may be any device or electronic circuit configured and / or operable to process or execute one or more sets of electronic instructions, such as algorithms. These instructions may be stored in the memory 141. Examples of the processor 142 may include a central processing unit (CPU), a microprocessor, and / or an application specific integrated circuit (ASIC).

[0022] In some embodiments, the algorithm may be operable to prevent stale images from being presented by display 130. Based at least in part on determining that one or more of the images in buffer 120 are stale, controller 140 may prevent the images from being presented by display 130. In some such embodiments, controller 140 may send an error notification for presentation by display 130.

[0023] Video recorder 150 may be any device configured and / or operable to record, store, and / or save a series of received images. For example, video recorder 150 may be part of a black box for a vehicle. As such, video recorder 150 may comprise computer memory. The computer memory may be identical to or substantially similar to memory 141. Additionally, video recorder 150 may be communicatively coupled to imager 100, buffer 120, display 130, and / or controller 140. The received images may be received and saved as a video stream at a third frame rate. The video stream may include a series of image frames. Additionally, the third frame rate may be substantially constant. In some embodiments, the third frame rate may be approximately 60 frames per second. As a result, the third frame rate may be substantially greater than the first frame rate. In some embodiments, the third frame rate may be substantially equal to the time interval frequency. In some embodiments, the first number may be determined based at least in part on the highest expected ratio between the first frame rate and the third frame rate. In some embodiments, the series of received images may be continuously and cyclically retrieved from the storage slots 121 of the buffer 120. For example, the image stored in the first storage slot 121-1 may be recorded in the video recorder 150 as the first frame, after which the image stored in the second storage slot 121-2 may be recorded as the second frame. This may continue until the image stored in the nth storage slot 121-n is presented as the nth frame. Furthermore, after recording the image in the nth storage slot 121-n, the video recorder 150 may cyclically return to the first storage slot 121-1 and record the image stored therein as the n+1 frame of the video stream being recorded by the video recorder 150, and the process may proceed again through the storage slots 121, and so on. In some embodiments, video recorder 150 may be configured and / or operable to retrieve images directly from buffer 120 for recording.In other embodiments, video recorder 150 may be configured and / or operable to indirectly retrieve images for recording from display 130. In yet other embodiments, video recorder 150 may be configured and / or operable to indirectly retrieve images for recording from controller 140. In yet other embodiments, video recorder 150 may be part of controller 140, such as part of memory 141.

[0024] In some embodiments, the algorithm may be configured and / or operable to prevent stale images from being recorded by video recorder 150. As a result, based at least in part on determining that one or more of the images in buffer 120 are stale, controller 140 may prevent the images from being recorded by video recorder 150. In some such embodiments, controller 140 may send an error notification to the recording by video recorder 150.

[0025] Embodiments of the video system 100 may have one or more advantages. The video system 100 may enable the display 130 to present images captured by the imager 110 when the imager 110 and the display 130 have different frame rates. Furthermore, stale video may be detected and / or prevented despite the frame rate difference, significantly increasing safety. The difference in frame rates between the imager 110 and the display 130 means that some image repetitions are expected. For example, if the display 130 has a frame rate twice that of the imager 110, one would expect an image captured by the imager 100 to be presented by the display 130 approximately twice, due to the display 130 advancing through two frames in the time it takes for a new image to be captured by the imager 110. This is further complicated when the frame rate of the imager 110 is variable. Therefore, stale video may not be detected simply due to the presence of repetitions in the image presentation, as false detections would commonly occur. However, the counter may enable detection of stale video by detecting that images are being repeated at a greater amount than would be expected based on differences in frame rates. Additionally, video system 100 may enable video recorder 150 to record images captured by imager 110 even if some video recorders 150 are required to record the recorded video at a frame rate different from the frame rate provided by imager 110. Furthermore, video recorder 150 may record images captured by imager 110 at a substantially constant frame rate, as opposed to the variable frame rate provided by imager 110. Additionally, only storing new hash values ​​may have the advantage of substantially reducing memory requirements. This advantage has an increasing impact as frame rate differences increase.This is because as the difference in frame rates increases, the number of expected repetitions also increases due to the faster frame advancement of the display 130 relative to the imager 110. These repeated images will have the same hash value, and as a result, a smaller fraction of the assigned hash will be stored.

[0026] As used herein, the term "and / or" when used in connection with a list of two or more items means that any one of the listed items may be used by itself or any combination of two or more of the listed items may be used. For example, if a composition is described as containing components A, B, and / or C, the composition may contain A alone; B alone; C alone; A and B in combination; A and C in combination; B and C in combination; or A, B, and C in combination.

[0027] The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus comprising a list of elements may include not only those elements, but also other elements not expressly listed or inherent in such process, method, article, or apparatus. An element prefaced by "comprises a..." does not preclude the presence of additional identical elements in a process, method, article, or apparatus that comprises that element, without further constraints.

[0028] The term "substantially" and variations thereof are understood by those skilled in the art to describe a feature that is equal or approximately equal to a value or description. For example, a "substantially flat" surface is intended to indicate a flat or approximately flat surface. Furthermore, "substantially" is intended to indicate that two values ​​are equal or approximately equal. If there is a use of the term that is not clear to those skilled in the art, "substantially" may refer to values ​​that are within about 10% of each other, such as within about 5% of each other, or within about 2% of each other, in light of the context in which it is used.

[0029] While several embodiments have been described in this disclosure, numerous variations, modifications, adaptations, and alterations may be apparent to those skilled in the art. It should be understood that the present disclosure is intended to encompass all such variations, modifications, adaptations, and alterations that fall within the scope of the appended claims, unless such language expressly dictates otherwise.

Claims

1. 1. A system comprising: a camera configured to capture a series of images at a first frame rate; a buffer having a first number of memory storage slots; a display configured to continuously and cyclically present images from said memory storage slot at a second frame rate; A controller; Equipped with the buffer is configured to continuously and cyclically store the most recent image received from the camera for each time interval in the memory storage slot at a frequency of the time interval; The controller assigning a hash value to each frame of image displayed by said display; Store the new hash value in memory; maintaining a counter, wherein repeatedly assigning a hash value increments the counter by one, and assigning a new hash value resets the counter to zero; It is determined, at least partially, that one or more images in the buffer are outdated, based on the value of the counter. It is configured as follows: A system characterized by:

2. The system of claim 1 , wherein the controller is further configured to avoid storing repeated hash values ​​in the memory.

3. 2. The system of claim 1, wherein the memory retains only a second number of the stored new hash values, thereby retaining only the most recently stored new hash value.

4. The system of claim 3 , wherein the second number is greater than or equal to the first number.

5. The system of claim 4 , wherein the second number is equal to the first number.

6. The system of claim 1 , wherein the determination that one or more images in the buffer are out of date is further based at least in part on the value of the counter exceeding the first number.

7. The system of claim 1 , wherein the first frame rate is variable.

8. The system of claim 1 , wherein the first frame rate is less than the second frame rate.

9. The system of claim 1 , wherein the first number is determined based at least in part on a highest expected ratio between the first frame rate and the second frame rate.

10. The system of claim 1 , wherein the controller is further configured to prevent outdated images from being displayed by the display.

11. The system of claim 1 , wherein the display is configured to display a notification related to the determination of the buffer having one or more stale images.

12. 1. A system comprising: a camera configured to capture a series of images at a first frame rate; a buffer having a first number of memory storage slots; a video recorder configured to continuously and cyclically record images from said memory storage slot at a second frame rate; A controller; Equipped with the buffer is configured to continuously and cyclically store the most recent image received from the camera for each time interval in the memory storage slot at a frequency of the time interval; The controller assigning a hash value to each frame of image recorded by the video recorder; Store the new hash value in memory; maintaining a counter, wherein repeatedly assigning a hash value increments the counter by one, and assigning a new hash value resets the counter to zero; It is determined, at least partially, that one or more images in the buffer are outdated, based on the value of the counter. It is configured as follows: A system characterized by:

13. The system of claim 12 , wherein the controller is further configured to avoid storing repeated hash values ​​in the memory.

14. the memory retaining only a second number of the stored new hash values, thereby retaining only the most recently stored new hash value; The system of claim 12 , wherein the second number is greater than or equal to the first number.

15. The system of claim 14 , wherein the second number is equal to the first number.

16. The system of claim 12 , wherein the determination that one or more images in the buffer are out of date is further based at least in part on the value of the counter exceeding the first number.

17. The system of claim 12 , wherein the first frame rate is variable.

18. The system of claim 12 , wherein the first frame rate is less than the second frame rate.

19. The system according to claim 12, wherein the first number is determined at least in part based on the highest expected ratio between the first frame rate and the second frame rate.

20. The system according to claim 12, wherein the controller is further configured to prevent old images from being recorded by the video recorder.

Images