DISPLAY CONTROL DEVICE

The display control device addresses the uncertainty of system failures by integrating units to generate and synthesize videos with periodic changes, allowing users to visually distinguish normal and failure states, thereby improving safety and reducing anxiety.

FR3149107A1Pending Publication Date: 2024-11-29YAZAKI CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
FR2024004604
Authority / Receiving Office
FR · FR
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-22
Filing Date
2024-05-02
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

Existing display systems in vehicles fail to provide clear indication of normal operation or failure states, such as paralysis, leading to uncertainty and potential safety risks during driving.

Method used

A display control device that integrates a main video generation unit, a sub-video generation unit generating a periodically changing sub-video, and a video synthesis unit to output a combined video, allowing users to visually distinguish normal operation from failure states through changes in the sub-video content.

Benefits of technology

Enables users to confidently recognize the absence of failures and promptly identify their occurrence, enhancing safety and reducing anxiety by visually discerning changes in the displayed sub-video content.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000015_0000
    Figure 00000015_0000
  • Figure 00000016_0000
    Figure 00000016_0000
  • Figure 00000017_0000
    Figure 00000017_0000
Patent Text Reader

Abstract

A display control device (10) includes a main video generation unit (11) configured to generate a display video based on a predetermined main video signal, a sub-video generation unit (12) configured to generate a sub-video whose content changes periodically, and a video synthesis unit (14) configured to output a synthesis result of the display video and the sub-video as output video to a predetermined display device (22). At least one update cycle of the sub-video in the output video falls within a range of one cycle in which a user can visually recognize a change in the content of the sub-video. Figure for the abstract: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

Title of the invention: DISPLAY CONTROL DEVICE Technical field

[0001] The present invention relates to a display control device. STATE OF THE ART

[0002] In a vehicle, in order to support safe driving of a driver, for example, there is a display device that displays information, such as the presence of an obstacle and a pedestrian detected by a computer on the basis of real-time video captured by an on-board camera, to the driver in an easy-to-understand manner. Furthermore, in order to improve the convenience for the driver, a display system that controls the display of various types of information on the vehicle by an operation in which an on-board device and a mobile terminal cooperate with each other is being developed.

[0003] On the other hand, in various display devices that perform complicated processing using a computer or the like, there is a possibility that the operation is temporarily stopped due to various causes and normal display cannot be performed. For example, when a paralysis state occurs due to abnormal heat generation, resource shortage or the like on a computer on which a plurality of application software (applications) are activated, it is provided that all operations of the plurality of applications are at least temporarily stopped. In this case, important information such as an obstacle and a pedestrian are not displayed on the vehicle display system, and it is difficult to ensure safe driving. Furthermore, when the paralysis state occurs, the driver or the like cannot know that an abnormality such as a failure is occurring in the system.

[0004] As a technique for performing notification of the occurrence of the paralysis state, for example, there is a display control device disclosed in JP2018-121238A. The display control device alternately synthesizes two types of warning images for each of continuous frames of a video captured by a camera. The warning image is an image in which a notification image for warning of the occurrence of paralysis is set on a background image, and the two types of warning images are images in which display colors of the notification image and the background image are mutually reversed. According to the display control device, in a normal state in which paralysis does not does not occur, the warning is displayed as an image of an intermediate color so as not to be recognized, and in the event that paralysis occurs, the warning is perceived by fixedly displaying any of the warning images.

[0005] According to the technique of JP2018-121238A, when paralysis occurs, the occurrence of paralysis can be recognized since the warning image is displayed fixedly. However, in a normal state in which paralysis does not occur, there is no change in the image of the intermediate color between a notification image color and a background image color, a user may feel worried, because it is not possible to confirm that paralysis does not occur.

[0006] The present invention provides a display control device capable of causing a user to recognize that a fault such as paralysis does not occur and causing the user to correctly recognize the fault when the fault occurs. SUMMARY OF THE INVENTION

[0007] According to the present invention, a display control device comprises a main video generation unit configured to generate a display video on the basis of a predetermined main video signal, a sub-video generation unit configured to generate a sub-video whose content changes periodically, and a video synthesis unit configured to output a synthesis result of the display video and the sub-video as an output video to a predetermined display device. At least one update cycle of the sub-video in the output video is within a range of a cycle in which a user is able to visually recognize a change in the content of the sub-video.

[0008] According to the display control device of the present invention, it is possible to cause the user to recognize that the fault such as paralysis does not occur, and to cause the user to correctly recognize the fault when the fault occurs.

[0009] The present invention has been briefly described above. Furthermore, details of the present invention will be further clarified upon reading embodiments of the invention described below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] [Fig. 1] is a block diagram showing an example configuration of an in-vehicle display system comprising a display control device according to one embodiment of the present invention; [Fig.2] is a block diagram showing components inside a video display device; [Fig.3] is a state transition diagram showing an example of a content transition of displaying an additional image for confirmation; [Fig.4] is a front view showing an example of a screen display in a display device; and [Fig.5] is a block diagram showing another example of the on-board display system configuration. DESCRIPTION OF EMBODIMENTS

[0011] Specific embodiments of the present invention will be described below with reference to the drawings. <Configuration de dispositif de commande d’affichage> [Fig.l] shows an example configuration of an in-vehicle display system 100 comprising a display control device according to an embodiment of the present invention.

[0012] The in-vehicle display system 100 shown in [Fig.l] comprises a video display device 10, an external video generating device 20, a display driver 21, and a display device 22. For example, a mobile terminal such as a smartphone carried by a driver in the vehicle or a predetermined electronic control unit (ECU) installed on the vehicle can be used as the video display device 10.

[0013] The external video generating device 20 may generate, for example, real-time video captured by an on-board camera mounted on a subject vehicle, video generated on a vehicle side, or the like as external video SGI. The external video SGI generated by the external video generating device 20 is input to an external video input 10a of the video display device 10.

[0014] The video display device 10 can output, to a video output 10b, an internal video generated therein or a synthesized video SG5 generated on the basis of the external video SGI inputted to the external video input 10a. The synthesized video SG5 is inputted to the display device 22 via the display driver 21.

[0015] For example, an on-board display disposed near a dashboard of the vehicle or a display provided in the mobile terminal may be used as the display device 22. The display driver 21 functions as an interface necessary for connecting the video display device 10 to the display device 22.

[0016] The video display device 10 shown in [Fig.l] comprises, as main components, a video display application 11, a fault recognition display application 12, an internal video storage region 13, and a video synthesis unit 14. The internal video storage region 13 stores data of an internal video prepared in advance in a predetermined storage device.

[0017] The video display application 11 is application software integrated in advance with the video display device 10. The video display application has a function of outputting, as the main display video SG3, the internal video generated on the basis of the internal video data SG2 inputted by the internal video storage region 13 or the external video SGI according to a situation inputted into the external video input 10a.

[0018] The fault recognition display application 12 is application software for causing the user to recognize the occurrence of a fault such as paralysis when the fault occurs. The fault recognition display application 12 is also incorporated in advance into the video display device 10. The fault recognition display application 12 has the function of always generating and outputting a supplementary image SG4 whose display content is updated in a constant cycle.

[0019] The video synthesis unit 14 has the function of synthesizing the main display video SG3 outputted by the video display application 11 and the additional image SG4 outputted by the fault recognition display application 12 on the same video frame and outputting a synthesized result as synthesized video SG5. The function of the video synthesis unit 14 may be implemented by application software or may be implemented by an electronic component such as an image synthesis integrated circuit (IC).

[0020] In the video display device 10, there is a possibility that a temporary failure such as paralysis occurs due to the occurrence of abnormal heat generation or resource shortage. This failure is assumed to occur within the entire area of ​​an expected failure range 15 illustrated in [Fig.l].

[0021] When the video display device 10 is operating normally, the content of the supplementary image SG4 is periodically updated, and the content is reflected in the synthesized video SG5. In this case, since the user such as the driver can visually recognize the periodic change in the content of the supplementary image SG4 provided in the synthesized video SG5 displayed on a screen of the display device 22, the user can recognize that the operation of the system is normal. Thus, the user can feel a sense of security.

[0022] On the other hand, when the failure such as paralysis occurs in the video display device 10, the function in the expected failure range 15 stops, and the change of the content of the supplementary image SG4 in the synthesized video SG5 also stops. Accordingly, the user such as the driver can confirm visually that the content of the additional image SG4 provided in the synthesized video SG5 displayed on the screen of the display device 22 does not change, and recognize that an anomaly occurs in the operation of the system.

[0023] <Exemple de configuration interne de dispositif d’affichage vidéo> [Fig.2] shows an example of the internal configuration of the video display device 10 of [Fig.l].

[0024] For example, as the video display device 10 illustrated in [Fig.2], an electronic device as a mobile terminal such as a smartphone or various on-board ECUs often includes various hardware resources, basic software (OS), and various application software.

[0025] The video display device 10 of [Fig. 2] comprises electronic components such as a central processing unit (CPU), a memory, and a timer as hardware resources HW and electronic circuits. By running on the base software SW, the various application software can easily secure the resources required to perform the respective operations as needed.

[0026] The video display device 10 of [Fig. 2] comprises a standard application API and a supplementary application AP2 as various application software. The standard application API and the supplementary application AP2 correspond to the video display application 11 and the fault recognition display application 12 illustrated in [Fig. 1], respectively.

[0027] Accordingly, in the video display device 10 illustrated in [Fig. 1], the standard application API and the additional application AP2 of [Fig. 2] share at least partially the same basic software SW and hardware resources HW. Furthermore, the functions of the standard application API and the additional application AP2 of [Fig. 2] are executed substantially simultaneously.

[0028] Here, when the processing of the standard application API is executed, for example, if a processing speed decreases due to abnormal heat generation of the CPU or the like or a resource shortage state occurs, the operation of the standard application API may be brought into a paralysis state. Furthermore, since the same basic software SW and hardware resources HW are shared, when the operation of the standard application API enters the paralysis state, an effect thereof also affects the supplementary application AP2, and an operation of the supplementary application AP2 also enters the paralysis state at the same time.

[0029] Accordingly, in the video display device 10 illustrated in [Fig.l], when the operation of the video display application 11 becomes the paralysis state, the operation of the failure recognition display application 12 also becomes the paralysis state at the same time. As a result, a periodic update of the supplementary image SG4 is stopped, and even in the synthesized video SG5 displayed on the display device 22, the displayed supplementary image SG4 is not changed.

[0030] That is, a state in which the content of the supplementary image SG4 in the synthesized video SG5 displayed on the screen of the display device 22 changes indicates that an operation of the expected failure range 15 of the video display device 10 is normal. The state in which the change of the supplementary image SG4 in the synthesized video SG5 is stopped indicates that a failure occurs in the expected failure range 15 of the video display device 10. Accordingly, the user such as the driver can easily and visually recognize the presence or absence of an abnormality in the expected failure range 15 of the video display device 10 from the presence or absence of the change of the supplementary image SG4 displayed on the screen of the display device 22.Furthermore, when a specific condition is satisfied, a state in which the content of the main display video SG3 in the synthesized image SG5 is not changed and the additional image for confirmation 32 is changed indicates that a failure occurs in the external video generation device 20. The specific condition is that the user can know that the main display video SG3 is the external video SGI input by the external video generation device 20 and that the external video SGI is moving. For example, when the external video SGI is a real-time video obtained by capturing images of an exterior of the vehicle by an on-board camera mounted on a moving vehicle, the user recognizes that the external video SGI is moving.According to the in-vehicle display system 100, when the specific condition described above is satisfied, the user can easily and visually recognize the presence or absence of an abnormality in the external video generating device 20 from the presence or absence of a change in the main display video 31 and the additional image for confirmation 32 displayed on the screen of the display device 22. In addition, according to the in-vehicle display system 100, it is possible for the video display device 10 to make a notification of an error or leave an error log on the screen of the display device 22 by detecting that the external video generating device 20 has failed.

[0031] <Transition de contenu d’affichage d’image supplémentaire pour confirmation> [Fig.3] shows an example of the additional image display content transition for confirmation. In the example shown in [Fig. 3], it is assumed that a moving image obtained by combining four types of confirmation icon patterns P12, P13, and P14 is used as the supplementary image SG4 shown in [Fig. 1].

[0032] In the example of [Fig.3], it is assumed that the content of the supplementary image SG4 is periodically switched in the order of PI 1, P12, P13, P14, Pli, P12 using four types of confirmation icon patterns Pli, P12, P13, and P14.

[0033] The confirmation icon patterns P12, P13, and P14 in [Fig. 3] depict an arc-shaped or circumferential arrow pattern indicating an angle range of approximately 90 degrees, 180 degrees, 270 degrees, and 360 degrees, respectively.

[0034] Accordingly, as shown in [Fig. 3], when the confirmation icon patterns P11 to P14 are switched sequentially in a constant cycle, a state in which the operation of the device is normal can be expressed as a moving image in which an arc-shaped arrow appears to rotate. Furthermore, in the paralysis state, the moving image is naturally stopped without requiring special control, so that the user can reliably recognize the occurrence of a fault.

[0035] <Exemple d’affichage d’écran dans un dispositif d’affichage> An example of a screen display in the display device 22 is shown in [Fig. 4], In the example shown in [Fig. 4], the main display video 31 and the additional image for confirmation 32 are simultaneously displayed on a display screen of the display device 22. The main display video 31 corresponds to the external video SGI input by the external video generation device 20 into the video display device 10 or to a video generated by the video display application 11 on the basis of the internal video data SG2.

[0036] The additional image for confirmation 32 corresponds to the additional image SG4 output by the failure recognition display application 12 in the video display device 10. The failure recognition display application 12 repeatedly updates the contents of the additional image SG4 in a constant cycle (e.g., every 0.5 seconds), for example, like the confirmation icon patterns P1 to P14 shown in [Fig. 3] when normal operation is performed.

[0037] Accordingly, the content of the additional image for confirmation 32 displayed on the screen of the display device 22 of [Fig. 4] is also normally updated periodically. On the other hand, for example, when the operation of the video display application 11 becomes paralyzed due to the occurrence of abnormal heat generation or a resource shortage of the video display device 10, the operation of the recognition display application failure 12 also enters the paralysis state at the same time, and the update of the contents of the SG4 supplementary image stops.

[0038] Therefore, for example, on the screen of the display device 22 shown in [Fig. 4], when a periodic change occurs in the content of the additional image for confirmation 32, there is no abnormality in the operation of the system that displays the main display video 31. On the other hand, in a state in which there is no change in the content of the additional image for confirmation 32, there is a high possibility that the operation of the video display application 11 is in the paralysis state, which means that there is a problem with the display content of the main display video 31.

[0039] That is, the user such as the driver can recognize the presence or absence of the abnormality in the operation of the system due to the paralysis by visually recognizing the presence or absence of a change in the content of the additional image for confirmation 32 always displayed on the screen of the display device 22 without confirming the content of the main display video SG3.

[0040] In the example illustrated in [Fig. 4], the additional image for confirmation 32 is always displayed in a display region PO at a predetermined position on the screen, and the content of the additional image for confirmation 32 is updated in a constant cycle. Accordingly, the user can easily visually recognize the additional image for confirmation 32 and the presence or absence of a change thereof.

[0041] <Modification de forme d’affichage d’image supplémentaire SG4> With respect to a change in the display form of the supplementary image SG4 in the synthesized video SG5 displayed on the screen of the display device 22, for example, various modifications as described below can be considered. (1) The supplementary image SG4 is displayed as a moving image so as to express a change by a display technique such as a size, shape, movement, color, 3D, and shadow of any of characters, numbers, symbols, and digits, or any combinations thereof. However, since it is necessary for the user to visually check the change of the moving image easily, it is necessary to arrange the supplementary image SG4 at a position that is easily visually recognizable. A time period for updating the contents of the supplementary image SG4 is set in a range of, for example, about 0.1 seconds to 2 seconds.

[0042] (2) As a moving image in which a display position is periodically moves in any axial direction on three-dimensional coordinates recognizable by the user on the display screen, the image SG4 supplementary image can be modified as to its content. However, it is desirable to limit a moving range of the SG4 supplementary image or to limit a moving speed of the SG4 supplementary image so as not to hinder the user's visual recognition.

[0043] (3) As a pattern shape of the additional image SG4, a circle or a A triangle-like polygon or larger can be used. A moving image in which the pattern of this shape rotates can be adopted as an SG4 supplementary image. In addition, the SG4 supplementary image can be expressed as a moving image in which the color changes in a certain specific region. In addition, the SG4 supplementary image can be expressed as a moving image in which characters and numbers change in a specific region according to a specific rule.

[0044] <Avantages du système d’affichage embarqué 100> For example, there is a case where the driver of the vehicle wants to check the presence of an obstacle and a pedestrian in a region behind a vehicle body, which is a blind spot, by watching the video captured by the on-board camera. In the on-board display system 100 shown in [Fig. 1], the video captured by the on-board camera can be input as external video SGI from the external video generating device 20 into the video display device 10 and displayed on the screen of the display device 22. Normally, for example, in the case where a change in the video captured by the on-board camera is small, the user may feel concern as to whether an actual situation outside the vehicle is correctly displayed or whether a device failure occurs.On the other hand, according to the in-vehicle display system 100, in a state in which no paralysis occurs, even when no change occurs in the main display video 31, since the supplementary image SG4 changes periodically, the user can recognize that no paralysis occurs, and can feel a sense of security.

[0045] On the other hand, in the video display device 10 shown in [Fig. 1], when the operation of the video display application 11 becomes paralyzed, the operation of the fault recognition display application 12 simultaneously becomes paralyzed due to the system structure, and the change of the supplementary image SG4 stops. As a result, the driver can recognize the stoppage of the change of the supplementary image for confirmation 32 on the screen of the display device 22 shown in [Fig. 4], for example. As a result, the driver can understand that the operation of the system is abnormal, check more carefully for the presence of obstacles and pedestrians around the subject vehicle, and try to drive safely.

[0046] Furthermore, in the video display device 10 of [Fig. 1], it is not necessary to cause the function of the fault recognition display application 12 to have the function causing the driver to recognize the presence or absence of a fault independent of resources of the basic hardware and software of the video display application 11. Accordingly, it is not necessary to complicate a circuit configuration of the video display device 10 or increase the number of components in order to enable fault recognition, and an increase in device cost can be avoided.

[0047] <Exemple spécifique de vidéo externe> As described above, the external video generating device 20 can generate the real-time video captured by the on-board camera mounted on the subject vehicle as SGI external video. Specific examples of the SGI external video include, in addition to the video of the region behind the vehicle captured by the on-board camera for a rear monitor, a video all around the vehicle captured by one or a plurality of on-board cameras, a video including at least a front side of the vehicle captured by the on-board camera for a driving recorder, a video captured by the on-board camera for a driver monitor that captures the driver, or the like.

[0048] <Modification de système d’affichage embarqué> In the in-vehicle display system 100, as shown in [Fig. 5], vehicle information VI may be input into an external information input 10c of the video display device 10, and the video display application 11 may generate the main display video SG3 based on the vehicle information VI. Specific examples of vehicle information VI include information related to a vehicle such as the vehicle speed and a battery, information related to parameters of an in-vehicle device such as an air conditioner, a seat, or a head-up display (HUD), and information on a loading platform connected to the vehicle such as the actual temperature of a refrigerated vehicle. The video display application 11 generates the main display video SG3 in which one of the vehicle information VI is superimposed on the internal video data SG2 or the external video SGI.The video display device 10 outputs a synthesis result of the main display video SG3 and the supplementary image SG4 as a synthesized video SG5. According to the in-vehicle display system 100 shown in [Fig. 5], even when there is no change in the main display video 31 including the display of the vehicle information VI, the supplementary image for confirmation 32 changes periodically, and thus the user can recognize that there is no paralysis. In . Consequently, since the user can recognize that the displayed VI vehicle information is correct, the user can feel a sense of security.

[0049] <Description supplémentaire> Here, features of the embodiments of the display control device according to the present invention described above are briefly summarized and listed in the following [1] to [5]. [1] A display control device (a video display device 10) comprising: a main video generation unit (a video display application 11) configured to generate a display video (a main display video SG3) on the basis of a predetermined main video signal (an external video SGI or the like); a sub-video generation unit (a fault recognition display application 12) configured to generate a sub-video (a supplementary image SG4) whose content changes periodically; and a video synthesis unit (14) configured to output a synthesis result of the display video and the sub-video as an output video (synthesized video SG5) to a predetermined display device (a display device 22), wherein at least one update cycle of the sub-video in the output video is within a range of one cycle in which a user is able to visually recognize a change in the content of the sub-video.

[0050] According to the display control device having the above configuration [1], since the content of the sub-video generated by the sub-video generation unit changes periodically, the user can always recognize a change in the sub-video in the output video in a normal state in which no failure occurs, and can feel a sense of security. Furthermore, when the failure such as paralysis occurs, since the change of the sub-video in the output video stops, the user can visually recognize that there is an abnormality from the content of the output video.According to the display control device, even when a camera video or the like in which it is difficult to know whether a main video signal is stopped is displayed, the user can easily understand whether the failure such as paralysis occurs in a system by visually recognizing whether there is a change in the sub-video.

[0051] [2] The display control device according to [1] above, wherein the main video generation unit and the sub-video generation unit are arranged on a common electronic device, and are configured as application software using a common resource as at least a part of hardware (a hardware resource HW) or a part of basic software (SW) in the electronic device.

[0052] According to the display control device having the above configuration [2], since the main video generation unit and the sub-video generation unit are applications using the common resource, when the operation of the main video generation unit becomes paralyzed, an influence thereof naturally propagates and the operation of the sub-video generation unit also enters a paralysis state. Accordingly, there is no need to add a circuit having a complicated configuration for detecting the failure of the main video generation unit, and there is no need to make the hardware of the main video generation unit and the hardware of the sub-video generation unit independent of each other.

[0053] [3] The display control device according to [1] above, wherein the sub-video generation unit generates a moving image (e.g., confirmation icon patterns PI 1 to P14) whose content is updated at a constant time interval as a sub-video.

[0054] According to the display control device having the above configuration [3], since the time interval at which the moving image changes is constant, the user can easily visually recognize whether the moving image to be displayed changes. Furthermore, it is easy to distinguish between the main video and the sub-video.

[0055] [4] The display control device according to [1], wherein the sub-video generation unit controls a display position of the sub-video such that the display position (a display region PO) is constant within a display frame of the output video.

[0056] According to the display control device having the above configuration [4], since the sub-video is normally displayed at substantially the same position, the user can easily grasp the display position of the sub-video in the display frame. As a result, the visibility of the displayed sub-video is improved.

[0057] [5] The display control device according to any one of the points above- above [1] to [4], in which the main video generation unit allows video input from an external on-board device, and generates the display video using the video signal as the main video signal when a video signal from the on-board device is input.

[0058] According to the display control device having the above configuration [5], it is possible to easily configure a display system in which a terminal A mobile device such as a smartphone and an in-vehicle device cooperate with each other to display information such as a video. As a result, the convenience for the user operating the vehicle is improved.

Claims

Claims

1. A display control device comprising: a main video generation unit configured to generate a display video on the basis of a predetermined main video signal; a sub-video generation unit configured to generate a sub-video whose content changes periodically; and a video synthesis unit configured to output a synthesis result of the display video and the sub-video as an output video to a predetermined display device, wherein at least one update cycle of the sub-video in the output video is within a range of a cycle in which a user is capable of visually recognizing a change in the content of the sub-video.

2. The display control device according to claim 1, wherein the main video generation unit and the sub-video generation unit are arranged on a common electronic device, and are configured as application software using a common resource as at least a hardware part or a basic software part in the electronic device.

3. The display control device according to claim 1, wherein the sub-video generation unit generates a moving image whose content is updated at a constant time interval as a sub-video.

4. The display control device according to claim 1, wherein the sub-video generation unit controls a display position of the sub-video such that the display position is constant within a display frame of the output video.

5. The display control device according to any one of claims 1 to 4, wherein the main video generating unit allows video input from an external on-board device, and generates the display video using the video signal as the main video signal when a video signal from the on-board device is input.