Display control device

The display control device addresses the issue of user unease by combining main and periodically changing sub-videos to visually indicate system status, ensuring user awareness of freezes and malfunctions.

JP2024167720AInactive Publication Date: 2024-12-04YAZAKI CORP
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
JP2023083979
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-22
Publication Date
2024-12-04
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing display control devices fail to inform users when a freeze occurs, causing unease due to the lack of visual cues under normal conditions and inability to notify users of malfunctions.

Method used

A display control device that generates a main video and a periodically changing sub-video, combining them to create an output video where the sub-video's update cycle is within a range recognizable by the user, allowing visual confirmation of changes or lack thereof to indicate system status.

Benefits of technology

Enables users to recognize the absence of freezes under normal conditions and detect malfunctions by observing changes in the sub-video, enhancing user confidence and safety.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024167720000001_ABST
    Figure 2024167720000001_ABST
Patent Text Reader

Abstract

To provide a display control device that allows a user to recognize that a failure such as freezing has not occurred and to correctly recognize a failure when the failure occurs.SOLUTION: A display control device comprises: an application 11 of a main video generation unit that generates a display video on the basis of a main video signal; an application 12 of a sub-video generation unit that generates a sub-video whose content periodically changes; and a video composition unit 14 that outputs a result obtained by combining the display video and the sub-video to a prescribed display instrument as an output video. An update cycle of the sub-video in the output video is set within a range of a cycle in which the user can visually recognize the change in the content. When the main video generation unit freezes and the main video stops, the update of the sub-video also stops simultaneously. The user can correctly determine whether or not the system is operating normally by visually recognizing the presence / absence of the change in the displayed sub-video. The two applications 11, 12 operate on a common OS or shared hardware.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a display control device. [Background technology]

[0002] In vehicles, there are display devices that display information such as obstacles and pedestrians detected by a computer based on real-time images captured by an on-board camera in an easy-to-understand manner to the driver to support safe driving. In addition, display systems that control the display of various information on the vehicle by the cooperation of on-board devices and mobile terminals to improve convenience for the driver are also being developed.

[0003] On the other hand, in various display devices that perform complex processing using a computer or the like, there is a possibility that operation may temporarily stop due to various causes, resulting in a situation in which normal display cannot be performed. For example, when a freeze state occurs on a computer running multiple application software (apps) due to abnormal heat generation or lack of resources, it is expected that operation of all of the multiple apps will at least temporarily stop. In that case, important information such as obstacles and pedestrians will not be displayed on the display system on the vehicle, making it difficult to ensure driving safety. Furthermore, when a freeze state occurs, it will be impossible to inform the driver or the like that an abnormality such as a failure has occurred in the system.

[0004] As a technique for notifying the occurrence of a frozen state, for example, there is a display control device disclosed in Patent Document 1. This display control device alternately synthesizes two types of warning images for each successive frame of an image captured by a camera. The warning image is an image in which a notification image that warns of the occurrence of a freeze is drawn on a background image, and the two types of warning images are images in which the display colors of the notification image and the background image are inverted from each other. According to this display control device, during normal times when no freeze has occurred, the images are displayed as neutral colors so that the warning is not recognized, and when a freeze has occurred, one of the warning images is displayed fixedly, so that the warning is recognized. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2018-121238 A Summary of the Invention [Problem to be solved by the invention]

[0006] According to the technology of Patent Document 1, when freezing occurs, a fixed warning image is displayed, so that the occurrence of freezing can be recognized. However, in a normal state where freezing does not occur, the image, which is an intermediate color between the notification image color and the background image color, does not change, so that it is not possible to confirm that freezing has not occurred, which may make the user feel uneasy.

[0007] The present invention has been made in consideration of the above-mentioned situation, and its purpose is to provide a display control device that allows a user to recognize that no malfunction such as freezing has occurred, and that, if a malfunction does occur, allows the user to correctly recognize the malfunction. [Means for solving the problem]

[0008] In order to achieve the above object, a display control device according to the present invention is characterized by the following (1). (1) a main video generating unit that generates a display video based on a predetermined main video signal; a sub-picture generating unit for generating a sub-picture whose content changes periodically; a video synthesis unit that outputs a result of synthesizing the display video and the sub-video as an output video to a predetermined display device, an update cycle of the sub-image in the output image is within a range of cycles in which a user can visually recognize a change in the content of the sub-image; Display control device. Effect of the Invention

[0009] According to the display control device of the present invention, it is possible to allow a user to recognize that no malfunction such as freezing has occurred, and, if a malfunction does occur, it is possible for the user to correctly recognize the malfunction.

[0010] The present invention has been briefly described above. The details of the present invention will be further clarified by reading the best mode for carrying out the invention described below with reference to the accompanying drawings. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of an in-vehicle display system including a display control device according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram showing the various components inside the video display device. [Diagram 3] FIG. 3 is a state transition diagram showing an example of transition of the display content of the confirmation additional image. [Figure 4] FIG. 4 is a front view showing an example of a screen display on the display device. [Diagram 5] FIG. 5 is a block diagram showing another configuration example of the in-vehicle display system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A specific embodiment of the present invention will be described below with reference to the accompanying drawings. <Configuration of display control device> FIG. 1 shows an example of the configuration of an in-vehicle display system 100 including a display control device according to an embodiment of the present invention.

[0013] 1 includes an image display device 10, an external image generating device 20, a display driver 21, and a display device 22. For example, the image display device 10 can be a mobile terminal such as a smartphone that the driver brings into the vehicle, or a predetermined electronic control unit (ECU) installed in the vehicle.

[0014] The external image generating device 20 can generate, for example, real-time images captured by an on-board camera mounted on the vehicle, images generated on the vehicle side, etc. as the external image SG1. The external image SG1 generated by the external image generating device 20 is input to the external image input 10a of the image display device 10.

[0015] The video display device 10 can output to the video output 10b an internal video generated therein and a composite video SG5 generated based on an external video SG1 input to an external video input 10a. The composite video SG5 is input to a display device 22 via a display driver 21.

[0016] For example, an in-vehicle display disposed near the instrument panel of the vehicle or a display provided on a mobile terminal can be used as the display device 22. The display driver 21 functions as an interface required for connecting the video display device 10 and the display device 22.

[0017] The video display device 10 shown in FIG. 1 includes, as main components, a video display application 11, a fault recognition display application 12, an internal video storage area 13, and a video synthesis unit 14. The internal image storage area 13 holds data of internal images prepared in advance in a predetermined storage device.

[0018] The video display application 11 is application software that is pre-installed on the video display device 10. The video display application 11 has a function of outputting an internal video generated based on internal video data SG2 input from the internal video storage area 13 and an external video SG1 according to a situation input to the external video input 10a as a main display video SG3.

[0019] The fault recognition display application 12 is application software for making a user recognize the occurrence of a fault such as freezing when the fault occurs. The fault recognition display application 12 is also pre-installed on the video display device 10. This fault recognition display application 12 has a function of constantly generating and outputting an additional image SG4 whose display content is updated at a constant cycle.

[0020] The image synthesis unit 14 has a function of synthesizing the main display image SG3 output by the image display application 11 and the additional image SG4 output by the failure recognition display application 12 on the same image frame and outputting the result as a synthetic image SG5. The function of the image synthesis unit 14 may be realized by application software or may be realized by electronic components such as an image synthesis IC (Integrated Circuit).

[0021] In the video display device 10, a temporary failure such as freezing may occur due to abnormal heat generation, resource shortage, etc. This failure is expected to occur throughout the entire area of ​​the predicted failure range 15 shown in FIG.

[0022] When the video display device 10 is operating normally, the content of the additional image SG4 is periodically updated, and the updated content is reflected in the composite video SG5. In this case, a user such as a driver can visually recognize the periodic change in the content of the additional image SG4 included in the composite video SG5 displayed on the screen of the display device 22, and can therefore recognize that the system is operating normally. This provides the user with a sense of security.

[0023] On the other hand, if a fault such as freezing occurs on the video display device 10, functions within the predicted fault range 15 stop, and the content of the additional image SG4 in the composite video SG5 also stops changing. Therefore, a user such as a driver can visually confirm that the content of the additional image SG4 included in the composite video SG5 displayed on the screen of the display device 22 does not change, and can recognize that an abnormality has occurred in the operation of the system.

[0024] <Example of internal configuration of video display device> FIG. 2 shows an example of the internal configuration of the video display device 10 in FIG.

[0025] For example, electronic devices such as mobile terminals like smartphones and various in-vehicle ECUs often include various hardware resources, operating systems (OS), and various application software, like the video display device 10 shown in FIG.

[0026] 2 includes electronic components and circuits such as a CPU (Central Processing Unit), memory, and timers as hardware resources HW. Various application software programs run on the operating system SW, and can easily secure the resources required to execute each operation as needed.

[0027] 2 includes, as various application software, a standard application AP1 and an additional application AP2. The standard application AP1 and the additional application AP2 correspond to the image display application 11 and the fault recognition display application 12 shown in FIG.

[0028] Therefore, in the video display device 10 shown in Fig. 1, the standard application AP1 and the additional application AP2 in Fig. 2 at least partially share the same basic software SW and hardware resources HW. Furthermore, the respective functions of the standard application AP1 and the additional application AP2 in Fig. 2 are executed substantially simultaneously.

[0029] Here, when the standard application AP1 is being processed, if the processing speed drops due to abnormal heat generation in the CPU or if a resource shortage occurs, the standard application AP1 may freeze. In addition, since they share the same basic software SW and hardware resources HW, if the standard application AP1 freezes, the effect spreads to the additional application AP2, and the additional application AP2 also freezes at the same time.

[0030] 1, when the operation of the video display application 11 freezes, the operation of the failure recognition display application 12 also freezes at the same time. As a result, the periodic update of the additional image SG4 stops, and the additional image SG4 displayed in the composite video SG5 displayed on the display device 22 does not change.

[0031] That is, a state in which the content of the additional image SG4 in the composite video SG5 displayed on the screen of the display device 22 changes indicates that the operation of the expected failure range 15 of the video display device 10 is normal. In addition, a state in which the change of the additional image SG4 in the composite video SG5 has stopped indicates that a failure has occurred in the expected failure range 15 of the video display device 10. Therefore, a user such as a driver can easily visually grasp the presence or absence of an abnormality in the expected failure range 15 of the video display device 10 from the presence or absence of a change in the additional image SG4 displayed on the screen of the display device 22. Furthermore, when a specific condition is satisfied, a state in which the content of the main display video SG3 in the composite image SG5 does not change and the confirmation additional image 32 changes indicates that a failure has occurred in the external video generation device 20. The specific condition is that the main display video SG3 is the external video SG1 input from the external video generation device 20, and the user can tell that the external video SG1 is moving. For example, if the external image SG1 is a real-time image of the outside of the vehicle captured by an on-board camera mounted on a moving vehicle, the user can see that the external image SG1 is moving. Therefore, according to the on-board display system 100, when the above-mentioned specific conditions are met, the user can easily visually grasp the presence or absence of an abnormality in the external image generation device 20 from the presence or absence of changes in the main display image 31 and the additional confirmation image 32 displayed on the screen of the display device 22. In addition, according to the on-board display system 100, when a malfunction of the external image generation device 20 is detected, the image display device 10 can notify an error on the screen of the display device 22 or leave an error log.

[0032] <Display transition of additional image for confirmation> An example of the transition of the display content of the additional confirmation image is shown in Figure 3. In the example shown in FIG. 3, it is assumed that a moving image combining four types of confirmation icon patterns P11, P12, P13, and P14 is used as the additional image SG4 shown in FIG.

[0033] In addition, in the example of Figure 3, it is assumed that four types of confirmation icon patterns P11, P12, P13, and P14 are used, and the content of the additional image SG4 is switched periodically in the order of P11, P12, P13, P14, P11, P12, ...

[0034] The confirmation icon patterns P11, P12, P13, and P14 in FIG. 3 each represent an arc-shaped or circumferential arrow pattern that represents angle ranges of approximately 90 degrees, 180 degrees, 270 degrees, and 360 degrees, respectively.

[0035] Therefore, when the confirmation icon patterns P11 to P14 are switched in a regular cycle as shown in Fig. 3, the state where the device is operating normally can be expressed as an animation that looks like a rotating arc-shaped arrow. Also, in the frozen state, this animation stops naturally without requiring any special control, so that the user can be sure that a malfunction has occurred.

[0036] <Example of screen display on a display device> An example of a screen display on the display device is shown in FIG. 4, a main display video 31 and a confirmation additional image 32 are simultaneously displayed on the display screen of the display device 22. The main display video 31 corresponds to an external video SG1 input from the external video generating device 20 to the video display equipment 10, or a video generated by the video display application 11 based on the internal video data SG2.

[0037] The confirmation additional image 32 corresponds to the additional image SG4 output by the fault recognition display application 12 in the video display device 10. During normal operation, the fault recognition display application 12 repeatedly updates the content of the additional image SG4 at a fixed cycle (e.g., every 0.5 seconds) like the confirmation icon patterns P11 to P14 shown in Fig. 3.

[0038] Therefore, the content of the confirmation additional image 32 displayed on the screen of the display device 22 in Fig. 4 is usually updated periodically. On the other hand, if the operation of the video display application 11 freezes due to, for example, abnormal heat generation or resource shortage of the video display device 10, the operation of the fault recognition display application 12 also freezes at the same time, and the update of the content of the additional image SG4 stops.

[0039] 4, when periodic changes occur in the content of the confirmation additional image 32, there is no abnormality in the operation of the system that displays the main display video 31. On the other hand, when there is no change in the content of the confirmation additional image 32, it is highly likely that the operation of the video display application 11 is frozen, which means that there is a problem with the display content of the main display video 31.

[0040] In other words, a user such as a driver can determine whether or not there is an abnormality in the system operation caused by freezing, by visually checking whether or not there is a change in the content of the additional confirmation image 32 that is constantly displayed on the screen of the display device 22, without having to check the content of the main display video SG3.

[0041] 4, the confirmation additional image 32 is constantly displayed in a display area P0 at a predetermined position on the screen, and the content of this confirmation additional image 32 is updated at regular intervals. Therefore, the user can easily visually recognize the confirmation additional image 32 and whether or not there is a change in the image.

[0042] <Modification of the display form of the additional image SG4> Regarding changes in the display form of the additional image SG4 in the composite video SG5 displayed on the screen of the display device 22, various modifications such as those shown below can be assumed. (1) The additional image SG4 is displayed as a moving image so as to express changes as any or all of the following combinations: size, shape, movement, color, display technology such as 3D and shading, letters, numbers, symbols, and figures. However, since it is necessary for the user to easily visually confirm the changes in this moving image, the additional image SG4 must be placed in a position where it is easily visible. In addition, the time period for updating the content of the additional image SG4 is set to, for example, within a range of about 0.1 to 2 seconds.

[0043] (2) The content of the additional image SG4 may be changed as a moving image whose display position moves periodically in any axis direction on a three-dimensional coordinate system that the user can recognize on the display screen. However, it is preferable to limit the range in which the additional image SG4 can move and the speed at which the additional image SG4 moves so as not to interfere with the user's visibility.

[0044] (3) The pattern shape of the additional image SG4 may be a circle or a polygon with more than one side, such as a triangle. A moving image in which a pattern of this shape rotates may be adopted as the additional image SG4. The additional image SG4 may be expressed as a moving image in which the color changes in a certain region. Furthermore, the additional image SG4 may be expressed as a moving image in which letters and numbers change according to a certain rule in a certain region.

[0045] <Advantages of the in-vehicle display system 100> For example, a driver of a vehicle may want to check obstacles or pedestrians in blind spots such as the rear of the vehicle using images captured by an on-board camera. In the on-board display system 100 shown in FIG. 1, an image captured by the on-board camera can be input from the external image generating device 20 to the image display device 10 as an external image SG1 and displayed on the screen of the display device 22. Usually, when the image captured by the on-board camera does not change much, the user may feel uneasy as to whether the actual situation outside the vehicle is being displayed correctly or whether a malfunction of the device has occurred. In contrast, according to the on-board display system 100, when freezing does not occur, the additional image SG4 changes periodically even if there is no change in the main display image 31, so that the user can recognize that freezing is not occurring and can feel at ease.

[0046] On the other hand, in the video display device 10 shown in Fig. 1, when the operation of the video display application 11 freezes, the operation of the failure recognition display application 12 also freezes at the same time due to the system structure, and the change of the additional image SG4 stops. As a result, the driver can recognize that the change of the confirmation additional image 32 has stopped on the screen of the display device 22 shown in Fig. 4, for example. Therefore, the driver can understand that the operation of the system is abnormal, and can check obstacles and pedestrians around the vehicle more carefully, and can drive safely.

[0047] 1, there is no need to separate the function of the fault recognition display application 12, which has a function of making the driver recognize the presence or absence of a fault, from the hardware and basic software resources of the video display application 11. Therefore, there is no need to complicate the circuit configuration of the video display application 10 or increase the number of components in order to enable fault recognition, and an increase in the cost of the device can be suppressed.

[0048] <Examples of external video> As described above, the external image generating device 20 can generate real-time images captured by an on-board camera mounted on the vehicle as the external image SG1. Specific examples of this external image SG1 include images of the surroundings of the vehicle captured by one or more on-board cameras, images including at least the front of the vehicle captured by an on-board camera for a drive recorder, and images captured by an on-board camera for a driver monitor that captures the driver, in addition to the images of the rear of the vehicle captured by an on-board camera for a backup monitor.

[0049] <Modification of the in-vehicle display system> In the in-vehicle display system 100, as shown in FIG. 5, vehicle information VI can be input to the external information input 10c of the video display device 10, and the video display application 11 may generate a main display image SG3 based on the vehicle information VI. Specific examples of the vehicle information VI include information related to the vehicle, such as vehicle speed and battery, information related to the settings of in-vehicle devices such as an air conditioner, a seat, or a HUD (Head Up Display), and information on a loading platform connected to the vehicle, such as the actual temperature of a refrigerated truck. The video display application 11 generates a main display image SG3 by superimposing any of the vehicle information VI on the internal video data SG2 or the external video SG1. The video display device 10 outputs the result of combining the main display image SG3 and the additional image SG4 as a composite image SG5. Therefore, according to the in-vehicle display system 100 shown in FIG. 5, even if there is no change in the main display image 31 including the display of the vehicle information VI, the confirmation additional image 32 changes periodically, so that the user can recognize that freezing has not occurred. Therefore, the user can recognize that the displayed vehicle information VI is correct, and can feel at ease.

[0050] <Supplementary explanation> Here, the features of the above-described embodiment of the display control device according to the present invention will be briefly summarized and listed in the following [1] to [5]. [1] A main video generation unit (video display application 11) that generates a display video (main display video SG3) based on a predetermined main video signal (external video SG1, etc.); a sub-image generating unit (fault recognition display application 12) for generating a sub-image (additional image SG4) whose contents change periodically; a video synthesis unit (14) that outputs a result of synthesizing the display video and the sub-video as an output video (synthesized video SG5) to a predetermined display device (display device 22); an update cycle of the sub-image in the output image is within a range of cycles in which a user can visually recognize a change in the content of the sub-image; A display control device (video display device 10).

[0051] According to the display control device having the configuration of [1] above, the content of the sub-picture generated by the sub-picture generating unit changes periodically, so that in a normal state where no fault occurs, the user can always recognize the change in the sub-picture in the output video, which gives a sense of security. Also, when a fault such as freezing occurs, the change in the sub-picture in the output video stops, so the user can visually recognize that there is an abnormality from the content of the output video. Therefore, according to this display control device, even when a camera video or the like in which it is difficult to tell whether the main video signal has stopped is displayed, the user can easily know whether a fault such as freezing has occurred in the system by visually checking whether the sub-picture has changed.

[0052] [2] The main video generation unit and the sub-video generation unit are arranged on a common electronic device and are each configured as application software that utilizes a common resource as at least a part of the hardware (hardware resource HW) or a part of the operating system (SW) in the electronic device. A display control device according to [1] above.

[0053] According to the display control device having the configuration of [2] above, since the main video generating unit and the sub-video generating unit are applications that use common resources, if the operation of the main video generating unit freezes, the effect naturally spreads to cause the operation of the sub-video generating unit to freeze as well. Therefore, there is no need to add a circuit with a complex configuration to detect a failure in the main video generating unit, and there is no need to have separate hardware for each of the main video generating unit and the sub-video generating unit, so an increase in device costs can be suppressed.

[0054] [3] The sub-image generating unit generates, as the sub-image, a moving image (e.g., confirmation icon patterns P11 to P14) whose content is updated at regular time intervals. A display control device according to [1] above.

[0055] According to the display control device having the configuration described in [3] above, the time interval at which the moving image changes is constant, so that the user can easily visually check whether the displayed moving image is changing or not. In addition, it is also easy to distinguish between the main image and the sub-image.

[0056] [4] The sub-picture generation unit controls the display position of the sub-picture so that the display position (display area P0) of the sub-picture is constant within a display frame of the output video. A display control device according to [1] above.

[0057] According to the display control device having the configuration in [4] above, the sub-picture is always displayed in approximately the same position, so that the user can easily grasp the display position of the sub-picture within the display frame. This improves the visibility of the displayed sub-picture.

[0058] [5] The main video generation unit accepts video input from an external in-vehicle device, and when a video signal is input from the in-vehicle device, generates the display video using the video signal as the main video signal. A display control device according to any one of [1] to [4] above.

[0059] According to the display control device having the configuration described above in [5], a display system can be easily configured in which a mobile terminal such as a smartphone and an in-vehicle device cooperate to display information such as images. This improves the convenience of users who use the vehicle. [Explanation of symbols]

[0060] 10. Video display equipment 10a External video input 10b Video output 11 Video display app 12 Fault recognition display app 13 Internal video storage area 14 Video synthesis section 15 Expected failure range 20 External image generating device 21 Display Driver 22 Display device 31 Main display image 32 Additional image for confirmation 100 In-vehicle display system AP1 standard app AP2 Additional Apps HW Hardware resources P0 display area P11, P12, P13, P14 Confirmation Icon Pattern SG1 External video SG2 internal video data SG3 Main display image SG4 Additional Images SG5 Composite Image SW Basic software

Claims

1. a main video generating unit that generates a display video based on a predetermined main video signal; a sub-picture generating unit for generating a sub-picture whose content changes periodically; an image synthesis unit that outputs a result of synthesizing the display image and the sub-image as an output image to a predetermined display device; and wherein at least an update cycle of the sub-image in the output image is within a range of a cycle in which a user can visually recognize a change in the content of the sub-image. Display control device.

2. the main video generation unit and the sub-video generation unit are arranged on a common electronic device and are each configured as application software that utilizes a common resource as at least a part of the hardware or basic software of the electronic device. The display control device according to claim 1 .

3. the sub-image generating unit generates, as the sub-image, a moving image whose content is updated at regular time intervals; The display control device according to claim 1 .

4. the sub-image generating unit controls a display position of the sub-image so that the display position of the sub-image is constant within a display frame of the output image. The display control device according to claim 1 .

5. the main video generation unit allows video input from an external in-vehicle device, and when a video signal is input from the in-vehicle device, generates the display video using the video signal as the main video signal. The display control device according to claim 1 .

Citation Information

Patent Citations

  • Video camera

    JP2006352183A

  • Detecting image freezing in video display

    JP2022081438A

  • Method for testing portable electronic device

    JP2022140321A

  • Vehicle Camera System Having Live Video Indication

    US20160039341A1

  • Display control device of display and display video freeze notification method

    JP2018121238A