Display system

Abstract

To provide a display system that can accurately estimate a visible region in a display, and can execute brightness control of achieving both reduction of discomfort for a user in the visible region and reduction of power consumption.SOLUTION: A display system, when acquiring a user's neck direction angle and light-of-sight angle, calculates a first predetermined region R2 in a display surface 1a of a display 1 on the basis of the neck direction angle and a second predetermined region R3 on the basis of the line-of-sight angle. When one of the user's neck direction angle and the line-of-sight angle changes later than the other, the display system estimates a region obtained by adding a protruding region R31 included in the second predetermined region R3 and protruding from the first predetermined region R2 to the first predetermined region R2, as a visible region R1. The display system executes brightness control of reducing the brightness of pixel groups located in external regions R0 that are other than the visible region R1 compared to the brightness of pixel groups located in the visible region R1.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to a display system that acquires information including the orientation of the user's face and performs brightness control of the display. [Background technology]

[0002] Conventionally, a display system that can reduce power consumption by lowering the brightness of the display when the user is not looking at it is described in Patent Document 1, for example. The display system described in Patent Document 1 captures the user of the display, acquires user information such as the user's gaze and face orientation, and if it is determined based on this user information that the user is not looking at the display, it performs control to lower the brightness of the display. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Patent No. 5098438 [Overview of the project] [Problems that the invention aims to solve]

[0004] In recent years, displays have become larger, and when using large displays, brightness control is required to reduce power consumption even when the user is looking at the display. Specifically, when a user is looking at a large display, the display has both an area within the user's effective field of view and an area within their peripheral field of view. Therefore, there is room to reduce the brightness in the peripheral field of view and the area outside of it where the user cannot perceive details.

[0005] The display system described in Patent Document 1 can control brightness when the user is not looking at the display, but it cannot perform partial brightness control when the user is looking at the display. Therefore, it is conceivable to estimate the central field of view and the effective field of view based on the user's line of sight angle or the angle of their head and the position of the display, and then perform brightness reduction control in the area outside the effective field of view. For the sake of explanation, the angle of the user's line of sight will be referred to as the "line of sight angle," and the angle of the user's head will be referred to as the "head angle."

[0006] However, if the user's gaze cannot be detected, such as when the user is wearing sunglasses, or if there is a time lag between changes in the user's gaze angle and changes in head angle, the accuracy of estimating the area within the effective field of view will decrease. In such cases, if control is implemented to partially reduce brightness, the brightness of some areas within the user's actual effective field of view may decrease, potentially causing the user discomfort.

[0007] In view of the above, the present invention aims to provide a display system that can accurately estimate the viewing area, including the effective field of view, in a display, and that can perform brightness control that achieves both a reduction in user discomfort in the viewing area and a reduction in power consumption. [Means for solving the problem]

[0008] To achieve the above objective, the display system described in claim 1 is a display system that performs brightness control of a display (1), comprising: a head angle acquisition unit (21) that acquires information on the head angle (θ1), which is the angle of the user's face in the frontal direction relative to the display surface (1a) of the display, from a user imaging unit (31) that images the user of the display; a gaze angle acquisition unit (22) that acquires information on the gaze angle (θ2), which is the angle of the user's line of sight relative to the display surface, from the user imaging unit; a view area estimation unit (23) that estimates a part of the display surface that includes at least the user's effective field of view as a view area (R1) based on at least one of the gaze angle and the head angle; and a view area within the display The system includes a brightness control unit (24) that performs brightness control to make the brightness of pixel groups located outside the viewing area less than the brightness of pixel groups located in the viewing area. The viewing area estimation unit calculates a first predetermined area (R2) within a predetermined range on the display surface centered on the intersection of the display surface and a virtual line along the head angle, and a second predetermined area (R3) within a predetermined range centered on the intersection of the display surface and a virtual line along the viewing angle. The system estimates the viewing area as an added area obtained by adding an overhang area (R31) located outside the first predetermined area of ​​the second predetermined area to the first predetermined area.

[0009] This display system includes a viewing angle acquisition unit that acquires viewing angle information from a user imaging unit that images the user of the display, a head-turning angle acquisition unit that acquires head-turning angle information, and a viewing area estimation unit that estimates the viewing area of ​​the display that includes the user's effective field of view. If one of the acquired head-turning angle and viewing angle changes before the other, the viewing area estimation unit calculates a first predetermined area of ​​the display centered on the intersection of the display surface and the direction along the head-turning angle. In this case, the viewing area estimation unit also calculates a second predetermined area of ​​the display centered on the intersection of the display surface and the direction along the viewing angle, and estimates the area obtained by adding the area of ​​the second predetermined area that extends beyond the first predetermined area and the first predetermined area as the viewing area. Furthermore, this display system has a brightness control unit, which performs brightness control to make the brightness of pixel groups located in areas of the display other than the viewing area smaller than the brightness of pixel groups located in the viewing area. Therefore, this display system can accurately estimate the viewing area to include the user's actual effective field of view, even if there is a discrepancy in the timing of changes between the user's gaze angle and head angle, and it can also reduce power consumption by lowering the brightness in areas outside the viewing area.

[0010] The reference numerals in parentheses attached to each component indicate an example of the correspondence between that component and the specific components described in the embodiments described later. [Brief explanation of the drawing]

[0011] [Figure 1] This is a block diagram showing a display system according to an embodiment. [Figure 2] This is an explanatory diagram regarding the viewing area on the display surface of a display. [Figure 3] This is an explanatory diagram regarding the user's head angle and a first predetermined area on the display surface of the display. [Figure 4] This is an explanatory diagram regarding the user's viewing angle and a second predetermined area on the display surface of the display. [Figure 5]It is an explanatory diagram of a first estimation method of a visible region when the user's line-of-sight angle and neck orientation angle with respect to the display surface of the display match. [Figure 6] It is an explanatory diagram of a second estimation method of a visible region when the user's line-of-sight angle changes prior to the neck orientation angle. [Figure 7] It is an explanatory diagram of a second estimation method of a visible region when the user's line-of-sight angle changes with a delay with respect to the neck orientation angle. [Figure 8] It is an explanatory diagram of a third estimation method of a visible region when the user is wearing sunglasses. [Figure 9] It is a flowchart showing an example of a processing operation in brightness control.

Mode for Carrying Out the Invention

[0012] Hereinafter, embodiments of the present invention will be described based on the drawings. In each of the following embodiments, parts that are identical or equivalent to each other will be described with the same reference numerals.

[0013] (Embodiment) A display system according to an embodiment will be described. In this embodiment, a case where the display system is a vehicle-mounted display system mounted on a vehicle such as an automobile will be described as a representative example. Of course, it can also be applied to other uses.

[0014] For example, in the case of in-vehicle use, as shown in FIG. 1, this display system is configured to include a display 1, a display control unit 2, an in-vehicle device 3 including a user imaging unit 31, and an in-vehicle LAN 4. For example, in this display system, the user imaging unit 31 images the user of the display 1, the display control unit 2 acquires user information including at least the neck orientation angle θ1 of the user to be described later, and estimates the visible region R1 of the user on the display surface 1a of the display 1. Then, in this display system, the display control unit 2 executes brightness control to make the brightness of the pixel group located in the region other than the estimated visible region R1 of the display 1 smaller than the brightness of the pixel group located in the visible region.

[0015] The display 1 is an arbitrary display capable of controlling the luminance for each pixel or for each predetermined area. For example, the display 1 may be a self-emitting display having self-emitting elements such as OLEDs as pixels. OLED is an abbreviation for Organic Light Emitting Diode. For example, the display 1 may be a liquid crystal display that uses a plurality of micrometer-sized micro LEDs or the like as a backlight and can control the luminance for each minute area by local dimming, that is, partial driving. The display 1 performs an image display corresponding to the video output signal from the display control unit 2, for example. As shown in FIG. 2, for example, the display surface 1a of the display 1 has a horizontally long rectangular shape when viewed from the front, but is not limited thereto, and may have other shapes such as a square, a vertically long rectangle, or a curved surface shape. The display 1 is, for example, a large display in which the display surface 1a has an area larger than the effective visual field of the user when viewed from a user at a predetermined distance. For example, in the case of in-vehicle use, the display 1 is mounted on the instrument panel and is a wide display extending from the end on the passenger seat side to the end on the driver's seat side, but is not limited thereto.

[0016] Note that since the configurations of OLED displays and liquid crystal displays are well-known, detailed descriptions of the displays themselves are omitted in this specification. In addition, although OLEDs are cited as representative examples of self-emitting elements, the present invention is not limited thereto, and other self-emitting elements such as inorganic ELs and micro LEDs may be used. That is, the display 1 may be a display body capable of controlling the luminance in units of individual pixels or in units of pixel groups occupying areas smaller than a predetermined size. The area smaller than the predetermined size includes, for example, but is not limited to, a 1 mm square size.

[0017] The display control unit 2 is composed of a microcomputer equipped with, for example, a CPU, ROM, RAM, and I / O. CPU, ROM, RAM, and I / O are abbreviations for Central Processing Unit, Read Only Memory, Random Access Memory, and Input / Output, respectively. The display control unit 2 performs video output and brightness control of the display image on the display 1 based on video signals and information transmitted from the in-vehicle equipment 3, for example, by reading and executing various programs recorded on a recording medium (not shown). The display control unit 2 has a configuration that includes, for example, a head angle acquisition unit 21, a line of sight angle acquisition unit 22, a viewing area estimation unit 23, a brightness control unit 24, and a video output unit 25.

[0018] The head angle acquisition unit 21 acquires information on the user's head angle θ1 from the user imaging unit 31. The acquired information on the head angle θ1 is output to, for example, the viewing area estimation unit 23 and used to estimate the user's viewing area R1 on the display surface 1a of the display 1. The viewing area R1 is, for example, as shown in Figure 2, a part of the display surface 1a of the display 1, and is the user's central field of view V C Effective field of view V around it E This refers to the region that contains within it. Note that Figure 2 does not show a cross-section, but the central field of view V C Effective field of view V around it E Hatching has been applied to these areas to make them easier to see.

[0019] The head direction angle θ1 is, for example, the angle of a virtual straight line VL1 along the direction of the user's face as obtained by imaging with the user imaging unit 31, as shown in Figure 3. The head direction angle θ1 is defined, for example, as the angle between the virtual straight line VL1 and the reference direction, with the reference direction being 0°, which is the direction of the face of a user seated in the vehicle's seat looking forward (straight ahead) as shown by the dashed line in Figure 3. The head direction angle θ1 is obtained, for example, for the bearing along the vehicle width direction and the altitude along the vertical direction of the vehicle.

[0020] The line of sight acquisition unit 22 acquires information on the user's line of sight angle θ2, as shown in Figure 4, from the user imaging unit 31. When the line of sight acquisition unit 22 acquires information on the line of sight angle θ2, the acquired information on the line of sight angle θ2 is output to, for example, the viewing area estimation unit 23 and used to estimate the user's viewing area R1 on the display surface 1a of the display 1.

[0021] The line of sight angle θ2 is, for example, the angle of a virtual straight line VL2 along the user's line of sight obtained by imaging with the user imaging unit 31, as shown in Figure 4. The line of sight angle θ2 is defined, for example, similar to the head angle θ1, with the direction shown by the dashed line in Figure 4, i.e., the line of sight when a user seated in the vehicle's seat is looking forward, as the reference direction being 0°, and the angle of sight angle θ2 is defined as the angle between the virtual straight line VL2 and the reference direction. The line of sight angle θ2 is obtained, for example, for the bearing along the vehicle width direction and the altitude along the vertical direction of the vehicle.

[0022] Furthermore, the definitions of the head angle θ1 and the line of sight angle θ2 are not limited to the examples described above, and may be appropriately changed, for example, to the angle between the virtual line VL1 or VL2 and the normal direction to the display surface 1a.

[0023] The viewing area estimation unit 23 acquires information on the user's head angle θ1 of the display 1, for example, and estimates the viewing area R1 using at least the head angle θ1. If the gaze angle acquisition unit 22 acquires information on the gaze angle θ2, the viewing area estimation unit 23 acquires this information from the gaze angle acquisition unit 22 and, if necessary, estimates the viewing area R1 based on the gaze angle θ2. Details of the estimation of the viewing area R1 will be described later. The data of the viewing area R1 estimated by the viewing area estimation unit 23 is output to the brightness control unit 24, for example, and used for brightness control to reduce power consumption and improve the visibility of the viewing area R1.

[0024] For the sake of simplicity, the area of ​​the display surface 1a that is different from the visible area R1 will be referred to as the "external area R0," the group of pixels located in the visible area R1 of the display surface 1a will be referred to as the "visible area pixel group," and the group of pixels located in the external area R0 will be referred to as the "external pixel group."

[0025] The luminance control unit 24, for example, adjusts the luminance of the viewable area pixel group and the external pixel group of the display 1 for the image data input from the in-vehicle device 3, and outputs the data to the video output unit 25. The luminance control unit 24 determines the viewable area pixel group and the external pixel group based on the viewable area R1 estimated by the viewable area estimation unit 23, for example. The luminance control unit 24 performs luminance control so that the luminance of at least the external pixel group is less than the luminance of the viewable area pixel group. The luminance control unit 24 generates image data corresponding to the corrected set luminance for each of the viewable area pixel group and the external pixel group, for example. Then, the luminance control unit 24 outputs the image data with the adjusted set luminance to the video output unit 25, for example.

[0026] The video output unit 25 generates a video output signal based on image data before or after brightness adjustment, which is input from the brightness control unit 24, and outputs it to the display 1.

[0027] The in-vehicle device 3 includes at least a user imaging unit 31 that images the user of the display 1. In addition to the user imaging unit 31, the in-vehicle device 3 may also include, for example, a navigation system, an audio system, a multimedia system, a vehicle ECU, communication equipment, and various sensors that output signals corresponding to physical quantities applied to the vehicle on which the display system is installed. ECU is an abbreviation for Electronic Control Unit. The in-vehicle device 3 is connected to the display control unit 2 via, for example, an in-vehicle LAN 40, and inputs various signals from the user imaging unit 31 and other sources to the display control unit 2.

[0028] The user imaging unit 31 is an imaging device having a camera that images the user of the display 1. In automotive applications, the user imaging unit 31 is, for example, a driver status monitor (registered trademark) manufactured by Denso Corporation, and it images the driver seated in the driver's seat and the passenger seated in the passenger seat of the vehicle. The user imaging unit 31 estimates the position coordinates of the user's eyes, head angle θ1, and line of sight angle θ2 within the vehicle from the image data including the captured user's face, based on known image recognition technology. The user imaging unit 31 estimates the position coordinates of the user's eyes, head angle θ1, and line of sight angle θ2 as needed and outputs the estimation results to the display control unit 2.

[0029] The user imaging unit 31 may have a camera or the like capable of capturing the user's face and calculating the head angle θ1 and line of sight angle θ2 using known image recognition technology, or it may be any other known imaging device. Furthermore, the user imaging unit 31 only needs to be able to capture the user's face, and its arrangement is arbitrary. For example, the user imaging unit 31 may calculate the position of each of the user's eyes or the center position of both eyes, and calculate the line of sight angle θ2 for the right eye and the left eye respectively, or it may calculate the average value of both eyes as a representative value of the line of sight angle θ2.

[0030] The in-vehicle LAN 4 is an in-vehicle communication bus that connects, for example, the display control unit 2 and the in-vehicle equipment 3, enabling communication between them. LAN stands for Local Area Network.

[0031] The above describes the basic configuration of this display system when applied to in-vehicle use.

[0032] [Estimation of the field of view] Next, the estimation of the visibility area R1 by the visibility area estimation unit 23 will be explained with reference to the drawings. Figures 6 to 8 do not show cross-sections, but hatching is applied to the space that is located outside the first predetermined area R2, which will be described later, and is estimated to be part of the visibility area R1.

[0033] First, we will explain the first estimation method for the visual field R1.

[0034] Typically, when a user changes the direction of their face, as shown in Figure 5, for example, their head direction and the direction of their line of sight, i.e., the head direction angle θ1 and the line of sight angle θ2, are approximately the same. "Approximately the same" includes not only cases where the head direction angle θ1 and the line of sight angle θ2 are perfectly the same, but also cases where these angles are slightly different. In this case, the viewing area estimation unit 23 estimates the viewing area R1 based on the head direction angle θ1. Specifically, for example, when the viewing area estimation unit 23 acquires information on the head direction angle θ1, as shown in Figure 3, it calculates the intersection point P1 between a virtual straight line VL1 along the direction of the head direction angle θ1 and the display surface 1a, and calculates a first predetermined area R2, which is a predetermined range centered on the intersection point P1. For example, the first predetermined area R2 is at least the user's effective field of view V E The area is calculated as a predetermined range of viewing angles, centered on intersection P1, with 20° in the vertical direction and 30° in the horizontal direction, so as to include the area. The viewing area estimation unit 23 estimates the first predetermined area R2 calculated as described above as the viewing area R1.

[0035] Furthermore, for example, when the viewing area estimation unit 23 acquires information on the line of sight angle θ2, as shown in Figure 4, it calculates the intersection point P2 between the virtual straight line VL2 along the direction of the line of sight angle θ2 and the display surface 1a, and calculates a second predetermined area R3 which is a predetermined range centered on the intersection point P2. For example, the second predetermined area R3 is similar to the first predetermined area R2 in that it is at least the user's effective field of view V E It is calculated as a predetermined viewing angle range centered on intersection P2, including the specified area. Note that the numerical values ​​of the predetermined ranges in the calculation of the first predetermined area R2 and the second predetermined area R3 are not limited to the above example and may be changed as appropriate depending on the distance between the user and display 1.

[0036] Next, we will explain the second estimation method for the visual field R1.

[0037] The viewing area estimation unit 23 estimates the viewing area R1 using a second estimation method different from the first estimation method described above, when the head angle θ1 and the line of sight angle θ2 deviate by more than a predetermined amount. Two situations in which the head angle θ1 and the line of sight angle θ2 deviate by more than a predetermined amount are when the line of sight angle θ2 changes ahead of the head angle θ1, and when it changes with a delay relative to the head angle θ1. In other words, the second estimation method is executed when there is a time lag between the change in the head angle θ1 and the change in the line of sight angle θ2.

[0038] A situation where the line of sight angle θ2 changes ahead of the head angle θ1 is, for example, as shown in Figure 6, when the user changes only their line of sight without changing the direction of their face. In this situation, if the first predetermined region R2 is estimated as the visible region R1 and control is performed to make the brightness of the external pixel group smaller than that of the visible region pixel group, a portion of the external pixel group will be located within the user's actual effective field of view, and its brightness will be reduced, which may cause the user to feel uncomfortable. For this reason, the visible region estimation unit 23 calculates the first predetermined region R2 and the second predetermined region R3, respectively, and also calculates the overhang region R31 that is located outside the first predetermined region R2 within the second predetermined region R3. Then, the visible region estimation unit 23 estimates the sum of the regions obtained by adding the overhang region R31 to the first predetermined region R2 as the visible region R1. In other words, the visible region estimation unit 23 estimates the visible region R1 as a wider region obtained by adding the region on the side of the user's line of sight ahead to the first predetermined region R2. As a result, the viewing area R1 is estimated to be a broad area that includes the region located within the user's actual central and effective field of view.

[0039] On the other hand, the case where the line of sight angle θ2 changes with a delay relative to the head angle θ1 is, for example, as shown in Figure 7, a situation where the user changes only the direction of their face without changing their gaze. In this situation, if the estimation of the viewing area R1 using the first estimation method and brightness control that reduces the brightness of the external pixel group are performed, the user may feel uncomfortable, as in the case described above. Even when the line of sight angle θ2 changes with a delay relative to the head angle θ1, the viewing area estimation unit 23 estimates the sum of the area obtained by adding the overhang area R31 to the first predetermined area R2 as the viewing area R1, in the same way as described above. In this case, the viewing area R1 is estimated as a wide area obtained by adding the overhang area R31, which is located at the position where the user's gaze remains directed, i.e., on the side where the gaze lags behind, to the first predetermined area R2, which is located where the user's face is turned. Therefore, the inclusion of the user's effective field of view in the external area R0, and consequently the luminance control that causes the user to feel uncomfortable, is suppressed.

[0040] When the line of sight angle θ2 changes ahead of or behind the head angle θ1, there is a difference in the timing of the changes in the head angle θ1 and the line of sight angle θ2. However, according to the second estimation method, a predetermined range including the user's effective field of view can be estimated as the viewing area R1. The second estimation method is executed, for example, until the difference between the head angle θ1 and the line of sight angle θ2 falls below a predetermined level, and when the head angle θ1 and the line of sight angle θ2 are approximately the same, it switches to the first estimation method. In other words, the second estimation method is executed from the time one of the head angle θ1 and the line of sight angle θ2 changes before the other until the head angle θ1 and the line of sight angle θ2 are approximately the same.

[0041] Next, we will explain the third estimation method for the visual field R1.

[0042] When the visual recognition area estimation unit 23 cannot obtain information on the user's line-of-sight angle θ2, it estimates the visual recognition area R1 by a third estimation method. Examples of cases where information on the user's line-of-sight angle θ2 cannot be obtained include a situation where the user is wearing sunglasses. Even in such a situation, since the entire face of the user is imaged, there is no particular influence on the calculation of the neck orientation angle θ1. In this case, the visual recognition area estimation unit 23 calculates a first predetermined area R2 and estimates the angular range of the line-of-sight angle θ2 based on the neck orientation angle θ1. For example, as shown in FIG. 8, the visual recognition area estimation unit 23 estimates the angular range R θ2 of the line-of-sight angle θ2 centered on the virtual straight line VL1 along the neck orientation angle θ1, and calculates the maximum range of the second predetermined area R3 within the estimated angular range R θ2 . The visual recognition area estimation unit 23 calculates an adjacent area R4 that is an area adjacent to the first predetermined area R2 and is a portion of the maximum range of the second predetermined area R3 that protrudes from the first predetermined area R2. Then, the visual recognition area estimation unit 23 estimates the added area obtained by adding the adjacent area R4 to the first predetermined area R2 as the visual recognition area R1. Thereby, even when information on the user's line-of-sight angle θ2 cannot be obtained, the visual recognition area estimation unit 23 can estimate a wide area including the user's effective visual field as the visual recognition area R1.

[0043] Regarding the angular range R θ2 , for example, a database of the line-of-sight angle θ2 corresponding to the neck orientation angle θ1 is created in advance, stored in a recording medium not shown, and can be obtained by reading the database when the neck orientation angle θ1 is acquired. For example, when the neck orientation angle θ1 is 5°, the numerical value corresponding to the neck orientation angle θ1 = 5° is read from the above database, and R θ2 is estimated as R θ2 = 5° ± 5°. The database of the line-of-sight angle θ2 is, for example, an angle within a predetermined range centered on the neck orientation angle θ1, and the range of θ2 is set for each of a plurality of sections corresponding to the numerical value of the neck orientation angle θ1.

[0044] The third estimation method is performed, for example, when information on the user's gaze angle θ2 cannot be obtained, and switches to the first or second estimation method once the information on the user's gaze angle θ2 is obtained.

[0045] [Brightness control] Next, an example of brightness control for display 1 in this display system will be described.

[0046] For example, when the display control unit 2 meets predetermined start conditions, such as the vehicle's ignition being turned on or the display 1 being turned on, it executes the control flow shown in Figure 9.

[0047] In step S110, for example, the display control unit 2 acquires user information from the user imaging unit 31, including at least the head-direction angle θ1 among the user's eye position coordinates, line-of-sight angle θ2, and head-direction angle θ1.

[0048] In the subsequent step S120, for example, the display control unit 2 determines whether the user information acquired in step S110 includes the viewing angle θ2. If the determination is positive, the process proceeds to step S140; otherwise, it proceeds to step S130.

[0049] In step S130, because the user's line of sight angle θ2 cannot be directly obtained due to factors such as the user wearing sunglasses, the viewing area estimation unit 23 estimates the viewing area R1 using the third estimation method described above. As a result, the area estimated as the viewing area R1 is the first predetermined area R2 plus the adjacent area R4 located around it.

[0050] In step S140, for example, the display control unit 2 determines whether there is a discrepancy in the changes between the head angle θ1 and the viewing angle θ2, that is, whether one of the head angle θ1 and the viewing angle θ2 changed before the other. For example, if the determination in step S140 is positive, the display control unit 2 proceeds to step S160, and if the determination is negative, it proceeds to step S150.

[0051] In step S150, since the difference between the head angle θ1 and the line of sight angle θ2 is less than or equal to a predetermined value, i.e., they are approximately the same, the viewing area estimation unit 23 estimates the viewing area R1 using the first estimation method described above. As a result, the first predetermined area R2 calculated based on the head angle θ1 is estimated as the viewing area R1.

[0052] In step S160, since the line of sight angle θ2 is either preceding or lagging behind the head angle θ1, the viewing area estimation unit 23 estimates the viewing area R1 using the second estimation method described above. As a result, a wide area is estimated as the viewing area R1, which is the first predetermined area R2 plus the overhang area R31 of the second predetermined area R3 that extends beyond the first predetermined area R2 on the user's line of sight side.

[0053] Then, for example, the display control unit 2 estimates the viewing area R1 in any one of steps S130, S150, or S160, and then proceeds to step S170. In step S170, for example, the brightness control unit 24 sets the viewing area pixel group and the external pixel group based on the viewing area R1 estimated in any one of steps S130, S150, or S160. Then, for example, the brightness control unit 24 performs brightness control so that the brightness of at least the external pixel group is less than the brightness of the viewing area pixel group.

[0054] This allows for reducing the brightness of the external region R0, which is outside the user's effective field of view and where the user cannot perceive image details, without reducing the brightness of the portion of the display image on display 1 that is in the user's line of sight. Therefore, even when one of the head angle θ1 and the gaze angle θ2 is delayed relative to the other, or when the gaze angle θ2 cannot be obtained, the visible region R1 can be estimated accurately, reducing the delay in brightness control and enabling the display of an image that does not cause discomfort to the user. In addition, by reducing the brightness in the user's peripheral field of view and areas further out, a low power consumption effect is achieved.

[0055] The display control unit 2 repeats the processes from step S110 onward until a predetermined termination condition is met, such as the display 1 turning off after the processing of step S170 is completed. As a result, the viewing area R1 is estimated using the first estimation method when the head angle θ1 and the viewing angle θ2 are approximately the same, the second estimation method when one of the head angle θ1 and the viewing angle θ2 is delayed relative to the other, and the third estimation method when the viewing angle θ2 cannot be obtained. Then, by performing brightness control based on the viewing area R1 obtained by the first to third estimation methods, the delay in brightness control can be suppressed, and both user discomfort and power consumption can be reduced.

[0056] In addition, during the brightness control in step S170, in order to make the portion of the image displayed in the viewing area R1 easier to see, control may be performed to increase the brightness of some or all of the pixels in the viewing area, in accordance with the brightness control of the external pixel group. For example, if the in-vehicle device 3 has an illuminance sensor and the illuminance of the environment in which the display 1 is placed is above a predetermined level, this display system may perform control to increase the brightness of the portion in the area exposed to sunlight.

[0057] According to this embodiment, when the user's viewing angle θ2 cannot be obtained, and when one of the user's head angle θ1 and viewing angle θ2 is delayed relative to the other, a wider area than the first predetermined area R2 obtained based on the head angle θ1 is estimated as the viewing area R1. Then, this display system performs brightness control on the external pixel group to make the brightness smaller than that of the viewing area pixel group based on the estimated viewing area R1. This suppresses delays in the estimation of the viewing area R1 and, consequently, in the brightness control, while achieving the effect of low power consumption. Furthermore, the estimation accuracy of the viewing area R1 is improved, and since it is set as a wide area that includes the user's effective field of view, it becomes possible to display images that do not cause discomfort to the user.

[0058] (Other embodiments) This disclosure is described in accordance with the embodiments, but it is understood that this disclosure is not limited to such embodiments or structures. This disclosure also includes various modifications and variations within the equivalence range. In addition, various combinations and forms, as well as other combinations and forms including one, more, or less of those elements, fall within the scope and concept of this disclosure.

[0059] The display control unit 2 and its method described in this disclosure may be implemented by a dedicated computer provided by configuring a processor and memory programmed to perform one or more functions embodied by a computer program. Alternatively, the display control unit 2 and its method described in this disclosure may be implemented by a dedicated computer provided by configuring a processor by one or more dedicated hardware logic circuits. Alternatively, the display control unit 2 and its method described in this disclosure may be implemented by one or more dedicated computers configured by a combination of a processor and memory programmed to perform one or more functions and a processor configured by one or more hardware logic circuits. Furthermore, the computer program may be stored as instructions executed by the computer on a computer-readable non-transitional tangible recording medium.

[0060] It goes without saying that, in each of the above embodiments, the elements constituting the embodiment are not necessarily essential unless explicitly stated to be particularly essential or unless they are clearly considered essential in principle. Furthermore, in each of the above embodiments, when numerical values ​​such as the number, numerical values, quantities, or ranges of the components of the embodiment are mentioned, the embodiment is not limited to those specific numbers unless explicitly stated to be particularly essential or unless it is clearly limited to a specific number in principle. Furthermore, in each of the above embodiments, when the shape, positional relationship, etc., of the components are mentioned, the embodiment is not limited to those shapes, positional relationships, etc., unless explicitly stated or unless it is clearly limited to a specific shape, positional relationship, etc., in principle. [Explanation of Symbols]

[0061] 1…Display, 1a…Display surface, 21…Neck angle acquisition unit, 22…Viewing angle acquisition unit 23...Visibility area estimation unit, 24...Brightness control unit, θ1...Head angle, θ2...Line of sight angle R1: Visibility area, R2: First designated area, R3: Second designated area, R31: Overflow area R4...Adjacent area

Claims

1. A display system that performs brightness control of a display (1), A head-direction angle acquisition unit (21) acquires information on the head-direction angle (θ1), which is the angle of the user's face in the frontal direction relative to the display surface (1a), from the user imaging unit (31) that images the user of the display, A line of sight angle acquisition unit (22) acquires information on the line of sight angle (θ2), which is the angle of the user's line of sight relative to the display surface, from the user imaging unit, A viewing area estimation unit (23) estimates a portion of the display surface that includes at least the user's effective field of view as the viewing area (R1) based on at least one of the line of sight angle and the head angle, The display includes a brightness control unit (24) that performs brightness control to make the brightness of a group of pixels located in an area other than the viewing area of ​​the display less than the brightness of a group of pixels located in the viewing area. The aforementioned viewing area estimation unit, A display system in which, when information on both the head angle and the line of sight is acquired, and if one of the head angle and the line of sight changes before the other, a predetermined area within the display surface centered on the intersection of the display surface and a virtual line along the head angle is calculated as a first predetermined area (R2), a predetermined area centered on the intersection of the display surface and a virtual line along the line of sight is calculated as a second predetermined area (R3), and an overhang area (R31) located outside the first predetermined area within the second predetermined area is added to the first predetermined area to estimate the viewable area as an added area.

2. The aforementioned viewing area estimation unit, From the time when the line of sight angle changes before the head angle, until the difference between the head angle and the line of sight angle falls below a predetermined value, the summation region is estimated as the viewing region. The display system according to claim 1, wherein if the difference between the head angle and the line of sight angle is less than or equal to a predetermined value, the first predetermined area is estimated to be the viewing area.

3. The display system according to claim 1, wherein, if the viewing angle information is not obtained, the viewing area estimation unit calculates the first predetermined area and estimates the area obtained by adding the adjacent area (R4) adjacent to the first predetermined area to the first predetermined area as the viewing area.

4. The display system according to claim 3, wherein the viewing area estimation unit estimates an angle within a predetermined range centered on the head angle as the range of the line of sight angle, and estimates an area within the range of the line of sight angle that is located outside the first predetermined area among the second predetermined areas as the adjacent area.

5. The display system according to any one of claims 1 to 4, wherein the brightness control unit performs brightness control to make the brightness of some or all of the pixel group located in the viewing area higher than the brightness of the pixel group located in an area other than the viewing area.

Images