Display system

The display system accurately estimates the user's visible area using neck and viewing angle acquisition units to adjust brightness, addressing inaccuracies in existing systems and improving user comfort and power efficiency.

JP2024156501AActive Publication Date: 2024-11-06DENSO CORP
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
JP2023071015
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-24
Publication Date
2024-11-06
Estimated Expiration
2043-04-24

AI Technical Summary

Technical Problem

Existing display systems struggle to accurately estimate the user's visible area, particularly when the user is wearing sunglasses or when there is a time lag between the line of sight and head orientation, leading to inaccurate brightness control and user discomfort.

Method used

A display system that utilizes a neck orientation angle acquisition unit, viewing angle acquisition unit, and a visibility area estimation unit to estimate the visible area by calculating predetermined regions based on the user's head orientation and line of sight angles, even in cases of lag or obstruction, and adjusts brightness accordingly.

Benefits of technology

Accurately estimates the user's visible area, reducing power consumption and user discomfort by adjusting brightness only in areas outside the visible area, thus enhancing visibility and power efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024156501000001_ABST
    Figure 2024156501000001_ABST
Patent Text Reader

Abstract

To provide a display system that can accurately estimate a visible region in a display, and can execute brightness control of achieving both reduction of discomfort for a user in the visible region and reduction of power consumption.SOLUTION: A display system, when acquiring a user's neck direction angle and light-of-sight angle, calculates a first predetermined region R2 in a display surface 1a of a display 1 on the basis of the neck direction angle and a second predetermined region R3 on the basis of the line-of-sight angle. When one of the user's neck direction angle and the line-of-sight angle changes later than the other, the display system estimates a region obtained by adding a protruding region R31 included in the second predetermined region R3 and protruding from the first predetermined region R2 to the first predetermined region R2, as a visible region R1. The display system executes brightness control of reducing the brightness of pixel groups located in external regions R0 that are other than the visible region R1 compared to the brightness of pixel groups located in the visible region R1.SELECTED DRAWING: Figure 6
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a display system that obtains information including the direction of a user's face and controls the brightness of the display. [Background technology]

[0002] Conventionally, an example of a display system capable of reducing power consumption by lowering the brightness of a display when a user is not looking at the display is that described in Patent Document 1. The display system described in Patent Document 1 captures an image of a user of the display, acquires user information that is information on the user's line of sight and facial direction, and executes control to lower the brightness of the display when it is determined based on the user information that the user is not looking at the display. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 5098438 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, displays have become larger, and when using a large display, brightness control is required to reduce power consumption even when the user is looking at the display. Specifically, when a user looks at a large display, the large display has an area within the user's effective visual field and an area in the peripheral visual field, so there is room to reduce the brightness of the peripheral visual field and the area outside of it where the user cannot perceive details.

[0005] The display system described in Patent Document 1 is capable of controlling brightness when the user is not looking at the display, but is unable to perform partial brightness control when the user is looking at the display. Therefore, it is conceivable to estimate the areas of the central visual field and the effective visual field based on the angle of the user's line of sight or the angle of the direction of the face and the position of the display, and to perform control to reduce the brightness of the area outside the effective visual field. Hereinafter, for convenience of explanation, the angle of the user's line of sight is referred to as the "line of sight angle" and the angle of the direction of the user's face is referred to as the "head direction angle".

[0006] However, when the user's line of sight cannot be detected, such as when the user is wearing sunglasses, or when there is a time lag between the change in the user's line of sight angle and the change in the head angle, the accuracy of estimating the area within the effective visual field decreases. In such cases, if control to partially reduce the brightness is executed, the brightness of some areas within the user's actual effective visual field will decrease, which may cause the user to feel uncomfortable.

[0007] In view of the above, the present invention aims to provide a display system that can accurately estimate the viewing area, including the effective field of view, on a display, and can perform brightness control that reduces both the discomfort felt by the user in the viewing area and power consumption. [Means for solving the problem]

[0008] In order to achieve the above object, the display system according to claim 1 is a display system that performs brightness control of a display (1), and includes a neck direction angle acquisition unit (21) that acquires information on a neck direction angle (θ1) that is an angle of the front direction of the user's face with respect to a display surface (1a) of the display from a user imaging unit (31) that images a user of the display, a gaze angle acquisition unit (22) that acquires information on a gaze angle (θ2) that is an angle of the user's gaze direction with respect to the display surface from the user imaging unit, a visual recognition area estimation unit (23) that estimates an area that is part of the display surface and includes at least the user's effective visual field as a visual recognition area (R1) based on at least one of the gaze angle and the neck direction angle, and and a luminance control unit (24) that executes luminance control to make the luminance of pixels located in areas other than the visible area smaller than the luminance of pixels located in the visible area, and when information on both the head direction angle and the gaze angle is acquired and one of the head direction angle and the gaze angle changes before the other, the visible area estimation unit calculates an area of ​​a predetermined range on the display surface centered on the intersection of the display surface with a virtual line along the head direction angle as a first predetermined area (R2), calculates an area of ​​a predetermined range centered on the intersection of the display surface with the virtual line of the gaze angle as a second predetermined area (R3), and estimates, as the visible area, an added area obtained by adding an overhanging area (R31) of the second predetermined area located outside the first predetermined area to the first predetermined area.

[0009] This display system includes a gaze angle acquisition unit that acquires gaze angle information from a user imaging unit that images a user of the display, a neck direction angle acquisition unit that acquires neck direction angle information, and a visual recognition area estimation unit that estimates a visual recognition area of ​​the display that includes the effective visual field of the user. When one of the acquired neck direction angle and gaze angle changes before the other, the visual recognition area estimation unit calculates a first predetermined area of ​​the display centered on an intersection point between the display surface and the destination in the direction along the neck direction angle. In this case, the visual recognition area estimation unit calculates a second predetermined area of ​​the display centered on an intersection point between the display surface and the destination in the direction along the gaze angle, and estimates an area obtained by adding an area of ​​the second predetermined area that protrudes from the first predetermined area to the first predetermined area as the visual recognition area. The display system further includes a luminance control unit that executes luminance control to reduce the luminance of a pixel group located in an area other than the visual recognition area of ​​the display by the luminance control unit, compared to the luminance of a pixel group located in the visual recognition area. As a result, even if there is a discrepancy in the timing of changes in the user's gaze angle and head angle, this display system can accurately estimate the visual field to include the user's actual effective field of view, and can reduce power consumption by reducing the brightness in areas outside the visual field.

[0010] The reference symbols in parentheses attached to each component indicate an example of the correspondence between the component and specific components described in the embodiments described below. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram showing a display system according to an embodiment; [Diagram 2] FIG. 2 is an explanatory diagram of a visible region on a display surface of a display. [Diagram 3] 4A to 4C are diagrams illustrating the user's head angle and a first predetermined area on the display surface of a display. [Figure 4] 4 is an explanatory diagram of a user's line of sight angle and a second predetermined area on the display surface of a display. FIG. [Diagram 5]10A and 10B are explanatory diagrams relating to a first method for estimating a visual recognition area when a user's line of sight angle and head direction angle with respect to a display surface of a display coincide with each other. [Figure 6] 11 is an explanatory diagram of a second estimation method for the visual recognition area when the user's line of sight angle changes prior to the head direction angle. FIG. [Figure 7] 11 is an explanatory diagram of a second estimation method for the visual recognition area when the user's line of sight angle changes with a delay relative to the head direction angle. FIG. [Figure 8] 13 is an explanatory diagram of a third method for estimating the visible recognition area when a user is wearing sunglasses. FIG. [Figure 9] 13 is a flowchart showing an example of a processing operation in brightness control. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following embodiments, parts that are the same or equivalent to each other will be denoted by the same reference numerals.

[0013] (Embodiment) A display system according to an embodiment will be described. In this embodiment, a display system that is a vehicular display system mounted on a vehicle such as an automobile will be described as a representative example, but the display system can of course be applied to other uses.

[0014] For example, in the case of in-vehicle use, the display system includes a display 1, a display control unit 2, an in-vehicle device 3 including a user imaging unit 31, and an in-vehicle LAN 4, as shown in Fig. 1. In the display system, for example, the user imaging unit 31 captures an image of a user of the display 1, and the display control unit 2 acquires user information including at least a head orientation angle θ1 (described later) of the user, and estimates a visual recognition area R1 of the user on the display surface 1a of the display 1. Then, in the display system, the display control unit 2 executes brightness control to reduce the luminance of a pixel group located in an area of ​​the display 1 other than the estimated visual recognition area R1 to be smaller than the luminance of a pixel group located in the visual recognition area.

[0015] The display 1 is, for example, any display capable of controlling the brightness for each pixel or each predetermined area. The display 1 may be, for example, a self-luminous display having self-luminous elements such as OLEDs as pixels. OLED is an abbreviation for Organic Light Emitting Diode. The display 1 may be, for example, a liquid crystal display using a plurality of micrometer-sized micro LEDs as backlights and capable of controlling the brightness for each minute area by local dimming, i.e., partial driving. The display 1 displays, for example, an image corresponding to a video output signal from the display control unit 2. As shown in FIG. 2, the display 1 has a horizontally long rectangular shape when the display surface 1a is viewed from the front, but is not limited thereto, and may have other shapes such as a square, a vertically long rectangular shape, or a curved shape. The display 1 is, for example, a large display in which the display surface 1a has an area larger than the effective visual field of a user who is located a predetermined distance away. For example, in the case of an in-vehicle application, the display 1 is mounted on an instrument panel and is a wide display extending from the end on the passenger seat side to the end on the driver seat side, but is not limited to this.

[0016] Since the configuration of an OLED display or a liquid crystal display is well known, a detailed description of the display itself will be omitted in this specification. Although an OLED is given as a representative example of a self-luminous element, the present invention is not limited to this, and other self-luminous elements such as inorganic EL and micro LED may be used. In other words, the display 1 may be a display body capable of controlling the brightness of each pixel or a pixel group occupying an area of ​​a predetermined size or less. The area of ​​a predetermined size or less may be, for example, a 1 mm square size, but is not limited thereto.

[0017] The display control unit 2 is configured by, for example, a microcomputer equipped with a CPU, a ROM, a RAM, an I / O, and the like. Note that CPU, ROM, RAM, and I / O are abbreviations for Central Processing Unit, Read Only Memory, Random Access Memory, and Input / Output, respectively. The display control unit 2 performs video output and brightness control of the display image on the display 1 based on the video signal and information transmitted from the in-vehicle device 3, for example, by reading and executing various programs recorded on a recording medium (not shown). The display control unit 2 is configured to include, for example, a head direction angle acquisition unit 21, a line of sight angle acquisition unit 22, a visual recognition area estimation unit 23, a brightness control unit 24, and a video output unit 25.

[0018] The head direction angle acquisition unit 21 acquires information on the user's head direction angle θ1 from the user imaging unit 31. The acquired information on the head direction angle θ1 is output to, for example, a visual recognition area estimation unit 23, and is used to estimate the user's visual recognition area R1 on the display surface 1a of the display 1. The visual recognition area R1 is, for example, a part of the display surface 1a of the display 1, as shown in FIG. 2, and is a part of the user's central visual field V C and its surrounding effective visual field V E In addition, although Figure 2 does not show a cross section, the central visual field V C and its surrounding effective visual field V E These areas are hatched for ease of viewing.

[0019] The head direction angle θ1 is, for example, as shown in Fig. 3, the angle of a virtual straight line VL1 along the front direction of the user's face obtained by imaging with the user imaging unit 31. The head direction angle θ1 is defined as the angle between the virtual straight line VL1 and a reference direction, for example, the direction shown by the dashed line in Fig. 3, that is, the front direction of the face when the user seated in the seat of the vehicle is looking ahead (in front) of the vehicle, being taken as 0°. The head direction angle θ1 is obtained, for example, for the direction along the vehicle width direction and the altitude along the up-down direction of the vehicle.

[0020] 4 from the user imaging unit 31. When the gaze angle acquisition unit 22 acquires information on the gaze angle θ2, the acquired information on the gaze angle θ2 is output to, for example, the visual recognition area estimation unit 23 and used to estimate the user's visual recognition area R1 on the display surface 1a of the display 1.

[0021] The line of sight angle θ2 is, for example, the angle of a virtual line VL2 along the line of sight of the user obtained by imaging with the user imaging unit 31, as shown in Fig. 4. The line of sight angle θ2 is defined, for example, as the head angle θ1, as an angle between the virtual line VL2 and the reference direction, with the reference direction being the direction shown by the dashed line in Fig. 4, that is, the line of sight direction when the user seated in the seat of the vehicle is looking ahead of the vehicle, being set as 0°. The line of sight angle θ2 is obtained, for example, for the direction along the vehicle width direction and the altitude along the vertical direction of the vehicle.

[0022] The definitions of the head angle θ1 and the line of sight angle θ2 are not limited to the above examples, and may be changed as appropriate, such as the angle between the virtual straight line VL1 or VL2 and the normal direction to the display surface 1a.

[0023] The visual recognition area estimation unit 23, for example, acquires information on at least the neck direction angle θ1 of the user of the display 1, and estimates the visual recognition area R1 using at least the neck direction angle θ1. For example, when the gaze angle acquisition unit 22 acquires information on the gaze angle θ2, the visual recognition area estimation unit 23 acquires the information from the gaze angle acquisition unit 22, and estimates the visual recognition area R1 based on the gaze angle θ2 as necessary. Details of the estimation of the visual recognition area R1 will be described later. The data on the visual recognition area R1 estimated by the visual recognition area estimation unit 23 is output to, for example, the brightness control unit 24, and is used for brightness control to reduce power consumption and improve visibility of the visual recognition area R1.

[0024] For ease of explanation, in the following, the external area of ​​the display surface 1a that is different from the visible area R1 will be referred to as the "external area R0", the group of pixels located in the visible area R1 of the display surface 1a will be referred to as the "visible area pixel group", and the group of pixels located in the external area R0 will be referred to as the "external pixel group".

[0025] The luminance control unit 24 adjusts the luminance of each of the visible region pixel group and the external pixel group of the display 1 for image data input from the in-vehicle device 3, for example, and outputs the data to the video output unit 25. The luminance control unit 24 determines the visible region pixel group and the external pixel group based on, for example, the visible region R1 estimated by the visible region estimation unit 23. The luminance control unit 24 executes luminance control to make at least the luminance of the external pixel group smaller than the luminance of the visible region pixel group. The luminance control unit 24 generates, for example, image data corresponding to the corrected set luminance for each of the visible region pixel group and the external pixel group. Then, the luminance control unit 24 outputs, for example, the image data with the adjusted set luminance to the video output unit 25.

[0026] The video output unit 25 generates a video output signal based on image data before or after brightness adjustment input from the brightness control unit 24, and outputs the signal to the display 1, for example.

[0027] The in-vehicle device 3 includes at least a user imaging unit 31 that captures an image of a user on the display 1. In addition to the user imaging unit 31, the in-vehicle device 3 may include, for example, a navigation device, an audio device, a multimedia device, a vehicle ECU, a communication device, and various sensors that output signals corresponding to physical quantities applied to the vehicle in which the display system is mounted. ECU is an abbreviation for Electronic Control Unit. The in-vehicle device 3 is connected to the display control unit 2 via, for example, an in-vehicle LAN 40, and inputs various signals from the user imaging unit 31 and the like to the display control unit 2.

[0028] The user imaging unit 31 is an imaging device having a camera that captures an image of a user of the display 1. For in-vehicle use, the user imaging unit 31 is, for example, a Driver Status Monitor (registered trademark) manufactured by Denso Corporation, and captures an image of a driver sitting in the driver's seat of the vehicle and an occupant sitting in the passenger seat. The user imaging unit 31 estimates the eye position coordinates, head direction angle θ1, and gaze angle θ2 of the user in the vehicle from image data including the captured face of the user, for example, based on a known image authentication technology. The user imaging unit 31 estimates the eye position coordinates, head direction angle θ1, and gaze angle θ2 of the user at any time, and outputs the estimation result to the display control unit 2.

[0029] The user imaging unit 31 may be any other known imaging device as long as it has a camera or the like capable of capturing an image of the user's face and can calculate the head angle θ1 and the gaze angle θ2 using a known image authentication technology. The user imaging unit 31 may be disposed in any position as long as it can capture an image of the user's face. For example, the user imaging unit 31 may calculate the position of each of the user's eyes or the center position of both eyes, and calculate the gaze angle θ2 for each of the right eye and the left eye, or may calculate the average value of both eyes as a representative value of the gaze angle θ2.

[0030] The in-vehicle LAN 4 is, for example, an in-vehicle communication bus that connects the display control unit 2 and the in-vehicle device 3 and enables communication between them. LAN is an abbreviation for Local Area Network.

[0031] The above is the basic configuration of this display system when applied to vehicle-mounted applications.

[0032] [Estimation of visual recognition area] Next, the estimation of visual recognition area R1 by visual recognition area estimation unit 23 will be described with reference to the drawings. Although Fig. 6 to Fig. 8 do not show cross sections, hatching is applied to the space toward an area that is located outside a first predetermined area R2 described later and is estimated to be part of visual recognition area R1.

[0033] First, a first estimation method for the visual recognition area R1 will be described.

[0034] Normally, as shown in FIG. 5, for example, the user's face direction and line of sight direction, i.e., head direction angle θ1 and line of sight angle θ2, are approximately consistent, including when the user changes the direction of the face. Note that "approximately consistent" includes cases where the head direction angle θ1 and line of sight angle θ2 are completely consistent, as well as cases where these angles are slightly misaligned. In this case, the visual recognition area estimation unit 23 estimates the visual recognition area R1 based on the head direction angle θ1. Specifically, for example, when acquiring information on the head direction angle θ1, the visual recognition area estimation unit 23 calculates an intersection P1 between a virtual straight line VL1 along the direction of the head direction angle θ1 and the display surface 1a, as shown in FIG. 3, and calculates a first predetermined area R2, which is a predetermined range centered on the intersection P1. For example, the first predetermined area R2 is a predetermined area that is at least within the user's effective visual field V E The visual recognition area estimation unit 23 estimates the first predetermined area R2 calculated as above as the visual recognition area R1, for example, by 20° in the vertical direction and 30° in the horizontal direction, with the intersection point P1 as the center, so as to include the visual recognition area R1.

[0035] In addition, for example, when the visual recognition area estimation unit 23 acquires information on the line of sight angle θ2, as shown in FIG. 4, it calculates an intersection P2 between a virtual straight line VL2 along the line of sight angle θ2 and the display surface 1a, and calculates a second predetermined area R3 that is a predetermined range centered on the intersection P2. For example, the second predetermined area R3 is, like the first predetermined area R2, at least the effective visual field V of the user. E The range of a predetermined visual angle centered on the intersection point P2 is calculated so as to include the first predetermined region R2 and the second predetermined region R3. The numerical values ​​of the predetermined ranges in the calculation of the first predetermined region R2 and the second predetermined region R3 are not limited to the above example, and may be changed as appropriate depending on the distance between the user and the display 1, etc.

[0036] Next, a second estimation method for the visual recognition area R1 will be described.

[0037] In a situation where the head direction angle θ1 and the gaze angle θ2 deviate by a predetermined amount or more, the visual recognition area estimation unit 23 estimates the visual recognition area R1 by a second estimation method different from the above-mentioned first estimation method. There are two cases where the head direction angle θ1 and the gaze angle θ2 deviate by a predetermined amount or more: a case where the gaze angle θ2 changes before the head direction angle θ1, and a case where the gaze angle θ2 changes after the head direction angle θ1. In other words, the second estimation method is executed in a situation where a time lag occurs between a change in the head direction angle θ1 and a change in the gaze angle θ2.

[0038] The case where the line of sight angle θ2 changes before the head angle θ1 is, for example, as shown in FIG. 6, a situation where the user changes only the line of sight without changing the direction of the face. In this situation, if the first predetermined region R2 is estimated as the visual recognition region R1 and control is executed to make the external pixel group less bright than the visual recognition region pixel group, a part of the external pixel group is located within the actual effective visual field of the user and its brightness is reduced, so that the user may feel uncomfortable. For this reason, the visual recognition region estimation unit 23 calculates the first predetermined region R2 and the second predetermined region R3, and calculates the protruding region R31 located outside the first predetermined region R2 in the second predetermined region R3. Then, the visual recognition region estimation unit 23 estimates the summed region obtained by adding the protruding region R31 to the first predetermined region R2 as the visual recognition region R1. In other words, the visual recognition region estimation unit 23 estimates a wide region obtained by adding the region on the leading side of the user's line of sight to the first predetermined region R2 as the visual recognition region R1. As a result, the visual recognition area R1 is estimated as a wide area including the areas located in the user's actual central visual field and effective visual field.

[0039] On the other hand, the case where the line of sight angle θ2 changes with a delay relative to the head angle θ1 is, for example, as shown in FIG. 7, a situation where the user changes only the direction of the face without changing the line of sight. In this situation, if the estimation of the visual recognition area R1 by the first estimation method and the brightness control that reduces the brightness of the external pixel group are performed, the user may feel uncomfortable, as in the above case. In the case where the line of sight angle θ2 changes with a delay relative to the head angle θ1, the visual recognition area estimation unit 23 estimates the sum area obtained by adding the overflow area R31 to the first predetermined area R2 as the visual recognition area R1, as in the above case. In this case, the visual recognition area R1 is estimated as a wide area obtained by adding the overflow area R31, which is located at the position where the user keeps looking at the first predetermined area R2, located in front of the user's face, to the position where the user keeps looking, that is, the side where the line of sight is delayed. This prevents the user's effective field of view from being included in the external area R0, and thus prevents the brightness control that makes the user feel uncomfortable.

[0040] When the gaze angle θ2 changes before or after the head direction angle θ1, there is a difference in the timing of the changes in the head direction angle θ1 and the gaze angle θ2, but according to the second estimation method, a predetermined range of an area including the user's effective visual field can be estimated as the visual recognition area R1. The second estimation method is executed, for example, until the difference between the head direction angle θ1 and the gaze angle θ2 falls within a predetermined range, and when the head direction angle θ1 and the gaze angle θ2 substantially match, the method is switched to the first estimation method. In other words, the second estimation method is executed from when one of the head direction angle θ1 and the gaze angle θ2 changes before the other until the head direction angle θ1 and the gaze angle θ2 substantially match.

[0041] Next, a third estimation method for the visual recognition area R1 will be described.

[0042] When the visual recognition area estimation unit 23 is unable to obtain information on the user's gaze angle θ2, it estimates the visual recognition area R1 by a third estimation method. An example of a situation in which the user is unable to obtain information on the user's gaze angle θ2 is when the user is wearing sunglasses. Even in such a situation, the user's entire face is captured, so there is no particular effect on the calculation of the head direction angle θ1. In this case, the visual recognition area estimation unit 23 calculates a first predetermined area R2 and estimates the angular range of the gaze angle θ2 based on the head direction angle θ1. For example, as shown in FIG. 8, the visual recognition area estimation unit 23 estimates the angular range R of the gaze angle θ2 centered on a virtual straight line VL1 aligned with the head direction angle θ1. θ2 and estimate the estimated angle range R θ2 The visual recognition area estimation unit 23 calculates the maximum range of the second predetermined area R3 in the first predetermined area R2. The visual recognition area estimation unit 23 calculates an adjacent area R4, which is an area adjacent to the first predetermined area R2 and is a portion of the maximum range of the second predetermined area R3 that protrudes from the first predetermined area R2. The visual recognition area estimation unit 23 then estimates, as the visual recognition area R1, a sum area obtained by adding the adjacent area R4 to the first predetermined area R2. As a result, even if information on the user's line of sight θ2 cannot be obtained, the visual recognition area estimation unit 23 can estimate a wide area including the user's effective visual field as the visual recognition area R1.

[0043] In addition, the angle range R θ2 For example, a database of the line of sight angle θ2 corresponding to the head direction angle θ1 may be created in advance and stored in a recording medium (not shown), and the database may be read when the head direction angle θ1 is acquired. θ2 If the head angle θ1 is 5°, the value corresponding to the head angle θ1 = 5° is read from the database above, and R θ2 =5°±5°. The database of gaze angle θ2 is, for example, an angle within a predetermined range centered on head direction angle θ1, and a range of θ2 is set for each of a plurality of sections corresponding to the numerical value of head direction angle θ1.

[0044] The third estimation method is executed, for example, while information on the user's line of sight angle θ2 cannot be acquired, and is switched to the first or second estimation method when information on the user's line of sight angle θ2 is acquired.

[0045] [Brightness Control] Next, an example of brightness control of the display 1 in this display system will be described.

[0046] For example, when a predetermined start condition is satisfied, such as the ignition of the vehicle being turned on or the display 1 being turned on, the display control unit 2 executes the control flow shown in FIG.

[0047] In step S110, for example, the display control unit 2 acquires user information from the user imaging unit 31 including the user's eye position coordinates, the line of sight angle θ2, and at least the head direction angle θ1 out of the head direction angle θ1.

[0048] In the subsequent step S120, for example, the display control unit 2 determines whether or not the user information acquired in step S110 includes the gaze angle θ2, and if the determination is positive, the processing proceeds to step S140, and if the determination is negative, the processing proceeds to step S130.

[0049] In step S130, since the user's line of sight angle θ2 cannot be directly acquired due to factors such as the user wearing sunglasses, the visual recognition area estimation unit 23 estimates the visual recognition area R1 by the above-mentioned third estimation method. As a result, the area obtained by adding the adjacent area R4 located around the first predetermined area R2 is estimated as the visual recognition area R1.

[0050] In step S140, for example, the display control unit 2 determines whether or not a discrepancy has occurred between the changes in the head direction angle θ1 and the line of sight angle θ2, that is, whether or not one of the head direction angle θ1 and the line of sight angle θ2 has changed before the other. For example, if the determination in step S140 is positive, the display control unit 2 advances the process to step S160, and if the determination is negative, the process advances to step S150.

[0051] In step S150, since the difference between the head angle θ1 and the line of sight angle θ2 is equal to or smaller than a predetermined value, that is, they are substantially equal to each other, the visual recognition area estimation unit 23 estimates the visual recognition area R1 by the above-mentioned first estimation method. As a result, the first predetermined area R2 calculated based on the head angle θ1 is estimated as the visual recognition area R1.

[0052] In step S160, since the line of sight angle θ2 is ahead of or behind the head direction angle θ1, the visual recognition area estimation unit 23 estimates the visual recognition area R1 by the above-mentioned second estimation method. As a result, a wide area obtained by adding an overhanging area R31 that is overhanging the first predetermined area R2 on the user's line of sight side of the second predetermined area R3 to the first predetermined area R2 is estimated as the visual recognition area R1.

[0053] Then, for example, the display controller 2 estimates the visual recognition area R1 by any one of steps S130, S150, and S160, and then proceeds to the process at step S170. In step S170, for example, the luminance controller 24 sets the visual recognition area pixel group and the exterior pixel group based on the visual recognition area R1 estimated by any one of steps S130, S150, and S160. Then, for example, the luminance controller 24 executes luminance control to make at least the luminance of the exterior pixel group smaller than the luminance of the visual recognition area pixel group.

[0054] This allows the brightness of the external region R0, which is located outside the user's effective visual field and where the user cannot recognize the details of the image, to be reduced without reducing the brightness of the portion of the image displayed on the display 1 that is in front of the user's line of sight. Therefore, even when one of the head angle θ1 and the line of sight angle θ2 is delayed relative to the other or when the line of sight angle θ2 cannot be obtained, the visual recognition region R1 can be estimated with high accuracy, delays in brightness control can be suppressed, and an image that does not cause the user to feel uncomfortable can be displayed. In addition, the brightness of the user's peripheral visual field and areas outside of it is reduced, resulting in low power consumption.

[0055] The display control unit 2 repeats the above-described processes from step S110 onward until a predetermined end condition is met, such as when the display 1 is turned off after the process of step S170 is completed. As a result, the visual recognition area R1 is estimated by the first estimation method when the head direction angle θ1 and the gaze angle θ2 are approximately the same, the second estimation method when one of the head direction angle θ1 and the gaze angle θ2 is delayed relative to the other, and the third estimation method when the gaze angle θ2 cannot be acquired. Then, by executing brightness control based on the visual recognition area R1 obtained by the first to third estimation methods, it is possible to suppress delays in brightness control, and it is possible to reduce both the discomfort felt by the user and power consumption.

[0056] In the brightness control of step S170, in accordance with the brightness control for the external pixel group, in order to make the portion of the image displayed in the visible region R1 easier to see, control may be executed to increase the brightness of a part or all of the visible region pixel group as necessary. For example, in this display system, when the in-vehicle device 3 has an illuminance sensor and the illuminance of the environment in which the display 1 is placed is equal to or higher than a predetermined value, control may be executed to increase the brightness of the portion in the region exposed to sunlight.

[0057] According to this embodiment, when the user's line of sight angle θ2 cannot be obtained and when one of the user's head angle θ1 and line of sight angle θ2 is delayed relative to the other, an area wider than the first predetermined area R2 obtained based on the head angle θ1 is estimated as the visual recognition area R1. Then, the display system performs brightness control to reduce the brightness of the external pixel group based on the estimated visual recognition area R1 compared to the visual recognition area pixel group, thereby suppressing delays in the estimation of the visual recognition area R1 and therefore in the brightness control, while achieving the effect of reducing power consumption. In addition, the estimation accuracy of the visual recognition area R1 is improved and the visual recognition area R1 is set as a wide area including the user's effective visual field, making it possible to display an image that does not cause the user to feel uncomfortable.

[0058] (Other embodiments) Although the present disclosure has been described based on the embodiment, it is understood that the present disclosure is not limited to the embodiment or structure. The present disclosure also includes various modifications and modifications within the equivalent range. In addition, various combinations and forms, and other combinations and forms including only one element, more than one, or less than one, are also within the scope and concept of the present disclosure.

[0059] The display control unit 2 and the method described in the present disclosure may be realized by a dedicated computer provided by configuring a processor and a memory programmed to execute one or more functions embodied in a computer program. Alternatively, the display control unit 2 and the method described in the present disclosure may be realized by a dedicated computer provided by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, the display control unit 2 and the method described in the present disclosure may be realized by one or more dedicated computers configured by a combination of a processor and a memory programmed to execute one or more functions and a processor configured with one or more hardware logic circuits. In addition, the computer program may be stored in a computer-readable non-transitory tangible recording medium as instructions executed by a computer.

[0060] In each of the above embodiments, it goes without saying that the elements constituting the embodiment are not necessarily essential, except when it is specifically stated that they are essential or when it is clearly considered essential in principle. In addition, in each of the above embodiments, when the numbers, values, amounts, ranges, etc. of the components of the embodiment are mentioned, they are not limited to the specific numbers, except when it is specifically stated that they are essential or when it is clearly limited to a specific number in principle. In addition, in each of the above embodiments, when the shapes, positional relationships, etc. of the components are mentioned, they are not limited to the shapes, positional relationships, etc., except when it is specifically stated that they are essential or when it is clearly limited to a specific shape, positional relationship, etc. in principle. [Explanation of symbols]

[0061] 1...display, 1a...display surface, 21...head direction angle acquisition unit, 22...gaze angle acquisition unit 23: visual recognition area estimation unit, 24: brightness control unit, θ1: head direction angle, θ2: line of sight angle R1: visible region, R2: first predetermined region, R3: second predetermined region, R31: protruding region R4: Adjacent area

Claims

1. A display system for performing brightness control of a display (1), comprising: a head direction angle acquisition unit (21) that acquires information on a head direction angle (θ1), which is an angle of the front direction of the face of the user with respect to a display surface (1 a), from a user imaging unit (31) that images a user of the display; a gaze angle acquisition unit (22) that acquires information on a gaze angle (θ2) that is an angle of the user's gaze direction with respect to the display surface from the user imaging unit; A visual recognition area estimation unit (23) that estimates a part of the display surface and an area including at least the effective visual field of the user as a visual recognition area (R1) based on at least one of the line of sight angle and the head direction angle; a brightness control unit (24) that executes brightness control to make a brightness of a pixel group located in an area other than the visible area of ​​the display smaller than a brightness of a pixel group located in the visible area, The visual recognition area estimation unit is a display system in which, when information on both the head direction angle and the gaze angle is acquired, and one of the head direction angle and the gaze angle changes before the other, a region of a predetermined range on the display surface centered on the intersection of the display surface with a virtual line along the head direction angle is calculated as a first predetermined region (R2), a region of a predetermined range on the display surface centered on the intersection of the display surface with a virtual line of the gaze angle is calculated as a second predetermined region (R3), and an added region obtained by adding an overhanging region (R31) of the second predetermined region that is located outside the first predetermined region to the first predetermined region is estimated as the visible region.

2. The visual recognition area estimation unit is The addition area is estimated as the visual recognition area from when the gaze angle changes before the head direction angle until a difference between the head direction angle and the gaze angle becomes equal to or smaller than a predetermined value, The display system according to claim 1 , wherein when a difference between the head direction angle and the line of sight angle is equal to or smaller than a predetermined value, the first predetermined area is estimated as the visual recognition area.

3. The display system of claim 1, wherein when information on the line of sight is not acquired, the visual recognition area estimation unit calculates the first specified area and estimates the area obtained by adding an adjacent area (R4) adjacent to the first specified area to the first specified area as the visual recognition area.

4. 4. The display system according to claim 3, wherein the visual recognition area estimation unit estimates a predetermined range of angles centered on the head direction angle as the range of the gaze angle, and estimates an area of ​​the second predetermined area that is located outside the first predetermined area within the range of the gaze angle as the adjacent area.

5. The display system according to claim 1 , wherein the brightness control unit performs brightness control to make the brightness of some or all of the pixels located in the visible region higher than the brightness of the pixels located in a region other than the visible region.

Citation Information

Patent Citations

  • Auxiliary device for driving operation

    JP1997076815A

  • Display device, display method, and display program

    JP2015122711A

  • Display controller for vehicle, method for display for vehicle, and display program for vehicle

    JP2023004754A

  • Visibility detection device

    JP7753973B2

  • High dynamic range for head-mounted display device

    US20190138088A1