Display control device, display device, display system, and image display control method

Abstract

To suppress occurrence of visual discomfort, when a passenger(s) other than a driver is / are not onboard, felt by the driver due to seeing images on which distortion correction suitable for the passenger(s) is performed.SOLUTION: A display control device which is placed in a vehicle and controls image display on display devices which show images to the driver and passengers other than the driver of the vehicle includes a control unit having a function of changing a correction parameter which is used to correct image data of images so that distortions on images seen by the driver or the passengers are reduced, and when a display device's primary viewer is a passenger and a non-primary viewer is the driver, the control unit, as the result of a determination based on at least one of information on whether the passenger is onboard and information on where a passenger is seated, sets the correction parameter value to a first value of M-wp which is suitable for the passenger when it has been determined that the passenger is onboard, and sets the correction parameter value to a second value of S-wp which is suitable for the driver and differs from the first value, when it has been determined that the passenger is not onboard.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to a display control device, a display device, a display system, an image display control method, and the like. [Background technology]

[0002] There may be cases where an image displayed by a display device mounted in a vehicle (vehicle display device) is viewed by passengers other than the driver.

[0003] In this regard, for example, Patent Document 1

[0004] It is stated that "...if there is a passenger in the passenger seat in addition to the driver and it is desired that the passenger seated in the passenger seat can also see the display image consisting of vehicle information, etc., then in addition to the head-up display device described above (i.e., the head-up display device for the driver's seat), it is possible to consider disposing another head-up display device for the passenger seat separately inside the instrument panel in front of the passenger seat."

[0004] Furthermore, as an example of a process for correcting distortion of a displayed image, for example, Patent Document 2 describes a warping process in a HUD device (a correction process that adds in advance to image data a distortion with the opposite characteristics to the distortion caused by an optical system, etc.). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2012-108470 A [Patent Document 2] JP 2015-87619 A Summary of the Invention [Problem to be solved by the invention]

[0006] Through the study of the present inventors, the following problems have become clear: For example, there may be a case where the display device displays an image to be visually recognized by an occupant, but the occupant is not present.

[0007] Since the display image is intended to be viewed by a passenger, the image data is often subjected to correction processing, for example, to correct distortion of the image when viewed from the front by a passenger.

[0008] When there are no occupants present and the driver looks at the displayed image at an angle, for example to obtain information, the image is based on how it would appear to an occupant looking at it from the front, and so the driver may perceive the image as significantly distorted, which may cause discomfort to the driver.

[0009] The above-mentioned Patent Documents 1 and 2 make no mention of this problem, nor do they provide any description of how to address it.

[0010] One of the objectives of the present invention is to prevent the driver from feeling uncomfortable when viewing an image that has been subjected to distortion correction suitable for an occupant other than the driver when no occupants other than the driver are present.

[0011] Other objects of the present invention will become apparent to those skilled in the art by referring to the following exemplary aspects and best modes, as well as the accompanying drawings. [Means for solving the problem]

[0012] In order to facilitate an understanding of the outline of the present invention, embodiments according to the present invention will be exemplified below.

[0013] In a first aspect, display device teeth, A display device mounted in a vehicle and capable of displaying an image to a driver of the vehicle and passengers other than the driver. Place There was, a control unit having a function of changing a correction parameter used when correcting image data of the image so as to suppress distortion of the image as seen by the driver or the passenger; and, an image data correction unit that corrects the image data using the correction parameters; a display unit that is disposed at a position corresponding to the occupant and displays the image based on the corrected image data, The control unit is setting a value of the correction parameter to a first value suitable for the occupant when the occupant is present as a result of determination based on at least one of the occupant presence information and the occupant location information; When the occupant is absent, the value of the correction parameter is set to a second value suitable for the driver, the second value being different from the first value.

[0014] In the first aspect, when a passenger who is a primary viewer of an image displayed by a display device is absent, the correction parameters are changed so that the image is suitable for a driver who is a secondary viewer.

[0015] If the displayed image is left with correction processing, for example, for correcting image distortion when viewed from the front by a passenger, the driver may feel uncomfortable when viewing the displayed image from an oblique angle. In this embodiment, however, the correction parameters are changed to those suitable for a driver viewing the image from an oblique angle, and a corrected image is displayed so that the distortion as viewed by the driver is reduced. Thus, the driver's discomfort is suppressed.

[0016] In a second embodiment dependent on the first embodiment, before The crew: A passenger sitting in the front passenger seat of the vehicle. Or, The passenger may be a passenger seated in a rear row seat of the vehicle.

[0017] In the second aspect, it is clarified that the "occupant who is the primary viewer of the image displayed by the display device" described in the first aspect may include an occupant in the front passenger seat of the vehicle, and may also include an occupant in the rear seat of the vehicle.

[0018] For example, there may be cases where a display device for the passenger seat is installed inside the dashboard in front of (preferably in front of) the passenger sitting in the passenger seat, or on the surface of the dashboard, etc. In other words, the display device in this case can be said to be a display device that is placed in a position directly facing the passenger sitting in the passenger seat (however, the term "directly facing" is interpreted broadly and includes positions other than strictly in front), and in this case, the passenger sitting in the passenger seat is the "primary viewer of the display device."

[0019] Also, for passengers sitting in the back seats, a display device may be installed inside or on the surface of the dashboard in front of the passenger, preferably in a position directly facing the passenger (however, the term "directly facing" is interpreted broadly and includes positions other than strictly in front), to provide, for example, navigation information or entertainment information. In this case, the passenger sitting in the back seat becomes the "primary viewer of the display device."

[0020] In the above-mentioned case, when the "occupant who is the primary viewer of the display device" is absent, the driver must view the image displayed by the display device at an angle, and unless some countermeasure is taken, the image will become distorted. By applying the present invention, the image distortion is reduced, and the sense of incongruity is suppressed.

[0021] In a third aspect dependent on the first or second aspect, The correction parameters are A warping parameter for pre-distortion correction that pre-applies to the image data a distortion with characteristics opposite to the distortion that occurs when the image is displayed; And, When the outline of the display area of ​​the image is a rectangle including a rectangle and a square, if the rectangle appears to be deformed into a trapezoid or a parallelogram in response to an angle at which a viewer views the image, a transformation parameter for deformation suppression and correction for transforming the trapezoid or parallelogram into a rectangle; And, In the case of three-dimensional (3D) display, the information may include at least one of a 3D image correction parameter for correcting distortion of the 3D image.

[0022] In the third aspect, it is clarified that the correction parameter may be a warping parameter, may be a transformation parameter for deformation suppression correction, or may be a correction parameter for a 3D image.

[0023] By carrying out the above correction using appropriate correction parameters, image distortion can be effectively reduced, and the sense of discomfort can be suppressed.

[0024] In a fourth aspect dependent on any one of the first to third aspects, The control unit is When the operation of the display device is started, before the value of the correction parameter is determined, The above After the image is hidden and the value of the correction parameter is determined, The above An image may be displayed.

[0025] According to the fourth aspect, by displaying an image after determining the correction parameters to be applied, it is possible to reduce the discomfort felt by the viewer due to display distortion, compared to, for example, a case in which the correction parameters are changed after the image is displayed.

[0026] In a fifth aspect dependent on any one of the first to third aspects, The control unit is When the operation of the display device is started, 、 before Crew Member exists In the case, after the value of the correction parameter is determined, The above An image may be displayed.

[0027] In the fifth aspect, a display device whose primary viewer is a driver can immediately start displaying an image, and can present, for example, information useful for driving a vehicle without delay, etc. On the other hand, a display device whose primary viewer is a passenger can display an image with less distortion by disclosing the display after the presence or absence of the passenger is confirmed and the value of the correction parameter is determined.

[0028] In a sixth aspect dependent on any one of the first to third aspects, The control unit is When the operation of the display device is started, and, before While displaying the correction parameter by setting the correction parameter to the first value, it is determined whether or not the correction parameter needs to be changed, and if it needs to be changed, The above The value of the correction parameter may be changed.

[0029] According to the sixth aspect, even when the passenger is the main viewer, the image can be displayed quickly without waiting for the value of the correction parameter to be determined.

[0030] In a seventh aspect dependent on any one of the first to sixth aspects, The control unit is The above When the value of the correction parameter is changed while the image is being displayed, the value may be changed gradually from a pre-change value to a post-change value.

[0031] According to the seventh aspect, when it becomes necessary to change the correction parameter, the value is gradually changed, thereby preventing the appearance of the image from changing abruptly, thereby reducing the sense of discomfort felt by occupants, etc.

[0034] First aspect Subordinate to Eighth aspect In The display device includes: The head-up display device may further include an optical system that projects display light of the image displayed on the display unit onto a projection target member provided in the vehicle, and has a function of allowing at least one of the driver and the passenger to visually recognize a virtual image, Or, The display device may display the image of a real image on the display unit as a display panel, and allow at least one of the driver and the passenger to visually recognize the image of the real image.

[0035] Eighth aspect It is clarified that the display device may be a HUD device or a display device. It is possible to realize a high-performance HUD device or display device that can display images with little distortion to both the driver and passengers.

[0036] As the display device, a liquid crystal device (for example, a TFT liquid crystal device) or a micro LED display device, which is a self-luminous display device, may be used.

[0045] Those skilled in the art will easily understand that the exemplified embodiments according to the present invention can be further modified without departing from the spirit of the present invention. [Brief description of the drawings]

[0046] [Figure 1] FIG. 1(A) is a diagram showing an example of the path of the display light of each display device when two display devices (HUD devices) are placed in a vehicle with two seats in the front row, one for the driver and one for the passenger in the front seat, and FIG. 1(B) is a diagram showing an example of the path of the display light of each display device when no passenger is present. [Diagram 2] FIG. 2A is a diagram showing an example of the configuration of a HUD device having a warping correction function, and FIG. 2B is a diagram showing the effect of the warping correction process. [Diagram 3]FIG. 3(A) is a diagram showing an example of an image display before deformation suppression correction in which a trapezoid or parallelogram as the contour shape of the image display area is converted into a rectangle (including a rectangle and a square), FIG. 3(B) is a diagram showing an example of an image display after deformation suppression correction, and FIG. 3(C) is a diagram showing an example of the main configuration of a HUD device having a deformation suppression correction function. [Figure 4] FIG. 4 is a diagram showing an example of a display system in a vehicle with three seats in the front row. [Diagram 5] 5(A) and (B) are diagrams showing an example of basic display control in the display system of FIG. [Figure 6] 6A and 6B are diagrams showing an example of display control when the display device starts operating. [Figure 7] 7A and 7B are diagrams showing another example of display control when the display device starts operating. [Figure 8] 8(A) and (B) are diagrams showing still another example of display control when the display device starts operating. [Figure 9] Figure 9(A) shows an example in which one of the three display devices is used to display images to passengers sitting in the back seat, and Figure 9(B) shows an example in which two of the three display devices located close to the driver are both used to display images to the driver. [Figure 10] FIG. 10(A) is a diagram showing an example of a display system using a display device (such as a liquid crystal display device), and FIG. 10(B) is a diagram showing an example of a display system that combines a HUD device and a display device. [Figure 11] FIG. 11 is a diagram showing an example in which the correction parameter value is gradually changed when the correction parameter is changed. [Figure 12] FIG. 12 is a diagram illustrating an example of a main configuration of a display device and an example of a display system. [Figure 13] FIG. 13 is a flowchart showing an example of a display control procedure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0047] The best mode described below is used to easily understand the present invention. Therefore, those skilled in the art should be aware that the present invention is not unduly limited by the mode described below.

[0048] (First embodiment) FIG. 1(A) is a diagram showing an example of the path of the display light of each display device when two display devices (HUD devices) are placed in a vehicle with two seats in the front row, one for the driver and one for the passenger in the front seat, and FIG. 1(B) is a diagram showing an example of the path of the display light of each display device when no passenger is present.

[0049] 1, the X direction indicates the left-right direction of the vehicle 1, the Y direction indicates the height direction of the vehicle 1, and the Z direction indicates the front (front-rear direction) of the vehicle 1. As shown in the figure, the vehicle 1 is a passenger car with two seats in the front row and a steering wheel 8 on the right side.

[0050] Two display devices (HUD devices in this case) 100a and 100b are arranged along the width direction (X direction) of the vehicle 1 in a front portion on the windshield 2 side of the vehicle 1. The HUD device 100a is arranged at a position corresponding to (directly facing) the driver's seat 6, and the HUD device 100b is arranged at a position corresponding to the passenger seat 9.

[0051] Each of the HUD devices 100a, 100b can be installed, for example, inside a dashboard in the front part of the vehicle 1. Note that the display device is not limited to the HUD device, and a display device such as a liquid crystal device (for example, a TFT liquid crystal device) or a micro LED display device, which is a self-luminous display device, may be used (an example of using a display device will be described later with reference to FIG. 10).

[0052] In Fig. 1(A), a driver 7 is seated in a driver's seat 6, and a passenger (fellow passenger) 10 is seated in a passenger seat 9. No passenger is seated in a rear seat 11.

[0053] In addition, the driver 7 is present (seated) and the passenger 10The presence (seating) of a passenger in the rear seat 11 and the absence of a passenger in the rear seat 11 can be determined, for example, by performing image analysis on the image captured by the camera 200.

[0054] The HUD devices 100a and 100b project display light of an image onto the windshield 2, and allow the driver 7 and the passenger 10 to view the image (virtual image). The HUD device 100a is a display device in which the driver 7 is the primary viewer and the passenger 10 is the secondary viewer.

[0055] The fact that the ignition switch of the vehicle 1 is turned on means that the driver 7 is necessarily present. Therefore, the HUD device 100a always assumes that the driver 7 will be viewing the image, performs correction (distortion correction) to suppress image distortion, and allows the driver 7 to view the corrected image.

[0056] The driver 7 views the display image of the HUD device 100a from the front. In other words, the driver 7 can be said to be a viewer who is positioned directly facing the HUD device (display device) with the driver 7 as the main viewer, or the display image. Note that the meaning of "directly in front" or "directly facing" is to be interpreted in a broad sense, and a range that can be regarded as "directly in front" or "directly facing" may be defined with some variation.

[0057] The HUD device 100b is a display device for the occupant 10 as a primary viewer and the driver 7 as a secondary viewer. In other words, the HUD device 100b basically performs correction (distortion correction) to suppress image distortion, assuming that the occupant 10 will be viewing the image, and displays the corrected image as Crew: 10 However, when the occupant 10 is absent, the HUD device 100b can perform image distortion correction assuming that the driver 7 (secondary viewer) is viewing the image (see FIG. 1(B)).

[0058] The HUD devices 100a and 100b incorporate a display control device (processor: reference numeral 159 in FIG. 12, not shown in FIG. 1) having a function of changing correction parameters for distortion correction. An example of display control by the display control device will be described later.

[0059] Furthermore, the image distortion correction may include at least one of a "pre-distortion correction process (warping correction process)" that applies a distortion having the opposite characteristics to the distortion that occurs when the image is displayed in advance to the image data, a "deformation suppression correction process" that converts a trapezoid or parallelogram into a rectangle when the outline of the display area of ​​the image is a rectangle including a rectangle and a square and the rectangle appears to be deformed into a trapezoid or parallelogram depending on the angle at which the viewer views the image, and a "3D image correction process" that corrects distortion of a three-dimensional (3D) image in the case of a 3D display.

[0060] 1B, the passenger 10 is absent, and only the driver 7 is present in the vehicle. In this state, only the driver 7 can view the image displayed by the HUD device 100b, and the driver 7 views the image from an oblique angle.

[0061] If the image were to remain as it is after distortion correction intended for viewing by the occupant 10, the image would likely appear significantly distorted to the driver 7, which could cause discomfort.

[0062] Therefore, in this embodiment, in a case like that shown in Fig. 1(B), the correction parameters are changed (for example, the correction parameters are switched) to perform distortion correction on the image data suitable for the driver 7 who is the secondary viewer, and the distortion-corrected image is displayed to the driver 7 (details will be described later). This can reduce (suppress) the sense of discomfort.

[0063] In the example of Figure 1, there is no passenger in the rear seat 11, but if there is at least one passenger in the rear seat 11 and that passenger is seated in a position where he or she can see the image displayed at the front (or the front part) of the vehicle 1, it is also possible to make that passenger the primary viewer and the target of the image display (this point will be described later with reference to Figures 9 and 10).

[0064] In addition, in the example of Fig. 1, two display devices are used, but the number of display devices to be used is not particularly limited. For example, as in the example of Fig. 4, three HUD devices can be used.

[0065] By using multiple display devices with built-in display control devices, a high-performance display system can be realized that is easy to use and can display images with little distortion to both the driver and passengers.

[0066] Next, reference will be made to Fig. 2. In Fig. 2, a case where warping is performed as the distortion correction will be described.

[0067] FIG. 2A is a diagram showing an example of the configuration of a HUD device having a warping correction function, and FIG. 2B is a diagram showing the effect of the warping correction process.

[0068] The HUD device 100 is mounted on a vehicle (which can be broadly interpreted) 1. The HUD device 100 has a display unit (for example, a light-transmitting screen or a liquid crystal display device, etc.) 101, a reflecting mirror (flat mirror) 103, a curved mirror (for example, a concave mirror, the reflecting surface of which may be a free-form surface) 105 as an optical member that projects display light, an image generating unit 150, a control unit 160, and a ROM (storage device) 210 having an image conversion table 212 in which a warping parameter WP is stored as a correction parameter.

[0069] The control unit 160 can be constructed as a functional block included in a display control device (processor: reference numeral 159 in FIG. 12). The control unit 160 can selectively retrieve the warping parameters WP from the ROM 210. In other words, the control unit 160 has a function of appropriately changing the warping parameters WP to be used based on information on the presence or absence of occupants in the vehicle 1 and information on the location of the occupants.

[0070] An image displayed on display unit 101 is projected onto a projection possible area 5 of windshield 2, which is a projection target, via reflecting mirror 103 and curved mirror 105. Reference numeral 4 denotes the projection area of ​​the image.

[0071] A plurality of curved mirrors may be provided in the HUD device 100. In addition to the mirror (reflective optical element) of this embodiment, or instead of a part (or the whole) of the mirror (reflective optical element) of this embodiment, a configuration including a refractive optical element such as a lens, or a functional optical element such as a diffractive optical element may be adopted.

[0072] A portion of the display light of the image is reflected by the windshield 2 and enters the viewpoint (eye) A of the driver or the like located inside (or on) a predetermined eye box EB (which is actually three-dimensional, but for ease of explanation, is drawn as a rectangle of a specified area in Figure 2 (A)), and is imaged in front of the vehicle 1, whereby a virtual image V is displayed on a virtual virtual image display surface PS corresponding to the display surface 102 of the display unit 101.

[0073] The image on the display unit 101 is distorted due to the influence of the shape of the curved mirror 105, the shape of the windshield 2, and the like. In other words, distortion occurs due to the optical components including the optical system of the HUD device 100 and the windshield 2. In order to suppress this distortion, a distortion having the opposite characteristics to the distortion is given to the image in advance (warping process, or warping image correction process).

[0074] FIG. 2(B) shows an example of a virtual image V visually recognized by a driver (reference numeral 7 in FIG. 1) through the windshield 2. In FIG. 2(B), the virtual image V having a rectangular outer shape is provided with, for example, 5 vertically and 5 horizontally, a total of 25 reference points (reference pixel points or coordinate points) GD(i,j) (where i and j are variables that can take values ​​of 1 to 5). A distortion having an inverse characteristic to the distortion caused in the virtual image V by the warping process is given in advance to each reference point (each coordinate point) in the image (original image). Therefore, the distortion given in advance and the distortion that actually occurs are offset, and ideally, a virtual image V without a curve as shown in FIG. 1(B) is displayed. The number of reference points GD(i,j) can be appropriately increased by an interpolation process or the like.

[0075] Next, reference is made to Fig. 3. Fig. 3(A) is a diagram showing an example of image display before deformation suppression correction in which a trapezoid or parallelogram as the contour shape of the image display area is converted into a rectangle (including a rectangle and a square), Fig. 3(B) is a diagram showing an example of image display after deformation suppression correction, and Fig. 3(C) is a diagram showing an example of the configuration of the main parts of a HUD device having a deformation suppression correction function.

[0076] In Fig. 3(A) and (B), it is assumed that the driver is driving a vehicle with three seats at the front (with the steering wheel on the right side). Also, in Fig. 3(A) and (B), similar to the example of Fig. 2(A), the contour (outline) of the display area of ​​the virtual image (image) V on the virtual image display surface PS is a rectangle including a rectangle and a square. Note that the rectangle showing the contour of the image display area is shown by a thick solid line in Fig. 3(A) and (B).

[0077] As shown in Fig. 3(A), when a driver 7 as a viewer views an image (indicated by three open circles in the figure) in front of the driver's seat (directly facing the driver), the contour G1 of the image area is a rectangle. When viewing an image in the center position (in front of the passenger seat in the center), the image is viewed from an angle, so the contour G2 of the image display area is a trapezoid. Note that it may also be a parallelogram.

[0078] When viewing an image at the leftmost position (front position of the passenger seat on the left edge), the image is viewed from an even more oblique angle, so the contour G3 of the image display area becomes an even more distorted trapezoid.

[0079] In this way, the rectangle appears to be distorted into a trapezoid or a parallelogram depending on the angle at which the image is viewed. If this continues, the driver 7 who is viewing the image will feel uncomfortable.

[0080] Therefore, the control unit 160 performs a deformation suppression correction to convert the trapezoid (or parallelogram) into a rectangle. As a result, as shown in FIG. 3B, the contours G2 and G3 (indicated by dotted lines) of the image display area are converted into G2' and G3' (indicated by thick solid lines), respectively. This reduces image distortion and suppresses the driver's discomfort.

[0081] The configuration of Fig. 3(C) is the same as the configuration shown in the lower right of Fig. 2(A). However, in the case of Fig. 3(C), the image conversion table 212 of the ROM (storage device) 210 stores conversion parameters for deformation suppression correction (in other words, conversion parameters JM for correction to convert a trapezoid or parallelogram into a rectangle).

[0082] The transformation parameters JM are, for example, parameters that determine the contents of a projective transformation matrix that transforms a trapezoid into a rectangle, or parameters that determine the contents of a projective transformation (affine transformation) matrix that transforms a parallelogram into a rectangle.

[0083] The control unit 160 can appropriately retrieve the transformation parameters from the ROM 210 and perform the deformation suppression and correction process. In other words, the control unit 160 has a function of appropriately changing the transformation parameters.

[0084] The warping correction process described in Fig. 2 and the deformation suppression correction process described in Fig. 3 can also be applied to a case where 3D (three-dimensional) display is performed using binocular disparity. In this case, the distortion of the 3D image can be reduced by performing the above-mentioned distortion correction process on each of the images for the left and right eyes of the viewer (driver).

[0085] It is also possible to use both the warping correction process described in Fig. 2 and the deformation suppression correction process described in Fig. 3. In this case, for example, it is preferable to perform the deformation suppression correction process on the image data and then perform the warping correction process.

[0086] Second embodiment Next, reference is made to Fig. 4. Fig. 4 is a diagram showing an example of a display system in a vehicle with three seats in the front row. In the example of Fig. 4, the steering wheel 8 is located on the left side inside the vehicle.

[0087] In FIG. 4, three display devices (here, HUD devices) 110a, 110b, and 110c are arranged in the width direction (X direction) of the vehicle 1 in the front portion on the windshield 2 side of the vehicle 1.

[0088] HUD device 110a is disposed at a position corresponding to (directly facing) the driver's seat 16, HUD device 110b is disposed at a position corresponding to the adjacent passenger seat (center passenger seat) 18, and HUD device 110c is disposed at a position corresponding to the adjacent passenger seat (right-most passenger seat) 20.

[0089] A driver 15 is seated in the driver's seat 16, a passenger (fellow passenger) 17 is seated in the passenger seat 18, and a passenger (fellow passenger) 19 is seated in the passenger seat 20. No passenger is present in the rear seat 21. In addition, the presence (seated) of the driver 15 and the presence (seated) of the passengers 17 and 19, and the absence of a passenger in the rear seat 21 can be determined, for example, by performing image analysis on the image captured by the camera 200.

[0090] Each of the HUD devices 110a to 110c projects display light of an image onto the windshield 2, allowing the driver 15 and passengers 17 and 19 to visually recognize the image (virtual image).

[0091] The HUD device 110a is a display device in which the driver 15 is the primary viewer and the passengers 17 and 19 are secondary viewers. However, since the driver 15 is always present, similar to the example in Fig. 1, the HUD device 110a always corrects image distortion assuming that the driver 15 will be viewing the image, and displays the corrected image to the driver 15.

[0092] The HUD device 110b is a display device for which the passenger 17 is the primary viewer and the driver 15 is the secondary viewer. The HUD device 110c is a display device for which the passenger 19 is the primary viewer and the driver 15 is the secondary viewer.

[0093] Information capable of identifying the main viewer of each HUD device is input and set in advance in each of the HUD devices 110a to 110c (in the control unit 160 of the display control device).

[0094] For example, for the display device 110a, which is installed at a position corresponding to the driver 15 among the three display devices 110a to 110c, by inputting, for example, "1" on the initial setting screen, the driver 15 is set as the primary viewer.

[0095] Similarly, for display device 110b disposed at a position corresponding to the central passenger seat 18, inputting, for example, "2" on the initial setting screen sets the occupant 17 seated in the front row next to the driver's seat 16 (in other words, seated in the central passenger seat 18) as the primary viewer. Similarly, for display device 110c, inputting "3" on the initial setting screen sets the primary viewer as occupant 19 seated in the front row, in the passenger seat 20 at the far right.

[0096] In this way, after the settings are made for all the display devices, at least one of the information on the presence or absence of occupants and the information on the location of occupants is input in parallel to each of the display devices 110a-110c, and the control unit 160 of each of the display devices 110a-110c determines whether or not the correction parameters need to be changed, and the correction parameters are automatically optimized. Thus, a high-performance display system that is easy to use and can display images with little distortion to both the driver and the occupants can be realized.

[0097] Next, reference is made to Fig. 5. Figs. 5(A) and (B) are diagrams showing a basic display control example in the display system of Fig. 4. Fig. 5 is drawn corresponding to the respective arrangements of the display device, the driver, and the passenger (passenger) in Fig. 4. In addition, in the examples of Figs. 5(A) and (B), it is assumed that warping correction is performed as image distortion correction. The above points are the same in Figs. 6 to 8.

[0098] 5(A) and (B), an eye box EB10 for the driver 15 and eye boxes EB20 and EB30 for the passengers 17 and 19 are set on the near side. Image display areas G10, G11, and G12 set on the virtual image display surface PS are shown in front of the vehicle 1. The arrows in the figures indicate the line of sight of the driver 15 and the passengers (co-passengers) 17 and 19, respectively.

[0099] In the example of Fig. 5(A), there are a driver 15 and passengers (fellow passengers) 17 and 19. Therefore, in each of the display devices (HUD devices) 110a to 110c, the correction parameters used for image distortion correction are those that can display images with less distortion to the main viewer of each display device (in other words, the driver 15 and passengers (fellow passengers) 17 and 19 who are seated facing each display device).

[0100] Here, the correction parameters that are specified assuming that the main viewer will view the image are referred to as main parameters (indicated as "M-wp" in Figs. 5(A) and (B)). Note that in this specification, "M-wp" may be referred to as "first value of the correction parameters."

[0101] In addition, the correction parameter specified assuming that the driver, who is the secondary viewer, views the image when the occupant (passenger), who is the primary viewer, is absent, is referred to as a sub-parameter (sub-parameter: indicated as "S-wp" in FIG. 5(B)). Note that in this specification, "S-wp" may be referred to as a "second value of the correction parameter."

[0102] In Fig. 5(B), there is no passenger (fellow passenger) 17 in the passenger seat in the center. Accordingly, in the example of Fig. 5(B), the display device (HUD device) 110b arranged in the center of the vehicle 1 uses a sub-parameter (sub-parameter: S-wp) when correcting image distortion.

[0103] Next, reference is made to Fig. 6. Fig. 6(A) and (B) are diagrams showing an example of display control when the display device starts operating.

[0104] In FIG. 6(A), when the ignition switch (IGN) of the vehicle 1 is turned on and the operation of each display device (HUD device) 110a to 110c starts, each display device 110a to 110c (the display unit 160 of the display control device) does not display an image before the value of the correction parameter is determined.

[0105] In Fig. 6(B), the presence or absence of an occupant, the location of the occupants, etc. are detected, and the values ​​of the correction parameters are determined based on the results, after which each of the display devices 110a to 110c (the control unit 160 thereof) displays an image. The correction parameters used to display each image in Fig. 6(B) are the same as those in the example of Fig. 5(B).

[0106] In this way, by displaying the image after the correction parameters to be applied have been determined, it is possible to reduce the discomfort felt by the viewer due to display distortion, compared to, for example, a case in which the correction parameters are changed after the image is displayed.

[0107] Next, reference is made to Fig. 7. Fig. 7(A) and (B) are diagrams showing another example of display control when the display device starts operating.

[0108] In FIG. 7(A), when the ignition switch (IGN) of the vehicle 1 is turned on and the operation of each display device (HUD device) 110a to 110c starts, the control unit 160 of the display device (HUD device) 110a, which has the driver 15 as the primary viewer, corrects the image data using correction parameters (primary parameters M-wp) suitable for the driver 15 and starts displaying the image.

[0109] On the other hand, the control units 160 of the display devices 110b and 110c, whose main viewers are the occupants (passengers) 17 and 19, do not display images until the correction parameters are determined.

[0110] In Fig. 7(B), the control unit 160 of the display devices 110b and 110c detects the presence or absence and location of occupants, determines correction parameters, and then displays images corrected using the determined correction parameters. The correction parameters used to display each image in Fig. 7(B) are the same as those in the examples of Fig. 5(B) and Fig. 6(B).

[0111] In this way, for a display device whose primary viewer is the driver, it is possible to immediately start displaying an image, thereby presenting, for example, information useful for driving a vehicle without delay, etc. On the other hand, for a display device whose primary viewer is a passenger, it is possible to display an image with less distortion by starting the display after the presence or absence of the passenger is confirmed and the value of the correction parameter is determined.

[0112] Next, reference will be made to Fig. 8. Fig. 8(A) and (B) are diagrams showing still another example of display control when the display device starts operating.

[0113] In FIG. 8(A), when the ignition switch (IGN) of the vehicle 1 is turned on and the operation of each display device (HUD device) 110a to 110c starts, the control unit 160 of the display device 110a, which has the driver 15 as the primary viewer, corrects the image data using a correction parameter (primary parameter M-wp: first value of the correction parameter) suitable for the driver 15 and starts displaying the image.

[0114] In addition, the control unit 160 of the display devices 110b, 110c, which have the occupants 17, 19 as the primary viewers, also first corrects the image data using correction parameters (primary parameter M-wp: first value of the correction parameter) suitable for the occupants (passengers) 17, 19, and then begins displaying the image.

[0115] In FIG. 8(B), the control unit 160 of the display devices 110b, 110c, whose primary viewers are the occupants 17, 19, determines whether or not the correction parameters need to be changed based on the presence or absence of an occupant, occupant position information, etc., while causing the above display to be performed, and if a change is required, changes the value of the correction parameter, corrects the image data using the changed correction parameter, and displays the image.

[0116] In Fig. 8(B), since the passenger (fellow passenger) 17 in the center passenger seat is absent, the correction parameter for the display device 110b is changed from M-wp (first value of the correction parameter) to S-wp (second value of the correction parameter). The correction parameters used to display each image in Fig. 8(B) are the same as those in the examples of Fig. 5(B), Fig. 6(B), and Fig. 7(B).

[0117] When the display control in FIG. 8 is adopted, even when the occupant is the primary viewer, the image can be displayed quickly without waiting for the correction parameter value to be determined, thereby suppressing delays in image display.

[0118] (Third embodiment) Please refer to Fig. 9. Fig. 9(A) is a diagram showing an example in which one of three display devices is used to display an image to a passenger sitting in the back seat, and Fig. 9(B) is a diagram showing an example in which two of the three display devices located near the driver are used to display an image to the driver. In Fig. 9(A) and (B), the same reference numerals are used to designate parts common to Fig. 1.

[0119] In FIG. 1, two display devices (here, HUD devices) 100a and 100b are arranged in a passenger vehicle with two seats in the front row, but in the example of FIG. 9, a display device 100d is added between the driver's seat 6 and the auxiliary seat 9.

[0120] In the above-mentioned embodiment, for ease of explanation, the primary viewers are the driver and passengers seated in the front row of the vehicle 1. However, in the example of FIG. 9(A), the display device 100d displays an image for the primary viewer, that is, a passenger 22 seated, for example, in a central position of the rear seat 11 (or, more broadly, a position where the display by the display device 100d can be seen).

[0121] In other words, when an occupant is considered to be the primary viewer, that occupant may be a passenger sitting in the front passenger seat of vehicle 1, or a passenger sitting in the rear seat (rear seat) of vehicle 1.

[0122] The passengers seated in the rear seats are included in the passengers who are the primary viewers because there may be cases in which information such as navigation information, device operation information, and entertainment information is provided to these passengers as well.

[0123] The display device 100d is installed inside the dashboard or on the surface of the dashboard, in front of the occupant (occupant 22 in Figure 9(A)), preferably in a position directly facing the occupant (however, the term directly facing is interpreted broadly to include positions other than strictly directly in front of the occupant).

[0124] 9(B), the passenger 22 seated in the back seat is absent. In this case, the correction parameters of the image displayed by the display device 100d are changed so that the image is suitable for viewing by the driver 7. This reduces image distortion and suppresses the sense of incongruity felt by the driver 7.

[0125] 9(B), when the passenger 10 in the front passenger seat 9 is also absent, the image displayed by the display device 100b is seen by the driver 7. Therefore, the correction parameters of the image displayed by the display device 100b are changed so that the image is suitable for viewing by the driver 7. This reduces image distortion and suppresses the sense of discomfort felt by the driver 7.

[0126] (Fourth embodiment) Please refer to Fig. 10. Fig. 10(A) is a diagram showing an example of a display system using a display device (such as a liquid crystal display device), and Fig. 10(B) is a diagram showing an example of a display system in which a HUD device and a display device are mixed.

[0127] As described above, as the display device, in addition to the HUD device, a display device that displays a real image on a display unit as a display panel and allows at least one of the driver and passengers to view the real image can be used. As the display device, a liquid crystal device (e.g., a TFT liquid crystal device) or a micro LED display device, which is a self-luminous display device, can be used. By applying the present invention to the display device, a high-performance display device that can display an image with little distortion to both the driver and passengers can be realized.

[0128] In FIG. 10(A), the HUD devices 100a, 100d, and 100b in the configuration of FIG. 9(A) described above are replaced with display devices 300a, 300b, and 300c.

[0129] In FIG. 10B, the display device 100a in FIG. 10A is replaced with a HUD device 100a to construct a display system in which a head-up display (HUD) device and a display device are mixed.

[0130] In this way, in the present invention, a display system can be freely constructed using only a HUD device, only a display device, or a combination of a HUD device and a display device, making it possible to construct a user-friendly display system.

[0131] Fifth embodiment Next, reference will be made to Fig. 11. Fig. 11 is a diagram showing an example in which the correction parameter value is gradually changed when the correction parameter is changed.

[0132] When changing the value of the correction parameter during display of an image, the control unit 160 of the display device may gradually change the value from the value before the change to the value after the change. By gradually changing the value of the correction parameter when it becomes necessary to change the parameter, a sudden change in the appearance of the image can be suppressed, and thus the sense of discomfort felt by the driver and passengers can be reduced.

[0133] 11, the value of the correction parameter gradually changes from a first value N1 to a second value N2 over the period from time t1 to time t2. A characteristic line Q1 indicates a linear change.

[0134] The characteristic line Q2 changes gradually near time t1, and therefore, it is possible to suppress abrupt changes in the appearance of the image. As it approaches time t2, the change becomes more abrupt, and therefore it is possible to suppress a delay in switching the correction parameter value.

[0135] In the characteristic line Q3, the change is gradual near time t1 and near time t2, and abrupt changes in the appearance of the image are suppressed. On the other hand, the change is abrupt except near time t1 and near time t2, and this makes it possible to suppress delays in switching the correction parameter values.

[0136] Sixth embodiment Next, reference is made to FIG. 12. FIG. 12 is a diagram showing an example of the essential configuration of a display device and an example of a display system. The essential configuration of the display device has been shown previously in FIG. 2 and FIG. 3, but FIG. 12 shows a more detailed configuration. In addition, in the example of FIG. 12, three HUD devices are used as the display device. In addition, the example of FIG. 12 shows a configuration that can handle both warping correction and deformation suppression correction.

[0137] 12, the display system includes a communication unit 110, an ECU (electronic control unit) 120, an in-vehicle image capturing camera 200, an occupant (passenger) detection unit 130 that detects an occupant (or passengers including the driver), and first to third HUD devices 100a, 100b, and 100c. Each HUD device has the same configuration. The main configuration of the third HUD device 100c will be described below.

[0138] The third HUD device 100c has a display unit 101, a display control unit 145, an image generation unit 150 (including an image data correction unit 155), at least one processor (display control device) 159 (including a control unit 160, which has a projective transformation correction control unit 161 and a warping correction control unit 163), a database 170 in which navigation data, display image data, etc. are stored, and a ROM 210 as a storage device.

[0139] The ROM 210 has an image transformation table 212. The image transformation table 212 has a projective transformation matrix JM. The projective transformation matrix JM includes primary transformation parameters M-jm and secondary transformation parameters S-jm.

[0140] Furthermore, the image conversion table 212 has warping parameters WP, which include a main warping parameter (first value) M-wp and a secondary warping parameter (second value) S-wp.

[0141] The processor (display control device) 159 (control unit 160) has a function of appropriately changing the transformation parameters and warping parameters of the projective transformation. The image data correction unit 155 corrects the image data of the display image using the correction parameters (at least one of the transformation parameters and the warping parameters) so as to suppress distortion of the image as seen by the driver or passengers. The display unit 101 displays an image based on the corrected image data.

[0142] According to this configuration, when no passengers (fellow passengers) are present, an image with distortion correction suitable for the driver viewing the image from an oblique angle can be displayed on the display unit 101, thereby reducing the sense of discomfort. This makes it possible to realize a high-performance display device and display system capable of displaying images with little distortion to both the driver and passengers.

[0143] Seventh embodiment Next, reference is made to Fig. 13. Fig. 13 is a flowchart showing an example of a display control procedure.

[0144] In step S1, the presence or absence and location of passengers (or passengers including the driver) are detected.

[0145] In step S2, image correction parameters (projection transformation parameters, warping parameters, etc.) for each display content are determined based on the detection results (at least one of the presence or absence of an occupant and their location), and if a change is required, the parameters are changed (parameter determination and change processing).

[0146] In step S3, the image data of each display content is corrected using the determined (or changed) image correction parameters (image data correction process).

[0147] In step S4, at least one of displaying a virtual image by the HUD device and displaying a real image by a display (TFT liquid crystal, micro LED, etc.) is executed (image display process).

[0148] As described above, according to the present invention, even when the occupant who is the primary viewer of the display device is absent, image distortion is reduced, and, for example, the discomfort felt by the driver viewing the image is suppressed.

[0149] The present invention can be used in both monocular HUD devices, in which display light for the same image is incident on each of the left and right eyes, and parallax HUD devices, in which images with parallax are incident on each of the left and right eyes.

[0150] Furthermore, the display device may be a liquid crystal display device, a micro LED device, or any other type of display device.

[0151] In this specification, the term "vehicle" may be broadly interpreted as a vehicle. Terms related to navigation (e.g., signs, etc.) are also broadly interpreted, taking into consideration the perspective of navigation information in a broad sense that is useful for driving a vehicle. HUD devices also include those used as simulators (e.g., aircraft simulators, simulators as game devices, etc.).

[0152] The present invention is not limited to the above-described exemplary embodiments, and those skilled in the art could easily modify the above-described exemplary embodiments to the extent that they fall within the scope of the claims. [Explanation of symbols]

[0153] 1 Vehicle (own vehicle), 2 Projected member (reflective / translucent member, windshield, etc.), 6···Driver's seat, 7···Driver, 8···Steering wheel, 9···Passenger's seat, 10····Occupant, 11···Rear seat, 100···HUD device (broadly speaking, display device), 101···Display unit (liquid crystal panel, etc.), 102···Image display surface of the display unit, 110···Communication unit, 120···ECU (electronic control unit), 130···Occupant (passenger) detection unit, 145···Display control unit, 150···Image generation unit, 155···Image data correction unit, 159...processor (display control device), 160...control device, 161...projection transformation correction control device, 163...warping correction control device, 170...database, 210...storage device (ROM), 200...in-vehicle imaging camera, 212...image conversion table, 300...display device, JM...projection transformation matrix, M-jm...main transformation parameter (first value), S-jm...sub-transformation parameter (second value), WP...warping parameter, M-wp...main warping parameter (first value), S-wp...sub-warping parameter (second value)

Claims

1. A display device that is mounted in a vehicle and allows a driver of the vehicle and passengers other than the driver to view an image, A control unit having a function of changing a correction parameter used when correcting image data of the image so as to suppress distortion of the image as seen by the driver or the passenger; an image data correction unit that corrects the image data using the correction parameters; a display unit that is disposed at a position corresponding to the occupant and displays the image based on the corrected image data, The control unit is setting a value of the correction parameter to a first value suitable for the occupant when the occupant is present as a result of determination based on at least one of the occupant presence information and the occupant location information; When the occupant is absent, the value of the correction parameter is set to a second value suitable for the driver and different from the first value. Display device.

2. The occupant: A passenger sitting in the front passenger seat of the vehicle. Or, A passenger sitting in a rear seat of the vehicle. The display device according to claim 1 .

3. The correction parameters are A warping parameter for pre-distortion correction that pre-applies to the image data a distortion with characteristics opposite to the distortion that occurs when the image is displayed; And, When the outline of the display area of ​​the image is a rectangle including a rectangle and a square, if the rectangle appears to be deformed into a trapezoid or a parallelogram in response to an angle at which a viewer views the image, a transformation parameter for deformation suppression and correction for transforming the trapezoid or parallelogram into a rectangle; And, In the case of three-dimensional (3D) display, a 3D image correction parameter for correcting distortion of the 3D image; At least one of The display device according to claim 1 .

4. The control unit is when an operation of the display device is started, the image is not displayed before the value of the correction parameter is determined, and the image is displayed after the value of the correction parameter is determined. The display device according to claim 1 .

5. The control unit is When the operation of the display device is started, If the occupant is present, the image is displayed after the value of the correction parameter is determined. The display device according to claim 1 .

6. The control unit is When the operation of the display device is started, determining whether or not the correction parameter needs to be changed while displaying the correction parameter set to the first value, and changing the value of the correction parameter if the correction parameter needs to be changed; The display device according to claim 1 .

7. The control unit is when changing the value of the correction parameter while the image is being displayed, the value is gradually changed from a value before the change to a value after the change. The display device according to claim 1 .

8. The display device includes: The head-up display device further includes an optical system that projects a display light of the image displayed on the display unit onto a projection member provided in the vehicle, and has a function of allowing at least one of the driver and the passenger to visually recognize a virtual image. Or, A display device that displays the real image on the display unit as a display panel and allows at least one of the driver and the passenger to visually recognize the real image. The display device according to claim 7.

Images