Operation support device

Abstract

This system allows the user to select the desired operating switch without forcing them to move their finger to the desired switch, even if the position of the user's finger contact is off from the desired operating switch. [Solution] The operation support device includes a contact detection unit 14 that detects the user's finger contact with a first display device 2 provided on the vehicle 1 and displaying a plurality of operation switches 21, and a display control unit 15 that controls the display of a second display device 3 positioned closer to the vehicle's windshield 5 than the first display device. When the display control unit detects finger contact with the first display device in an area other than the plurality of operation switches, it displays an image of the area around the finger contact location on the second display device along with the finger contact location, and suggests to the user the selection of the operation switch closest to the finger contact location from among the plurality of operation switches.

Description

Technical Field

[0001] The present invention relates to an operation support device.

Background Art

[0002] In Patent Document 1, in order to facilitate the operation of an operation unit on which a plurality of operation switches are arranged, an image of the hand of a user (a vehicle occupant) operating the operation unit is synthesized with an operation menu image and displayed on a display unit different from the operation unit.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the technique described in Patent Document 1, when the position of the user's finger is deviated from the desired operation switch on the operation unit, even if the vehicle is in operation, the user who is the driver of the vehicle needs to move the position of the finger to the desired operation switch while visually checking the display unit.

[0005] Therefore, in view of the above problems, an object of the present invention is to enable a user to select a desired operation switch without forcing the user to move the finger to the desired operation switch when the position of the finger contact by the user is deviated from the desired operation switch.

Means for Solving the Problems

[0006] The gist of the present disclosure is as follows.

[0007] (1) An operation support device comprising: a contact detection unit that detects a user's finger contact with a first display device provided in a vehicle and displaying a plurality of operation switches; and a display control unit that controls the display of a second display device positioned closer to the windshield of the vehicle than the first display device, wherein when the display control unit detects the finger contact with the first display device in an area other than the plurality of operation switches, it displays an image of the area around the finger contact position from the image displayed on the first display device on the second display device together with the finger contact position, and proposes to the user the selection of the operation switch closest to the finger contact position from among the plurality of operation switches.

[0008] (2) The operation support device according to (1) above, wherein the display control unit displays the surrounding image on the second display device at a larger size than when it is displayed on the first display device.

[0009] (3) The operation support device according to (1) or (2) above, wherein the display control unit performs an operation corresponding to the operation switch closest to the position of the finger contact when the finger contact continues at the same position on the first display device for a predetermined threshold time from the time the selection is proposed to the user.

[0010] (4) The operation support device according to any one of (1) to (3) above, wherein the second display device is a meter display that displays the status information of the vehicle, and the display control unit hides at least a portion of the status information of the vehicle when displaying the surrounding image on the second display device.

[0011] (5) The operation support device according to any one of (1) to (4) above, wherein the display control unit maintains the display content of the second display device for a predetermined time when the user's finger leaves the first display device while the surrounding image is displayed on the second display device. [Effects of the Invention]

[0012] According to the present invention, even if the position of the user's finger contact is deviated from the desired operating switch, the user can select the desired operating switch without forcing the user to move their finger to the desired operating switch. [Brief explanation of the drawing]

[0013] [Figure 1] This diagram schematically shows a part of the configuration of a vehicle equipped with an operation support device according to an embodiment of the present invention. [Figure 2] This diagram schematically shows the interior of the vehicle in front of the driver's and passenger's seats. [Figure 3] This figure shows an example of the screens of the first and second display devices when a portion of the image on the first display device is projected onto the second display device. [Figure 4] This figure shows an example of a method for suggesting the selection of an operating switch to the user. [Figure 5] This flowchart shows the control routine for operation support processing in this embodiment. [Modes for carrying out the invention]

[0014] Embodiments of the present invention will be described in detail below with reference to the drawings. In the following description, similar components will be given the same reference numerals.

[0015] Figure 1 is a schematic diagram showing a part of the configuration of a vehicle 1 equipped with an operation support device according to an embodiment of the present invention. In this embodiment, vehicle 1 is a four-wheeled automobile.

[0016] As shown in Figure 1, the vehicle 1 is equipped with a first display device 2, a second display device 3, and an electronic control unit (ECU) 10. The first display device 2 and the second display device 3 are electrically connected to the ECU 10 via an in-vehicle network compliant with standards such as CAN (Controller Area Network) or Ethernet.

[0017] The first display device 2 is provided in the vehicle 1 and displays various information to the driver of the vehicle 1. The first display device 2 is provided in the vehicle interior so as to be visible to the driver of the vehicle 1. FIG. 2 is a diagram schematically showing the interior of the vehicle 1 in front of the driver's seat and the passenger seat. In the present embodiment, as shown in FIG. 2, the first display device 2 is incorporated in a portion between the driver's seat and the passenger seat of the dashboard 4, that is, in the center console. In this case, the first display device 2 is also referred to as a center display.

[0018] In the present embodiment, the first display device 2 displays multimedia information, map information, etc. That is, the first display device 2 is an infotainment display and can be operated by the driver of the vehicle 1. As shown in FIG. 2, the first display device 2 displays a plurality of operation switches 21 for starting multimedia applications (such as a television (TV), radio (AM, FM), music content, video content, etc.). That is, a plurality of switches 21 are arranged on the first display device 2. When the driver of the vehicle 1 requests to start a specific multimedia application, the driver selects the operation switch 21 corresponding to that multimedia application with a finger. The first display device 2 is configured as a touch screen, for example.

[0019] The first display device 2 transmits input data input to the first display device 2 by the driver of the vehicle 1 to the ECU 10. On the other hand, the first display device 2 displays information corresponding to the signal transmitted from the ECU 10 to the driver of the vehicle 1.

[0020] The second display device 3 is provided in the vehicle 1 and displays various information to the driver of the vehicle 1. The second display device 3 is provided in the vehicle interior so as to be visible to the driver of the vehicle 1. The second display device 3 is arranged at a position closer to the windshield 5 of the vehicle 1 than the first display device 2. That is, the second display device 3 is arranged at a position where it is more easily visible to the driver of the vehicle 1 during driving of the vehicle 1 than the first display device 2. In the example shown in FIG. 2, the second display device 3 is incorporated as an instrument panel in the dashboard 4 in front of the steering wheel 6, that is, in the dashboard 4 in front of the driver's seat.

[0021] In this embodiment, the second display device 3 displays the state information of the vehicle 1, specifically, information necessary for driving the vehicle 1 such as vehicle speed, engine speed, remaining fuel, SOC (State Of Charge) of the battery, and warning lights. That is, the second display device 3 is a meter display, and is configured as, for example, a liquid crystal display (LCD: Liquid Crystal Display) or an organic EL (Electro Luminescence) display. The second display device 3 displays information corresponding to the signal transmitted from the ECU 10 to the driver of the vehicle 1. In this embodiment, the second display device 3 cannot be operated by the driver of the vehicle 1.

[0022] The ECU 10 executes various controls of the vehicle 1. As shown in FIG. 1, the ECU 10 includes a communication interface 11, a memory 12, and a processor 13. The communication interface 11 and the memory 12 are connected to the processor 13 via signal lines. In this embodiment, one ECU 10 is provided, but a plurality of ECUs may be provided for each function. Further, the communication interface 11, the memory 12, and the processor 13 may be configured as one integrated circuit, or may be configured as separate circuits, respectively.

[0023] The communication interface 11 has an interface circuit for connecting the ECU 10 to the in-vehicle network. The ECU 10 is connected to other in-vehicle devices via the communication interface 11. The communication interface 11 transmits the signal received from the first display device 2 to the processor 13. Further, the communication interface 11 transmits the signal output from the processor 13 to the first display device 2 and the second display device 3.

[0024] Memory 12 includes, for example, volatile semiconductor memory (e.g., DRAM (Dynamic Random Access Memory), SRAM (Static Random Access Memory), etc.) and non-volatile semiconductor memory (e.g., ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), flash memory, etc.). Memory 12 stores temporary data, computer programs used for various processes by the processor 13 (control programs for the ECU 10), ECU 10 setting data, log data, vehicle information, etc.

[0025] The processor 13 has one or more CPUs (Central Processing Units) and their peripheral circuits. The processor 13 executes computer programs stored in the memory 12. The processor 13 may also have other arithmetic circuits such as a logical operation unit, a numerical operation unit, or a graphics processing unit.

[0026] In this embodiment, the ECU 10 functions as an operation support device that assists the driver of the vehicle 1 in operating the display. In particular, in this embodiment, when the driver of the vehicle 1 is driving the vehicle 1 and is attempting to select a specific operation switch 21 on the first display device 2, the ECU 10 assists the driver in operating the display by projecting the image displayed on the first display device 2 onto the second display device 3, which has higher visibility than the first display device 2. Note that the ECU 10 is an example of an operation support device.

[0027] As shown in Figure 1, the processor 13 of the ECU 10 has a contact detection unit 14 and a display control unit 15. The contact detection unit 14 and the display control unit 15 are functional modules that are realized by the execution of a computer program stored in the memory 12 of the ECU 10 by the processor 13 of the ECU 10. These functional modules may also be realized by dedicated arithmetic circuits provided in the processor 13.

[0028] The driver of vehicle 1, who is the user of the first display device 2, basically operates the first display device 2 with their fingers. In response, the contact detection unit 14 detects the user's finger contact with the first display device 2. In other words, the contact detection unit 14 detects when the user's finger comes into contact with the first display device 2. For example, the first display device 2 has a large number of electrodes arranged in a grid, and the contact detection unit 14 detects the user's finger contact by detecting the change in capacitance that occurs when the user's finger approaches an electrode. Furthermore, the contact detection unit 14 identifies the location of the user's finger contact on the first display device 2 by identifying the electrode where the change in capacitance occurred.

[0029] The display control unit 15 controls the display on the second display device 3. When the second display device 3 is displaying status information of the vehicle 1, the display control unit 15 updates the status information of the vehicle 1 displayed on the second display device 3 in accordance with changes in the status of the vehicle 1. For example, when the speed of the vehicle 1 changes, the display control unit 15 changes the value of the vehicle speed displayed on the second display device 3.

[0030] When the contact detection unit 14 detects that a user's finger has touched any one of the operation switches 21, it executes the operation corresponding to that operation switch 21. In this embodiment, in this case, the contact detection unit 14 launches the multimedia application corresponding to the operation switch 21 that the user's finger has touched.

[0031] However, the driver of vehicle 1 cannot always accurately select the desired operation switch 21 on the screen of the first display device 2 with their finger. In particular, while driving vehicle 1, the driver of vehicle 1 needs to operate the first display device 2 while being aware of the surrounding situation of vehicle 1, making it difficult to touch the desired operation switch 21 with their finger.

[0032] Therefore, in this embodiment, when the display control unit 15 detects a user's finger contact with the first display device 2 in an area other than the multiple operation switches 21, it displays the image surrounding the finger contact location, along with the finger contact location, on the second display device 3. This makes it possible for the user to visually confirm any misalignment of the finger contact location via the easily visible second display device 3, even while the vehicle 1 is in operation.

[0033] Furthermore, even if the user's finger position deviates from the desired operation switch 21, the user's finger contact with the first display device 2 is highly likely to occur near the desired operation switch 21. For this reason, the display control unit 15 suggests to the user the operation switch 21 closest to the position of the user's finger contact. This allows the user to select the desired operation switch 21 without forcing the user to move their finger to the desired operation switch 21, even if the position of the user's finger contact deviates from the desired operation switch 21.

[0034] Figure 3 shows an example of the screens of the first display device 2 and the second display device 3 when a portion of the image from the first display device 2 is projected onto the second display device 3. In the example shown in Figure 3, the first display device 2 displays a first operation switch 21a, a second operation switch 21b, and a third operation switch 21c. The first operation switch 21a corresponds to the FM radio multimedia application, the second operation switch 21b corresponds to the AM radio multimedia application, and the third operation switch 21c corresponds to the TV multimedia application. That is, when the first operation switch 21a is selected, the FM radio is activated; when the second operation switch 21b is selected, the AM radio is activated; and when the third operation switch 21c is selected, the TV is activated.

[0035] In the example shown in Figure 3, the user's finger is in contact with the first display device 2 near the third operation switch 21c. In this case, the dashed rectangle represents the area around the finger contact point, and the image SI of the area around the finger contact point is displayed on the second display device 3 along with the finger contact point FP. Note that the area defined as the area around the finger contact point may be of a different size than in the example shown in Figure 3, or it may be a circular area, as long as it includes the finger contact point and at least one of the operation switches 21 in the vicinity of the finger contact point.

[0036] Furthermore, when the display control unit 15 displays an image of the area around the finger contact point on the second display device 3, it hides at least a portion of the vehicle status information of the vehicle 1 that was displayed on the second display device 3. This improves the visibility when a portion of the image on the first display device 2 is displayed on the second display device 3. For example, when the display control unit 15 displays an image of the area around the finger contact point on the second display device 3, it hides the vehicle speed information that was displayed on the second display device 3.

[0037] Furthermore, as shown in Figure 3, the display control unit 15 displays the image surrounding the finger contact point on the second display device 3 at a larger size than when it is displayed on the first display device 2. This makes it easier for the user to recognize the relative positional relationship between the user's finger and the desired operation switch 21 via the second display device 3.

[0038] As described above, the display control unit 15 suggests to the user the selection of the operation switch 21 that is closest to the user's finger contact position among the multiple switches 21 of the second display device 3. In the example in Figure 3, since the third operation switch 21c is closest to the finger contact position FP, the display control unit 15 suggests to the user the selection of the third operation switch 21c.

[0039] Figure 4 shows an example of how the user is prompted to select an operation switch. In the example in Figure 4, the display control unit 15 prompts the user to select the third operation switch 21c by additionally displaying a speech bubble icon BI on the second display device 3. The speech bubble contains the message, "Do you want to select TV?". The display control unit 15 may also prompt the user to select the third operation switch 21c in a manner different from the example in Figure 4. For example, the display control unit 15 may prompt the user to select the third operation switch 21c by emphasizing the display of the third operation switch 21c on the second display device 3 more than the displays of other operation switches (second operation switch 21b). Alternatively, the display control unit 15 may prompt the user to select the third operation switch 21c by voice through a speaker provided in the vehicle 1.

[0040] The display control unit 15 determines that the user has approved the proposal if the user's finger continues to make contact with the same position on the first display device 2 for a predetermined threshold time from the time it proposes the selection of the operation switch closest to the position of the finger contact, and then executes the operation corresponding to that operation switch. In the example shown in Figures 3 and 4, the display control unit 15 determines that the user has approved the proposal if the user's finger continues to make contact with the same position on the first display device 2 for a threshold time from the time it proposes the selection of the third operation switch 21c, and then starts the TV.

[0041] The following describes the processing flow for executing the control described above, with reference to Figure 5. Figure 5 is a flowchart of the control routine related to the operation support processing in this embodiment. This control routine is repeatedly executed by the processor 13 of the ECU 10, for example, according to a computer program stored in the memory 12 of the ECU 10.

[0042] First, in step S101, the display control unit 15 of the processor 13 determines whether or not the contact detection unit 14 of the processor 13 has detected a user's finger contact with the first display device 2. If it is determined that no user's finger contact has been detected, this control routine terminates. On the other hand, if it is determined that a user's finger contact has been detected, this control routine proceeds to step S102.

[0043] In step S102, the display control unit 15 determines whether the finger contact position is within the area of ​​the operation switch 21 on the first display device 2. That is, the display control unit 15 determines whether the user's finger is in contact with one of the operation switches 21 on the first display device 2.

[0044] If it is determined in step S102 that the finger contact position is within the area of ​​the operation switch 21, the control routine proceeds to step S103. In step S103, the display control unit 15 executes the operation corresponding to the operation switch 21 where the user's finger is located. After step S103, the control routine terminates.

[0045] On the other hand, if it is determined in step S102 that the finger contact position is outside the area of ​​the operation switch 21, the control routine proceeds to step S104. In step S104, the display control unit 15 displays the image surrounding the user's finger contact position, along with the finger contact position, on the second display device 3 from the image displayed on the first display device 2.

[0046] Next, in step S105, the display control unit 15 determines whether the user's finger has left the first display device 2. That is, the display control unit 15 determines whether the user's finger contact with the first display device 2 has been released (resolved). If it is determined that the user's finger has not left the first display device 2, the control routine proceeds to step S106.

[0047] In step S106, the display control unit 15 proposes to the user the selection of the operation switch 21 closest to the user's finger contact location, in the manner described above. Next, in step S107, the display control unit 15 determines whether the user's finger contact at the same location on the first display device 2 continued for a threshold time. The threshold time is predetermined and, for example, is a value within the range of 2 to 5 seconds.

[0048] If it is determined in step S107 that the user's finger contact has not continued for the threshold time, the control routine returns to step S105. If the determination in step S105 is again rejected, the suggestion to the user continues in step S106.

[0049] On the other hand, if it is determined in step S107 that the user's finger contact has continued for a threshold time, the control routine proceeds to step S108. In step S108, the display control unit 15 determines that the user has approved the proposal and executes the operation corresponding to the operation switch 21 proposed to the user.

[0050] After step S108, the control routine proceeds to step S109. In step S109, the display control unit 15 returns the display on the second display device 3 to the default display. That is, the display control unit 15 erases the image from the first display device 2 from the second display device 3 and displays the status information of the vehicle 1 on the second display device 3. After step S109, the control routine terminates.

[0051] Furthermore, if it is determined in step S105 that the user's finger has left the first display device 2, the control routine proceeds to step S110. In step S110, the display control unit 15 maintains the display content of the second display device 3 for a predetermined time. The predetermined time is set in advance and is, for example, within the range of 2 to 5 seconds.

[0052] After step S110, the control routine proceeds to step S109. As described above, in step S109, the display control unit 15 returns the display of the second display device 3 to the default display. After step S109, the control routine terminates.

[0053] As can be seen from the flowchart above, the display control unit 15 maintains the display content of the second display device 3 for a predetermined time when the user's finger leaves the first display device 2 while an image of the area around the finger contact position is displayed on the second display device 3. This makes it easier for the user to move their finger to the desired position when continuing to operate the first display device 2.

[0054] Furthermore, when the position of finger contact changes while the image surrounding the finger contact position is displayed on the second display device 3, the display control unit 15 changes the range of the image displayed on the first display device 2 that is displayed on the second display device 3 according to the change in the finger contact position. This allows the user to recognize the movement of their finger on the second display device 3 in real time, and consequently makes it easier for the user to move their finger to the desired position.

[0055] Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes can be made within the scope of the claims. For example, the first display device 2 may be located in the center console, immediately to the left of the steering wheel 6, or in other locations, as long as it is operable by the driver of the vehicle 1.

[0056] Furthermore, the second display device 3 may be a meter display located in the center of the vehicle 1 above the center cluster, a head-up display located on the windshield 5 in front of the driver's seat, etc., as long as it is positioned closer to the windshield 5 of the vehicle 1 than the first display device 2.

[0057] Furthermore, the multiple switches 21 displayed on the first display device 2 may be switches for functions other than launching multimedia applications, such as setting the audio volume or setting a destination. The user (driver of vehicle 1) may also approve or reject the selection by voice when the operation switch 21 closest to the user's finger contact position is suggested.

[0058] Furthermore, the computer program that enables a computer to implement the functions of each part of the processor 13 of the ECU 10 may be provided in the form of a recording medium readable by a computer, or in the form of a computer program product. The recording medium readable by a computer may be, for example, a magnetic recording medium, an optical recording medium, or a semiconductor memory. [Explanation of symbols]

[0059] 1 vehicle 2 1st display device 3 2nd display device 5. Windshield 14 Contact detection unit 15 Display Control Unit 21 Operation switches

Claims

1. A contact detection unit that detects user finger contact with a first display device provided in the vehicle and displaying multiple operation switches, A display control unit that controls the display of a second display device, which is positioned closer to the vehicle's windshield than the first display device, and Equipped with, The display control unit, when it detects finger contact with the first display device in an area other than the plurality of operation switches, displays an image of the area surrounding the finger contact location from the image displayed on the first display device, along with the finger contact location, on the second display device, and proposes to the user the selection of the operation switch closest to the finger contact location from among the plurality of operation switches, as an operation support device.

2. The operation support device according to claim 1, wherein the display control unit displays the surrounding image on the second display device at a larger size than when it is displayed on the first display device.

3. The operation support device according to claim 1 or 2, wherein the display control unit performs an operation corresponding to the operation switch closest to the position of the finger contact when the finger contact continues at the same position on the first display device for a predetermined threshold time from the time the selection is proposed to the user.

4. The second display device is a meter display that displays the status information of the vehicle, The operation support device according to claim 1 or 2, wherein the display control unit hides at least a portion of the vehicle status information when displaying the surrounding image on the second display device.

5. The operation support device according to claim 1 or 2, wherein the display control unit maintains the display content of the second display device for a predetermined time when the user's finger leaves the first display device while the surrounding image is displayed on the second display device.

Images