VEHICLE DISPLAY DEVICE

Abstract

Vehicle display device with a display section (30), a pointing device (10) which includes an operating section (11) and a detection section (12) which detects an operating finger operating the operating section (11), and a display control section (20) for displaying an operation screen (40; 50; 60; 70) on the display section (30), the operation screen containing a plurality of selection targets (41, 42, 43, 44, 45; 51 - 55; 61 - 67; 71A - 71Z) whose selection operation can be performed using the pointing device (10), wherein the display control section (20) is arranged to Generating an image in which a cursor (46) is moved to select a selection target from the plurality of selection targets (41, 42, 43, 44, 45; 51-55; 61-67; 71A-71Z), and / or an image in which the plurality of selection targets (41, 42, 43, 44, 45; 51-55; 61-67; 71A-71Z) arranged along a specific axis on the operation screen (40; 50; 60; 70) are scrolled on the operation section (11) in accordance with a position of the operation finger detected by the detection section (12), Generating an image containing an object or group of objects (47; 57; 69; 77) over a specific area on the operating screen (40; 50; 60; 70), and when the operating finger is detected by the detection section (12), executing processing of changing a shape of a part (47a, 47b; 57a, 57b; 69a, 69b; 77a, 77b) of the object or object group (47; 57; 69; 77) with respect to the shape of the part (47a, 47b; 57a, 57b; 69a, 69b; 77a, 77b) of the object or object group (47; 57; 69; 77) in a case where the operating finger is not detected by the detection section (12).

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention

[0001] The invention relates to a vehicle display device that displays an operation screen. 2. Description of related technology

[0002] A vehicle display device is known (see, for example, JP 2004-345549 A) which displays a screen (an operation screen) enabling various operations of a device in the vehicle by using a pointing device which detects a position of a user's finger (hereinafter referred to as an "operation finger") in an operation section.

[0003] JP 2004-345549 A discloses a method for detecting the position of the operating finger in the operating section and moving a cursor that selects an icon on the operating screen based on position information of the operating finger when the user touches the operating section of the pointing device and performs a sliding operation of a certain extent or more.

[0004] Furthermore, a method for scrolling a plurality of selection targets (for example, a list) on the operation screen according to a movement of the operation finger in the operation section of the pointing device is known.

[0005] US 2016 / 0 062 507 A1 discloses an input device separate from a display unit for switchably displaying images arranged hierarchically in hierarchy levels and configured to input a user's finger operation on an operating surface for operating the images. The input device comprises a detection device for detecting an operating state of the finger to the operating surface at a time of the finger operation and a control device for reducing the sharpness of at least a portion of the image displayed on the display unit upon determining that the operating state is a proximity state, compared to determining that the operating state is a contact state. SUMMARY OF THE INVENTION

[0006] However, if the operating finger leaves the operating section even by the slightest amount due to vibration or the like of a vehicle, there is a possibility that the pointing device can no longer recognize the operating finger. As long as the user moves the operating finger, the user cannot determine whether the pointing device recognizes the operating finger and can thus operate the operation screen, or whether the pointing device does not recognize the operating finger and can thus not operate the operation screen. In other words, as long as the user checks whether the cursor can be moved or whether the list can be scrolled, etc., the user cannot determine whether the pointing device can operate the operation screen.

[0007] The invention provides a vehicle display device as specified in the appended claims, which enables a user to easily determine whether a pointing device can operate an operation screen.

[0008] A vehicle display device includes a display section, a pointing device including an operation section and a detection section that detects an operation finger operating the operation section, and a display control section for displaying an operation screen on the display section, the operation screen including a plurality of selection targets whose selection operation can be performed using the pointing device.The display control section is configured to generate an image in which a cursor is moved to select a selection target from the plurality of selection targets and / or an image in which the plurality of selection targets arranged along a specific axis on the operation screen are scrolled on the operation section according to a position of the operation finger detected by the detection section, generate an image including an object or an object group over a specific area of the operation screen, and execute processing of setting a part of the object or the object group to a different display mode from the other part of the object or the object group when the operation finger is detected by the detection section.

[0009] In the case where the pointing device (the detection section) detects the operation finger according to the above configuration, the part of the object or object group displayed on the operation screen is in the display mode different from its other part (for example, color, luminance, shape, size, displacement, or the like are different). Accordingly, the user visually recognizes the presence or absence of a partial change in the display mode of the object or object group on the operation screen, and thus can easily determine whether the operation screen can be operated by the pointing device.

[0010] Furthermore, in the vehicle display device according to the above embodiment, the display control section may be configured to generate an image in which the part of the object and the object group in the different display mode is moved along a moving direction of the operation finger when the operation finger detected by the detection section is moved with respect to the operation section.

[0011] In the case where the operation finger detected by the pointing device (the detection section) is moved according to the above configuration, the part of the object or object group in the different display mode is moved along the moving direction of the operation finger. For example, there is a case where the operation finger may leave the operation screen due to vibration of a vehicle or the like while the operation finger is being moved. In this case, the user visually recognizes whether the part in the different display mode is moved along the moving direction of the operation finger, and thus can easily determine whether a moving operation of the cursor or a scrolling operation of a list or the like by the pointing device can be appropriately continued.

[0012] In the vehicle display device according to the above aspect, the display control section may further be configured to generate the operation screen by composing a foreground image including the plurality of selection targets and a background image including the object or object group.

[0013] According to the above configuration, the object or object group is displayed as the background of the plurality of selection targets (a plurality of icons or the list configured by including a plurality of icons) or the like displayed as the foreground. Thus, it is possible to suppress the occurrence of a situation in which the visibility of the plurality of selection targets, the cursor indicating a selection status of the plurality of selection targets, or the like deteriorates due to a change in the display mode of the object group or the like.

[0014] In the vehicle display device according to the above aspect, the display control section may be configured to determine the part of the object or the object group based on a position on the operation screen, the position corresponding to a state signal from the pointing device.

[0015] In the vehicle display device according to the above aspect, the plurality of selection targets may be arranged in a row in a first direction on the operation screen, and the object or object group may be arranged along the first direction. The display control section may be configured to generate an image in which the part of the object or object group is offset from the other part of the object or object group in a second direction crossing the first direction when the operation finger is detected by the detection section.

[0016] In the vehicle display device according to the above aspect, the part of the object group may include a plurality of grains, and a displacement amount of the plurality of grains in the second direction may be reduced when the grains separate from a position on the operation screen in the first direction, the position corresponding to a state signal from the pointing device.

[0017] In the vehicle display device according to the above aspect, the object or the object group may be arranged in a virtual three-dimensional space, and the display control section may be configured to generate an image in which the part of the object or the object group is highlighted in a certain direction in the virtual three-dimensional space with respect to the other part of the object or the object group when the operating finger is detected by the detection section.

[0018] The invention can provide the vehicle display device that allows the user to easily determine whether the operation screen can be operated by the pointing device. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Features, advantages and technical and industrial significance of embodiments of the invention will now be described with reference to the accompanying drawings in which like reference numerals designate like elements. Fig. 1 is a block diagram schematically showing an example of a configuration of a display device, Fig. 2 is a diagram showing an example of an in-vehicle mode of the display device (a pointing device and a display device), Fig. 3A shows a diagram of an example of a foreground image of a display device according to a first embodiment, Fig. 3B shows a diagram of an example of a background image of the display device according to the first embodiment, Fig. 3C shows a diagram of an example of an operation screen of the display device according to the first embodiment, Fig. 4A is a diagram showing one example of the foreground image of the display device according to the first embodiment, Fig. 4B shows a diagram of one example of the background image of the display device according to the first embodiment, Fig. 4C shows a diagram of one example of the operation screen of the display device according to the first embodiment, Fig. 5A is a diagram showing one example of the foreground image of the display device according to the first embodiment, Fig. 5B shows a diagram of one example of the background image of the display device according to the first embodiment, Fig. 5C shows a diagram of one example of the operation screen of the display device according to the first embodiment, Fig. 6A is a diagram showing another example of the foreground image of the display device according to the first embodiment, Fig. 6B shows a diagram of another example of the background image of the display device according to the first embodiment, Fig. 6C shows a diagram of another example of the operation screen of the display device according to the first embodiment, Fig. 7A is a diagram showing another example of the foreground image of the display device according to the first embodiment, Fig. 7B is a diagram showing another example of the background image of the display device according to the first embodiment; Fig. 7C shows a diagram of another example of the operation screen of the display device according to the first embodiment, Fig. 8A is a diagram showing another example of the foreground image of the display device according to the first embodiment, Fig. 8B shows a diagram of another example of the background image of the display device according to the first embodiment, Fig. 8C shows a diagram of another example of the operation screen of the display device according to the first embodiment, Fig. 9A is a diagram showing yet another example of the operation screen of the display device according to the first embodiment, Fig. 9B is a diagram showing yet another example of the operation screen of the display device according to the first embodiment, Fig. 9C is a diagram showing yet another example of the operation screen of the display device according to the first embodiment, Fig. 10A is a diagram showing an example of a foreground image of a display device according to a second embodiment, Fig. 10B is a diagram showing an example of a background image of the display device according to the second embodiment, Fig. 10C is a diagram showing an example of an operation screen of the display device according to the second embodiment, Fig. 11A is a diagram showing one example of the foreground image of the display device according to the second embodiment, Fig. 11B is a diagram showing one example of the background image of the display device according to the second embodiment, Fig. 11C is a diagram showing one example of the operation screen of the display device according to the second embodiment, Fig. 12A is a diagram showing one example of the foreground image of the display device according to the second embodiment, Fig. 12B shows a diagram of one example of the background image of the display device according to the second embodiment, and Fig. 12C is a diagram showing one example of the operation screen of the display device according to the second embodiment, DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] Embodiments of the invention are described below with reference to the drawings.

[0021] [First embodiment] Fig. 1 is a block diagram schematically illustrating an example of a configuration of a display device 1 according to this embodiment. The display device 1 includes a pointing device 10, an electronic control unit (ECU) (a display control section) 20, and a display device (a display section) 30. The display device 1 is mounted on, for example, a vehicle, and displays a screen (an operation screen) including a plurality of selection targets (a plurality of icons or the like) on the display device 30, the selection operation of which can be performed using the pointing device 10. Hereinafter, a "vehicle" means the vehicle on which the display device 1 is mounted unless otherwise specified.

[0022] The pointing device 10 is an input device that determines a position on the operation screen and is, for example, an integrated touch panel. The pointing device 10 includes an operation section 11 and a recognition section 12.

[0023] The operation section 11 is a section of the pointing device 10 that is operated by a finger (an operation finger) of a user (for example, a passenger such as a driver of the vehicle). The operation section 11 is, for example, a touch operation screen of the integrated touch panel.

[0024] The detection section 12 detects the operating finger on the operation section 11. The detection section 12 is, for example, an electrostatic block of the integrated touch panel. The electrostatic block has, for example, a structure such that electrodes (an electrostatic sensor) extend linearly in an X direction and a Y direction on a plane, respectively, with an insulator therebetween, and outputs a detection signal of these electrodes (a signal corresponding to a change amount of electric charges stored in the electrodes). The detection section 12 sequentially (i.e., every certain cycle) outputs a signal related to a user operation state (for example, a detection signal output by the electrostatic block, hereinafter referred to as a "state signal") to the ECU 20.For example, the detection section 12 outputs the state signal corresponding to each coordinate (an x-coordinate and a y-coordinate) in a predetermined xy coordinate system of the operation section 11.

[0025] The pointing device 10 is not limited to the integrated touch panel or the like, as long as it includes the operation section 11 and the detection section 12 that detects the operation finger on the operation section 11. For example, the pointing device 10 may be a gesture recognition device that includes a predetermined operation space (an example of the operation section 11) and a processing device (an example of the detection section 12) that detects a position of the operation finger in the operation space based on an image of the operation space captured by a camera.

[0026] The pointing device 10 is provided at a suitable location in a vehicle cabin, and is preferably arranged at a position where the driver can easily operate the pointing device 10, particularly a position where the driver's hand can reach the pointing device 10 while maintaining a driving posture. As shown in Fig. 2 (a representation of an example of an in-vehicle mode of the display device 1), the pointing device 10 (in this example, the integrated touch pad) may be arranged, for example, in a console housing or around the console housing.

[0027] The ECU 20 is an example of the display control section that displays the operation screen on the display device 30, and is an electronic control unit that executes processing of generating the operation screen. The ECU 20 moves a cursor that selects one of the plurality of selection targets on the operation screen according to the position of the operation finger on the operation section 11 detected by the detection section 12. The ECU 20 is configured to include, for example, a microcomputer as a central component and realizes various types of control processing by executing various programs stored in a ROM in a CPU.As a functional section realized by executing one or more programs in the CPU, the ECU 20 includes a state receiving section 21, a foreground image generating section 22, a background image generating section 23, and a composition processing section 24. In addition, the ECU 20 includes a storage section 29 as a predetermined storage area of a non-volatile internal memory.

[0028] The state receiving section 21 performs processing of receiving the state signal received from the pointing device 10 every certain cycle. The state receiving section 21 sends the received state signal to the foreground image generating section 22 and the background image generating section 23.

[0029] The foreground image generation section 22 generates a foreground image of the operation screen, including: the plurality of selection targets (a plurality of target components such as icons) whose selection operation can be performed using the pointing device 10; and a cursor for selecting one selection target from the plurality of selection targets. The cursor indicates the selected target component by emphatically displaying the target component such as the icon. For example, the cursor displays the selected target component in a mode of increasing a luminance of the selected target component above that of the other target components, a mode of displaying the color of the selected target component in a mode different from the color of the other target components, a mode of surrounding the selected target component with a frame, or the like.

[0030] The foreground image generation section 22 determines a cursor position (i.e., which of the plurality of target components is selected) based on the state signal received from the state reception section 21. Specifically, the foreground image generation section 22 first calculates the position on the operation screen corresponding to the state signal. For example, the foreground image generation section 22 first stores correspondence information in advance in the storage section 29, the first correspondence information correlating change amounts of the x-coordinate and the y-coordinate on the pointing device 10 based on the state signal with change amounts of the x-coordinate and the y-coordinate on the operation screen.In this way, the foreground image generation section 22 can calculate, based on the first correspondence relationship information, the change amounts of the x-coordinate and the y-coordinate on the operation screen corresponding to an operation of the pointing device 10. With initial values of the x-coordinate and the y-coordinate (representative coordinates indicating an initial position of the cursor when activating the display device 1) on the operation screen as reference values, the foreground image generation section 22 can successively update the x-coordinate and the y-coordinate on the operation screen according to the successively received state signal from the pointing device 10.

[0031] Then, the foreground image generation section 22 determines a position of the cursor according to the calculated position on the operation screen. For example, the foreground image generation section 22 stores second correspondence relationship information in the storage section 29 in advance, the second correspondence relationship information correlating the position of the cursor (e.g., a center of gravity coordinate of the cursor) on the operation screen with the position on the operation screen. In this way, the foreground image generation section 22 can determine the position (the x-coordinate and the y-coordinate) of the cursor on the operation screen based on the second correspondence relationship information. Then, the foreground image generation section 22 generates a foreground image in a mode of arranging the plurality of target components, such as icons, at predetermined positions and arranging the cursor at the determined position on the operation screen (i.e.,on the one target component located at the same position as the specific position on the operating screen).

[0032] The background image generation section 23 generates a background image of the operation screen including a graphic object or a graphic object group as a collection of a number of the graphic objects (individual objects). The graphic object or the graphic object group is arranged over a specific area (for example, an area including an area where the plurality of target components are arranged) on the operation screen. The graphic object may be, for example, a curved-shape surface object arranged over the specific area on the operation screen and imitating a water surface. The graphic object group may also be, for example, a collection of a number of granular graphic objects (the individual objects) arranged over the specific area on the operation screen.

[0033] The background image generating section 23 calculates, similarly to the foreground image generating section 22, for example, based on the first correspondence relationship information, the position on the operation screen corresponding to the state signal received from the state receiving section 21.

[0034] Furthermore, the background image generation section 23 determines whether the pointing device 10 (the recognition section 12) recognizes the operation finger based on the state signal received from the state reception section 21. That is, in the case where the pointing device 10 is the integrated touch panel, the background image generation section 23 determines whether the user's finger is in contact with the integrated touch panel. In the case where the pointing device 10 is the gesture recognition device, the background image generation section 23 determines whether the user's finger is held within the operation space.

[0035] When the background image generation section 23 determines that the pointing device 10 (the recognition section 12) does not recognize the operating finger, it generates the background image including the graphic object or graphic object group in a predetermined display mode. The background image generation section 23 may generate the background image including the graphic object or graphic object group, for example, in the predetermined display mode in which a difference that visually attracts the user's attention is not included in the entire graphic object or graphic object group.

[0036] On the other hand, when the background image generation section 23 determines that the pointing device 10 (the recognition section 12) recognizes the operation finger, it changes a display mode of a part of the graphic object to be different from a display mode of the other parts of the graphic object, or changes a display mode of a part of the graphic object group to be different from a display mode of the other parts of the graphic object group. That is, the background image generation section 23 defines third correspondence relationship information in advance and stores the third correspondence relationship information in the storage section 29, wherein the third correspondence relationship information correlates the part of the graphic object or the part of the graphic object group in the background image with the position on the operation screen.The background image generation section 23 determines, based on the third correspondence relationship information, the part of the graphic object or the part of the graphic object group in the background image that corresponds to the calculated position on the operation screen. Then, the background image generation section 23 generates the background image for which the specific part of the graphic object has the display mode different from the other parts of the graphic object, or for which the specific part of the graphic object group has the display mode different from the other parts of the graphic object group. The "different display mode" means a display mode with a difference that is easily recognizable by the user viewing the operation screen.The "different display mode" may include a display mode with a different color, a display mode with a different luminance, a display mode with a different size or a different shape (specifically, a display mode in which a shape or size of the individual object corresponding to the part of the graphic object group is different from a shape or size of any other individual object), a display mode with a different displacement amount (including a displacement amount in a virtual three-dimensional space in the background image), and the like. The background image generated by the background image generating section 23 will be described in detail below.

[0037] The composition processing section 24 executes processing of composing the foreground image generated by the foreground image generating section 22 and the background image generated by the background image generating section 23 to generate the operation screen. Then, the composition processing section 24 sends a command signal containing the generated operation screen to the display device 30. In this way, the operation screen can be displayed on the display device 30.

[0038] The storage section 29 prestores the first correspondence relationship information, the first correspondence relationship information defining a correspondence relationship between a change amount of an xy coordinate (the x coordinate and the y coordinate) on the pointing device 10 and a change amount of an xy coordinate (the x coordinate and the y coordinate) on the operation screen. The storage section 29 also prestores the second correspondence relationship information, the second correspondence relationship information defining a correspondence relationship between the position (the x coordinate and the y coordinate) on the operation screen and the position of the cursor on the operation screen.The storage section 29 also stores the third correspondence relationship information in advance, the third correspondence relationship information causing the part of the graphic object or the part of the graphic object group in the background image to correlate with the position on the operation screen. On the operation screen, an x-axis direction is an example of the first direction, and a y-axis direction is an example of the second direction.

[0039] The display device 30 is arranged, for example, at a position remote from the pointing device 10 and displays the operation screen that can be operated by the pointing device 10 according to the command signal from the ECU 20 (the composition processing section 24). The display device 30 is arranged at a suitable position in the vehicle cabin, that is, at a position where the display device 30 can be easily and visually recognized by the user (the driver). As shown, for example, in Fig. As shown in Figure 2, the display device 30 may be arranged in a central portion of an instrument panel. Alternatively, the display device 30 may be a display device such as a head-up display (HUD) that displays the control screen directly in front of the user's eyes.

[0040] Next, an example of the operation screen will be explained with reference to Fig. 3A to Fig. 5C, which is generated by the display device 1 according to this embodiment, specifically, by the foreground image generating section 22, the background image generating section 23, and the composition processing section 24, and which is displayed on the display device 30.

[0041] Fig. 3A to Fig. 5C are diagrams each illustrating one example of the operation screen of the display device 1 (specifically, the foreground image generation section 22, the background image generation section 23, and the composition processing section 24) according to this embodiment. Fig. 3A, Fig. 3B and Fig. 3C show a foreground image 40a, a background image 40b, and an operation screen 40, respectively, in a case where the pointing device 10 (the detection section 12) does not detect the operation finger. Furthermore, the Fig. 4A, Fig. 5A, the Fig. 4B, Fig. 5B and the Fig. 4C, Fig. 5C show the foreground image 40a, the background image 40b, and the operation screen 40, respectively, in a case where the pointing device 10 (the detection section 12) detects the operation finger. In particular, Fig. 4A, Fig. 4B and Fig. 4C respectively the foreground image 40a, the background image 40b and the operation screen 40 at a time when a Fig. 3A, Fig. 3B and Fig. 3C (a state in which the pointing device 10 (the detection section 12) does not detect the operation finger) transitions to a state in which the pointing device 10 (the detection section 12) detects the operation finger. Fig. 5A, Fig. 5B and Fig. 5C respectively show the foreground image 40a, the background image 40b and the operation screen 40 after the operation finger detected by the detection section 12 starts to move out from the Fig. 4A, Fig. 4B and Fig. 4C shown state.

[0042] First, the operation screen 40 is displayed with reference to the Fig. 3A to 3C in the case where the pointing device 10 (the detection section 12) does not detect the operating finger.

[0043] As in Fig. 3A, the foreground image 40a includes icons 41 to 45 as selection targets and a cursor 46 that selects one of the icons 41 to 45. Icon 41 is a switching component that switches to an operation screen for an in-vehicle air conditioner. Icon 42 is a switching component that switches to an operation screen for making a phone call. Icon 43 is a switching component that switches to an operation screen for an in-vehicle navigation device. Icon 44 is a switching component that switches to an operation screen for an in-vehicle audio device. For example, icon 45 is a switching component that switches to an operation screen for making various types of settings of the display device 1.Positions of the icons 41 to 45 on the operation screen 40 are defined in advance, and the icons 41 to 45 are arranged sequentially from a left end to a right end at a slightly lower position in the vertical direction from the center of the foreground image 40a (ie, the operation screen 40). To emphasize a state in which one of the icons 41 to 45 is selected, a mode of surrounding the selected icon 41 to 45 with a frame for the cursor 46 is applied to this example.

[0044] The foreground image generation section 22 determines a position of the cursor 46 on the foreground image 40a (ie, the operation screen 40) based on the state signal received from the state receiving section 21. Then, the foreground image generation section 22 arranges the icons 41 to 45 at the predetermined positions and generates the foreground image 40a of the operation screen 40 on which the cursor 46 is arranged at the determined position (specifically, the same position as one of the icons 41 to 45). Since the pointing device 10 (the recognition section 12) positions the operation finger in the Fig. 3A to 3C, the position of the cursor 46 corresponds to a position of termination of the last movement operation of the cursor 46 or the predetermined initial position of the cursor 46. In this example, the cursor 46 is as shown in Fig. 3A, is arranged at the same position as the icon 41 (ie, on the icon 41) and is in a state of selecting the icon 41.

[0045] As in Fig. 3B, the background image 40b includes a graphic object group (hereinafter referred to as "object group") 47 configured by including a large number of the granular graphic objects (hereinafter simply referred to as "grains"). The object group 47 is arranged in a mode of arranging the grains as components in a lateral line over a specific area on the operation screen 40, specifically, a movement area of the cursor 46 in the lateral direction of the operation screen 40 at the bottom of the background image 40b (i.e., the operation screen 40). When the background image generation section 23 determines that the pointing device 10 (the recognition section 12) does not recognize the operation finger, it generates the background image 40b including the object group 47 in the predetermined display mode, i.e.,the display mode in which, in this example, the grains are arranged on the lateral line (object group 47 is arranged along the lateral direction).

[0046] Then, the composition processing section 24 sets the foreground image 40a in Fig. 3A and the background image 40b in Fig. 3B to generate the operating screen 40. As in Fig. 3C, the operation screen 40 includes the icons 41 to 45 arranged in a row from the left end to the right end at a position slightly lower in the vertical direction from the center, and the cursor 46 arranged at the same position as the icon 41 (ie, on the icon 41). The operation screen 40 also includes the object group 47 containing the grains arranged in a line from the left end to the right end at a position further below the icons 41 to 45 and the cursor 46.

[0047] Next, the operation screen 40 is displayed with reference to the Fig. 4A to 4C at a time when the state in which the pointing device 10 (the detection section 12) does not detect the operation finger switches to a state in which the pointing device 10 (the detection section 12) detects the operation finger.

[0048] If the condition of the Fig. 3C shown operating screen 40, as shown in Fig. 4A, switches to the state in which the pointing device 10 (the recognition section 12) recognizes the operating finger, the position of the cursor 46 is not changed. Thus, the foreground image generation section 22 generates the same foreground image 40a as in Fig. 3A.

[0049] When the background image generating section 23 determines that the pointing device 10 (the recognition section 12) recognizes the operating finger based on the status signal from the status receiving section 21, it generates the background image 40b as shown in Fig. 4B, in which grains 47a as part of the object group 47 correlated with the position on the operation screen 40 corresponding to the status signal are in a display mode different from the grains of the other part of the object group 47. In particular, the grains 47a are noticeably offset in the vertical direction (the x-axis direction) with respect to the other grains in the object group 47.

[0050] The composition processing section 24 then sets the foreground image 40a in Fig. 4A and the background image 40b in Fig. 4B to generate the operating screen 40. As in Fig. 4C, the operation screen 40 includes the icons 41 to 45 arranged in a line from the left end to the right end at the slightly lower position from the center in the vertical direction, and the cursor 46 arranged at the same position as the icon 41 (ie, on the icon 41). The operation screen 40 also includes the object group 47 arranged from the left end to the right end in the mode in which the grains 47a are noticeably offset in the vertical direction with respect to the other grains in the object group 47 at the position further below the icons 41 to 45 and the cursor 46.In this example, the grains 47a include five grains, with a lateral position on the operation screen 40 corresponding to the state signal from the state receiving section 21 (the pointing device 10) located at the center, displayed in a mode of partially overlapping the icon 41 in a background selected by the cursor 46. Furthermore, a displacement amount of each of the grains included in the grains 47a differs in the vertical direction. Specifically, among the grains 47a, the grain closest to the lateral position on the operation screen 40 corresponding to the state signal has the largest displacement amount in the vertical direction, and the displacement amount of the grain in the vertical direction gradually decreases as the grain moves away from the lateral position on the operation screen 40 corresponding to the state signal.

[0051] As described above, when the pointing device 10 (the detection section 12) detects the operation finger, the part (the grains 47a) of the object group 47 is set to the display mode different from the other part of the object group 47. That is, the grains 47a of the object group 47 that are correlated with the position on the operation screen 40 corresponding to the state signal from the pointing device 10 (ie, the position of the cursor 46) are set to the display mode different from the other grains in the object group 47. In this way, the user can easily understand the operation state on the operation screen 40. For example, there is a case where the user's operation finger may leave the operation section 11 due to vibration of the vehicle or the like.At this time, the user recognizes that the grains 47a are displaced (raised) in the vertical direction, and thus can easily determine whether the operation screen 40 can be operated by the pointing device 10 (ie, whether an operable state continues). In addition, the user can visually recognize that the grains 47a are displaced (raised) in the vertical direction at substantially the same lateral position as the cursor 46 on the operation screen 40, and thus can easily understand (an indication of) the position of the cursor 46 at the start of operation using the pointing device 10. In particular, when the user is the driver of the vehicle, the user cannot stare at the display device 30. To overcome this problem, the grains 47a, as the part of the object group 47, are raised in the vertical direction.In this way, without staring at the display device 30, the user can easily understand whether the operation screen 40 can be operated by the pointing device 10, the position of the cursor 46, and the like. Furthermore, the grains 47a are displayed as the background of the icons 41 to 45 and the cursor 46. Thus, even if the grains 47a are raised in the vertical direction, the occurrence of a situation in which the visibility of the icons 41 to 45 and the cursor 46 deteriorates can be suppressed.

[0052] Next, the operation screen 40 is displayed with reference to the Fig. 5A to 5C after the operating finger detected by the detection section 12 starts to move.

[0053] If the Fig. 4A to 4C shown state into one shown in the Fig. 5A to 5C, the user performs an operation of moving the cursor 46 to the right by moving the operating finger in the right direction on the operating section 11.

[0054] As in Fig. 5A, the position of the cursor 46 is determined from the position shown in Fig. 4A along with the movement of the operating finger on the operating section 11. That is, the foreground image generating section 22 determines (changes) the position of the cursor 46 to the same position as the icon 42 (ie, to the icon 42) on an immediately right side of the icon 41 based on the state signal from the state receiving section 21 and generates the image shown in Fig. Foreground image 40a shown in Figure 5A.

[0055] When the background image generating section 23 as in the case of Fig. 4B based on the status signal from the status receiving section 21 determines that the pointing device 10 (the recognition section 12) recognizes the operating finger, it generates the background image 40b as shown in Fig. 5B, in which grains 47b as a part of the object group 47 correlated with the position on the operation screen 40 corresponding to the status signal are in a different display mode from the other grains of the object group 47. At this time, the grains constituting the part of the object group 47 raised in the vertical direction are shifted from the grains 47a to the grains 47b according to the movement of the operation finger on the operation section 11. That is, the grains 47b are arranged on a right side of the grains 47a, and the grains constituting the part of the object group 47 raised in the vertical direction are shifted to the right along a movement direction of the operation finger on the operation section 11.That is, the third correspondence relationship information stored in the above-described storage section 29 is set such that the grains constituting the part of the object group 47 raised in the vertical direction are shifted in the same direction by following the movement of the position on the operation screen 40 corresponding to the state signal from the state receiving section 21.

[0056] Then, the composition processing section 24 sets the foreground image 40a in Fig. 5A and the background image 40b in Fig. 5B to generate the operating screen 40. As in Fig. 5C, the operation screen 40 includes the icons 41 to 45 arranged on a line from the left end to the right end (in the lateral direction) at the slightly lower position from the center in the vertical direction, and the cursor 46 arranged at the same position as the icon 42 (ie, on the icon 42). The operation screen 40 also includes the object group 47 arranged from the left end to the right end in the mode in which the grains 47b are noticeably offset in the vertical direction with respect to the other grains at the position further below the icons 41 to 45 and the cursor 46.In this example, the grains 47b include the five grains with the lateral position on the operation screen 40 corresponding to the state signal from the state receiving section 21 (the pointing device 10) being in the center, which are displayed in a mode of partially overlapping the icon 42 in the background selected by the cursor 46.

[0057] When the operating finger, which is recognized by the pointing device 10 (the recognition section 12) as described above, is moved on the operating section 11, the part of the object group 47 in the other display mode is moved along the moving direction of the operating finger. That is, the grains (the grains 47a, 47b) as the part of the object group 47 displayed in the other display mode (the vertical direction lifting mode) are moved in the same direction (an outlined arrow in Fig. 5C) by following the movement of the position on the operation screen 40 corresponding to the state signal from the state receiving section 21 (the pointing device 10). For example, there is the case where the operation finger may leave the operation section 11 due to the vibration of the vehicle or the like while the operation finger is moving on the operation section 11. In this case, the user visually recognizes whether the part of the object group 47 in the other display mode is moved (shifted) along the movement direction of the operation finger, and thus can easily determine whether the movement operation of the cursor 46 by the pointing device 10 can be properly continued. In addition, the user recognizes the direction in which the grains constituting the part of the object group 47 displayed in the other display mode are shifted, thereby easily understanding the movement direction of the cursor 46 on the operation screen 40.

[0058] In this example, the object group 47 configured by arranging a number of the grains in the lateral direction is used; however, the grains of the object group 47 may be connected to create a graphic object. In this case, when the state in which the pointing device 10 (the detection section 12) does not detect the operation finger transitions to the state in which the pointing device 10 (the detection section 12) detects the operation finger, a part of the graphic object corresponding to the grains 47a is lifted in the vertical direction. Then, according to the movement of the operation finger detected by the pointing device 10, the part of the graphic object lifted in the vertical direction is shifted from the part corresponding to the grains 47a to a part corresponding to the grains 47b (that is, moved along the movement direction of the operation finger).Even in this modified example, the same operational effects can be realized as in this example.

[0059] In this example, the mode of significantly displacing (raising in the vertical direction) the part (the grains 47a, 47b) of the graphic object group with respect to the other part of the graphic object group is also applied as the “different display mode.” However, the invention is not limited to this mode. An operation screen 50 will be described below with reference to Fig. 6A to Fig. 8C, which contains another example of the “different display mode”.

[0060] Fig. 6A to Fig. 8C show diagrams of another example of the operation screen of the display device 1 (specifically, the foreground image generation section 22, the background image generation section 23, and the composition processing section 24) according to this embodiment. Fig. 6A, Fig. 6B and Fig. 6C show a foreground image 50a, a background image 50b, and the operation screen 50, respectively, in the case where the pointing device 10 (the detection section 12) does not detect the operation finger. Furthermore, the Fig. 7A, Fig. 8A, the Fig. 7B, Fig. 8B and the Fig. 7C, Fig. 8C show the foreground image 50a, the background image 50b, and the operation screen 50, respectively, in the case where the pointing device 10 (the detection section 12) detects the operation finger. In particular, Fig. 7A, Fig. 7B and Fig. 7C respectively the foreground image 50a, the background image 50b and the operation screen 50 at a time when a Fig. 6A, Fig. 6B and Fig. 6C (the state in which the pointing device 10 (the detection section 12) does not detect the operation finger) switches to the state in which the pointing device 10 (the detection section 12) detects the operation finger. Fig. 8A, Fig. 8B and Fig. 8C respectively show the foreground image 50a, the background image 50b and the operation screen 50 after the operation finger detected by the detection section 12 starts to move out from the Fig. 7A, Fig. 7B and Fig. 7C shown state.

[0061] First, the operation screen 50 is displayed with reference to the Fig. 6A to 6C in a case where a movement operation of a cursor 56 is not performed.

[0062] As in Fig. 6A, the foreground image 50a includes songs 51 to 55 as selection targets and the cursor 56 that selects one of the songs 51 to 55. When the cursor 56 selects one of the songs 51 to 55 using the pointing device 10, one of the songs 51 to 55 can be played from a specific audio source (for example, a CD or the like) in the audio device mounted in the vehicle. Positions of the songs 51 to 55 on the operation screen 50 are defined in advance and are arranged sequentially from an upper end to a lower end within a range in the lateral direction excluding a left end of the foreground image 50a (i.e., the operation screen 50). In this example, to emphasize a state in which one of the songs 51 to 55 is selected, a mode of surrounding the selected one of the songs 51 to 55 with the frame for the cursor 56 is applied.

[0063] The foreground image generation section 22 determines a position of the cursor 56 on the foreground image 50a (ie, the operation screen) based on the state signal received from the state receiving section 21. The foreground image generation section 22 then arranges the songs 51 to 55 at the predetermined positions and generates the foreground image 50a of the operation screen 50 on which the cursor 56 is arranged at the predetermined position (specifically, the same position as one of the songs 51 to 55). Since the pointing device 10 (the recognition section 12) positions the operation finger in the Fig. 6A to 6C, the position of the cursor 56 corresponds to a position at a termination of the last movement operation of the cursor 56 or a predetermined initial position of the cursor 56. As shown in Fig. 6A, the cursor 56 in this example is located at the same position as the song 52 (ie, on the song 52), and is in a state of selecting the song 52.

[0064] As in Fig. 6B, the background image 50b includes a graphic object group (hereinafter referred to as "object group") 57 containing a large number of granular graphic objects (hereinafter simply referred to as "grains"). The object group 57 is arranged such that the grains are arranged as components at equally spaced intervals from left to right and from top to bottom over a specific area on the background image 50b (i.e., the operation screen 50), specifically, an entire area on the operation screen 50 in the vertical direction and the lateral direction. When the background image generation section 23 determines that the pointing device 10 (the recognition section 12) does not recognize the operation finger, it generates the background image 50b containing the object group 57 in the predetermined display mode, i.e.,the display mode in which the grains in this example are arranged in the same color at equally spaced intervals from right to left and from top to bottom.

[0065] Then, the composition processing section 24 sets the foreground image 50a in Fig. 6A and the background image 50b in Fig. 6B to generate the operating screen 50. As in Fig. As shown in Fig. 6C, the operation screen 50 includes the songs 51 to 55 arranged sequentially in the vertical direction, and the cursor 56 arranged at the same position as the song 52 (ie, on the song 52). The operation screen 50 also includes the object group 57, which includes the grains of the same color arranged at the equally spaced intervals from right to left and from top to bottom across the entire area in a background.

[0066] Next, the operation screen 50 is displayed at a time when the state in which the pointing device 10 (the detection section 12) does not detect the operation finger switches to the state in which the pointing device 10 (the detection section 12) detects the operation finger, with reference to the Fig. 7A to 7C.

[0067] If the Fig. 6C shown state of the operating screen 50 in the Fig. 7A, in which the pointing device 10 (the recognition section 12) recognizes the operating finger, the position of the cursor 56 is not changed. Thus, the foreground image generation section 22 generates the same foreground image 50a as in Fig. 6A.

[0068] When the background image generating section 23 based on the status signal from the status receiving section 21 as shown in Fig. 7B, when the pointing device 10 (the recognition section 12) determines that the operation finger is recognized, it generates the background image 50b in which grains 57a, as part of the object group 57 correlated with the position on the operation screen 50 corresponding to the state signal, are in a different display mode from the grains of the other part of the object group 57. Specifically, among the grains included in the object group 57, the grains 57a are grains in two rows near a vertical position on the operation screen 50 corresponding to the state signal and have a different shape from the other grains in the object group 57.

[0069] Then, the composition processing section 24 sets the foreground image 50a in Fig. 7A and the background image 50b in Fig. 7B to generate the operating screen 50. As in Fig. As shown in Fig. 7C, the operation screen 50 includes the songs 51 to 55 arranged sequentially in the vertical direction, and the cursor 56 arranged at the same position as the song 52 (i.e., on the song 52). The operation screen 50 also includes the object group 57 having the grains arranged at equally spaced intervals from right to left and from top to bottom over the entire area in the background. The color of the grains 57a constituting the part of the object group 57 is displayed in the different mode from the color of the grains constituting the other part of the object group 57.In this example, the grains 57a include the grains in the two rows with the vertical position on the operation screen 50 corresponding to the state signal from the state receiving section 21 (the pointing device 10) being in the center, and are displayed in a mode of partially overlapping the song 52 in the background selected by the cursor 56.

[0070] In the case where the pointing device 10 (the detection section 12) detects the operating finger as described above, the part (the grains 57a) of the object group 57 is set to the display mode different from the other part of the object group 57. That is, the grains 57a of the object group 57 that are correlated with the position on the operation screen 50 corresponding to the status signal from the pointing device 10 are set to the display mode different from the other grains in the object group 57. In this way, the same operational effects as in the Fig. 3A to Fig. 5C. Specifically, the user recognizes a color change of the grains 57a as part of the object group 57, and thus can easily determine whether the operation screen 50 can be operated by the pointing device 10 (specifically, whether the operable state continues). Furthermore, by recognizing the color change of the grains 57a as the part at substantially the same vertical position as the cursor 56 on the operation screen 50, the user can easily understand (an indication of) the position of the cursor 56 at the start of the operation using the pointing device 10. In addition, the grains 57a are displayed as the background of the songs 51 to 55 and the cursor 56. Even if the color of the grains 57a is changed, the occurrence of a situation in which the visibility of the songs 51 to 55 and the cursor 56 deteriorates can be suppressed.

[0071] Next, the operation screen 50 is displayed with reference to the Fig. 8A to 8C after the operating finger detected by the detection section 12 starts to move.

[0072] If the Fig. 7A to 7C shown state into one shown in the Fig. 8A to 8C, the user performs an operation of moving the cursor 56 downward by moving the operating finger on the operating section 11 downward.

[0073] As in Fig. 8A, the position of the cursor 56 is determined together with the movement of the operating finger on the operating section 11 from the Fig. 7A. The foreground image generating section 22 determines (changes) the position of the cursor 56 to the same position as the song 53 (ie, on the song 53) on an immediately lower side of the song 52 based on the state signal from the state receiving section 21, and generates the image shown in Fig. Foreground image 50a shown in Figure 8A.

[0074] When the background image generating section 23 as in the case of Fig. 7B, based on the state signal from the state receiving section 21, determines that the pointing device 10 (the recognition section 12) recognizes the operation finger, and generates the background image 50b in which grains 57b, as part of the object group 57, correlated with the position on the operation screen 50 corresponding to the state signal, are in a different display mode from the other grains of the object group 57. At this time, grains constituting the other-color part of the object group 57 are shifted from the grains 57a to the grains 57b according to the movement of the operation finger on the operation section 11. That is, the grains 57b are arranged on a lower side of the grains 57a, and the grains constituting the other-color part of the object group 57 are shifted downward along the movement direction of the operation finger on the operation section 11.That is, the third correspondence relationship information stored in the above-described storage section 29 is set such that the grains constituting the other color part of the object group 57 are shifted in the same direction by following the movement of the position on the operation screen 50 corresponding to the state signal from the state receiving section 21.

[0075] Then, the composition processing section 24 sets the foreground image 50a in Fig. 8A and the background image 50b in Fig. 8B to generate the operating screen 50. As in Fig. As shown in Fig. 8C, the operation screen 50 includes the songs 51 to 55 arranged in a row in the vertical direction, and the cursor 56 arranged at the same position as the song 53 (ie, on the song 53). The operation screen 50 also includes the object group 57 having the grains arranged at equally spaced intervals from right to left and from top to bottom over the entire area in the background, and the color of the grains 57b as the part of the object group 57 is displayed in a different mode from the color of the other grains. In this example, the grains 57b include the grains in the two rows with the vertical position on the operation screen 50 corresponding to the state signal from the state receiving section 21 (the pointing device 10) being in the center, and are displayed in a mode of partially overlapping the song 53 in the background selected by the cursor 56.

[0076] As described above, when the operating finger recognized by the pointing device 10 (the recognition section 12) is moved on the operating section 11, the part of the object group 57 in the different display mode is moved along the moving direction of the operating finger. That is, the grains (the grains 57a, 57b) constituting the part of the object group 57 displayed in the other display mode (the mode of the different color from the other grains) are moved in the same direction (an outlined arrow in Fig. 8C) by following the movement of the position on the operation screen 50 corresponding to the status signal from the status receiving section 21 (the pointing device 10). Thus, the same operational effects as in the Fig. 3A to 5C. That is, the user visually recognizes whether the part of the object group 57 is moved (slid) along the movement direction of the operating finger in the different display mode, and can thus easily determine whether the movement operation of the cursor 56 by the pointing device 10 can be properly continued. In addition, the user recognizes the direction in which the part of the object group 57 is moved in the different display mode, and can thus easily understand the movement direction of the cursor 56 on the operation screen 50.

[0077] In this example, the object group 57 configured by arranging a number of grains at equally spaced intervals from right to left and top to bottom is used; however, for example, a planar graphic object (a plane object) covering the area where the grains of the object group 57 are arranged may be used. In this case, when the state in which the pointing device 10 (the detection section 12) does not detect the operation finger switches to the state in which the pointing device 10 (the detection section 12) detects the operation finger, a color of a part of the plane object corresponding to the grains 57a is changed. Then, the different-colored part of the plane object is shifted from the part corresponding to the grains 57a to a part corresponding to the grains 57b according to the movement of the operation finger detected by the pointing device 10.Even in this modified example, the same operational effects as in this example can be realized.

[0078] Furthermore, in this example, the graphic object group arranged in two dimensions is used. However, a virtual three-dimensional space may be set in the background image 50b, and the graphic object group arranged in this three-dimensional space may be used. An operation screen will be described below with reference to the Fig. 9A to 9C, which contains the graphic object group arranged in the virtual three-dimensional space.

[0079] The Fig. 9A to 9C show another example of the operation screen of the display device 1 (specifically, the foreground image generation section 22, the background image generation section 23, and the composition processing section 24) according to this embodiment. Fig. Fig. 9A shows an operation screen 60 in the case where the pointing device 10 (the detection section 12) does not detect the operation finger. Fig. 9B, Fig. 9C show the operation screen 60 in the case where the pointing device 10 (the detection section 12) detects the operation finger. In particular, Fig. 9B shows the operating screen 60 at a time when a Fig. 9A (the state in which the pointing device 10 (the detection section 12) does not detect the operation finger) switches to a state in which the pointing device 10 (the detection section 12) detects the operation finger. Fig. Fig. 9C shows the operation screen 60 after the operation finger detected by the detection section 12 starts to move out of the Fig. 9B shown state.

[0080] If the Fig. 9B shown state in the Fig. 9C, the user performs an operation of moving a cursor 68 to the right by moving the operating finger on the operating section 11 to the right.

[0081] As in Fig. 9A, the operation screen 60 includes, as components of a foreground image, icons 61 to 67 as selection targets and a cursor 68 that selects one of the icons 61 to 67. The icons 61 to 67 are arranged sequentially from left to right at a lower end of the operation screen 60. Since the pointing device 10 (the recognition section 12) points the operation finger in the direction shown in Fig. 9A, a position of the cursor 68 corresponds to a position at a termination of the last movement operation of the cursor 68 or a predetermined initial position of the cursor 68. In this example, the cursor 68 is located at the same position as the icon 64 (ie, on the icon 64), and is in a state of selecting the icon 64.

[0082] As in Fig. 9A, the operation screen 60 includes, as components of a background image, a graphic object group (hereinafter simply referred to as an "object group") 69 arranged in the virtual three-dimensional space on the operation screen 60. The object group 69 includes a large number of granular graphic objects (hereinafter simply referred to as "grains") arranged in the virtual three-dimensional space according to a certain rule, and arranged (along the lateral direction) from a left end to a right end (a region having a movement range of the cursor 68 moving in the lateral direction) of the operation screen 60.

[0083] If the Fig. 9A is switched to the state in which the pointing device 10 (the detection section 12) detects the operating finger, as shown in Fig. 9B, grains 69a constituting a part of the object group 69 correlated with the position on the operation screen 60 corresponding to the status signal are displayed in a different display mode from the other grains of the object group 69. That is, the grains 69a as part of the object group 69 are displaced (raised) in a certain direction (the vertical direction) in the virtual three-dimensional space with respect to the grains of the other part of the object group 69, and are displayed in a mode of partially overlapping the icon 64 in a background selected by the cursor 68. In this way, the user can, as in the Fig. 3A to Fig. 5C and Fig. 6A to Fig. 8C, the user can also easily understand the operation state on the operation screen 60. That is, the user recognizes a lift in the grains 69a that form part of the object group 69, and thus can easily understand whether the operation screen 60 can be operated by the pointing device 10, understand the position of the cursor 68, and the like.

[0084] When the operating finger detected by the detection section 12 starts to move out of the Fig. 9B shown state, as in Fig. 9C, the cursor 68 moves to the right along with the movement of the operating finger on the operating section 11, and is located at the same position as the icon 65 (i.e., on the icon 65). In addition, since the position on the operating screen 60 corresponding to the status signal is moved to the right according to the movement of the operating finger on the operating section 11, the grains constituting the part in the different display mode of the object group 69 are shifted from the grains 69a to the grains 69b. That is, the grains constituting the lifted part of the object group 69 are shifted to the right along the movement direction of the operating finger on the operating section 11. In this way, similar operational effects as in the Fig. 3A to Fig. 5C and in Fig. 6A to Fig. 8C. Specifically, the user visually recognizes whether the part of the object group 69 in the different display mode is moved (shifted) along the movement direction of the operating finger, and can thus easily determine whether the movement operation of the cursor 68 by the pointing device 10 can be properly continued. The user also recognizes a direction in which the grains constituting the part in the different display mode are shifted, and can thus easily understand the movement direction of the cursor 68 on the operation screen 60.

[0085] In this example, the object group 69 is arranged in the virtual three-dimensional space, and a design of the operation screen 60 can thereby be generated.

[0086] In this example, the object group 69 formed by arranging a large number of grains in the virtual three-dimensional space is used; however, for example, a graphic object (surface) formed by connecting the grains of the object group 69 may be used. In this case, when the state in which the pointing device 10 (the detection section 12) does not detect the operation finger switches to the state in which the pointing device 10 (the detection section 12) detects the operation finger, a part of the surface corresponding to the grains 69a is lifted in the virtual three-dimensional space. Then, the lifted part of the surface in the virtual three-dimensional space is shifted from the part corresponding to the grains 69a to a part corresponding to the grains 69b in accordance with the movement of the operation finger detected by the pointing device 10.Even in this modified example, the same operational effects as in this example can be realized.

[0087] [Second Embodiment] Next, a second embodiment will be described.

[0088] A display device 1 according to this embodiment uses a pointing device 10 to display, on a display device 30, an operation screen including a plurality of selection targets (a plurality of items constituting a scrollable list or the like), whose selection operation can be performed. The display device 1 according to this embodiment differs from the display device 1 according to the first embodiment particularly in a point that the operation screen including the plurality of selection targets (the plurality of items constituting the list or the like) arranged along a specific axis on the operation screen, whose scrolling operation in a direction of the specific axis can be performed using the pointing device 10, is displayed on the display device 30. Hereinafter, the description will focus on parts different from the first embodiment.

[0089] A configuration of the display device 1 according to this embodiment is as in the first embodiment in Fig. 1 shown.

[0090] An ECU 20 is another example of the display control section that displays the operation screen on the display device 30 and is an electronic control unit that executes processing for generating the operation screen. The ECU 20 scrolls the plurality of selection targets (the plurality of items constituting the list or the like) arranged along the specified axis on the operation screen according to a position of an operation finger on an operation section 11 detected by a detection section 12. As in the first embodiment, the ECU 20 includes a state receiving section 21, a foreground image generating section 22, a background image generating section 23, a composition processing section 24, and a storage section 29.

[0091] The foreground image generation section 22 determines an arrangement of the plurality of selection targets (the plurality of items constituting the list or the like) on the operation screen according to a content of the scrolling operation corresponding to a state signal from the state receiving section 21 (the pointing device 10). Then, the foreground image generation section 22 generates a foreground image of the operation screen including at least a part of the selection targets of the plurality of selection targets. This is because the number of the plurality of selection targets (the number of items constituting the list) that can be selected by the scrolling operation using the pointing device 10 is usually larger than the number of selection targets that can be displayed on the operation screen.

[0092] That is, the foreground image generation section 22 determines a scroll amount, a scroll direction, and the like of the list (the plurality of selection targets constituting the list) or the like based on the state signal received from the state reception section 21. For example, the foreground image generation section 22 confirms a content of the scroll operation (a type of scroll operation or the like) based on the state signal. For example, in the case where the pointing device 10 is an integrated touch panel, a "drag (trace)" scroll operation and a "browse" scroll operation are available as types of scroll operation. The "drag" scroll operation is an operation of moving a finger at a relatively slow speed while maintaining contact with an operation surface of the integrated touch panel.In contrast, the scrolling operation by "scrolling" is an operation of moving the finger at a relatively high speed in a mode of tearing off the operation surface of the integrated touch panel with the finger in a scrolling direction. For example, based on a determination as to whether the change amounts of the x-coordinate and the y-coordinate on the integrated touch panel based on the state signal received from the state receiving section 21 are greater than or equal to a certain threshold, the foreground image generating section 22 can determine whether the scrolling operation is performed by "dragging" or by "scrolling." If the foreground image generating section 22 determines that the scrolling operation is performed by "dragging," it determines the scrolling amount and the scrolling direction based on the change amounts of the x-coordinate and the y-coordinate on the integrated touch panel based on the state signal.For example, the foreground image generation section 22 stores fourth correspondence information in advance in the storage section 29, the fourth correspondence information correlating the scroll amount and the scroll direction with the change amounts of the x-coordinate and the y-coordinate on the integrated touch panel. In this way, the foreground image generation section 22 can determine the scroll amount and the scroll direction based on the fourth correspondence information. When the foreground image generation section 22 determines that the scroll operation is performed by "scrolling," it determines a specific value predefined for the scroll operation by "scrolling" as the scroll amount and determines the scroll direction based on the change amounts of the x-coordinate and the y-coordinate on the integrated touch panel based on the state signal.

[0093] Then, the foreground image generating section 22 determines the arrangement of the selection targets according to the determined scrolling amount and the determined scrolling direction, and generates the foreground image of the operation screen including at least the part of the selection targets of the plurality of selection targets (the plurality of items constituting the list).

[0094] Also, in the case where the pointing device 10 is a gesture recognition device, types corresponding to the scrolling operation by “dragging” and the scrolling operation by “scrolling through” on the integrated touch panel may be provided according to a speed of a gesture.

[0095] As in the first embodiment, the background image generating section 23 generates a background image of the operation screen including a graphic object arranged over a certain area of the operation screen or a graphic object group as an aggregation of a large number of the graphic objects (individual objects).

[0096] As in the first embodiment, the background image generating section 23 determines whether the pointing device 10 (the recognizing section 12) recognizes the operating finger based on the state signal received from the state receiving section 21.

[0097] When the background image generation section 23 determines that the pointing device 10 (the recognition section 12) does not recognize the operating finger, it generates the background image including the graphic object or graphic object group in a predetermined display mode, as in the first embodiment. Preferably, the background image generation section 23 may generate the background image including the graphic object or graphic object group in a predetermined display mode in which a difference that visually attracts the user's attention is not included in the entire graphic object or graphic object group.

[0098] On the other hand, when the background image generation section 23 determines that the pointing device 10 (the recognition section 12) recognizes the operation finger, it changes a display mode of a part of the graphic object to be different from a display mode of the other parts of the graphic object, or changes a display mode of a part of the graphic object group to be different from a display mode of the other parts of the graphic object group, as in the first embodiment. Then, according to the content of the scroll operation (the type of scroll operation, or the like), the background image generation section 23 scrolls (moves) the part of the graphic object or the graphic object group in the different display mode in the same direction as the scroll operation.

[0099] For example, the background image generation section 23, like the foreground image generation section 22, confirms the content of the scroll operation (the type of scroll operation, etc.) based on the state signal received from the state reception section 21. When the background image generation section 23 determines that the scroll operation is performed by "dragging," it determines the scroll amount and the scroll direction based on the fourth correspondence information. On the other hand, when the background image generation section 23 determines that the scroll operation is performed by "scrolling," like the foreground image generation section 22, it determines the specific value defined in advance for the scroll operation by "scrolling" as the scroll amount and determines the scroll direction based on the change amounts of the x-coordinate and the y-coordinate on the integrated touch panel based on the state signal.Then, according to the determined scroll amount and the determined scroll direction, the background image generation section 23 generates the background image in a mode in which the part of the graphic object or graphic object group in the different display mode is shifted (moved) in the same direction as the scroll operation, in a period corresponding to the content of the scroll operation. The background image generated by the background image generation section 23 will be described in detail below.

[0100] The "period corresponding to the content of the scroll operation" means a period defined in advance according to the content of the scroll operation (the type of scroll operation). In the case of the "drag" scroll operation, the "period corresponding to the content of the scroll operation" may be a period during which the scroll operation continues (i.e., a period during which the detection section 12 detects the operation finger). In the case of the "browse" scroll operation, a predetermined period is set for the "browse" scroll operation because a finger contact period (i.e., the period during which the detection section 12 detects the operation finger) is short. For example, in the "browse" scroll operation, the plurality of operation targets (the plurality of items constituting the list) are scrolled at a relatively high speed. Thus, the predetermined period may be a relatively short period.

[0101] Next, an example of the operation screen of the display device 1, specifically the foreground image generation section 22, the background image generation section 23 and the composition processing section 24 according to this embodiment will be described with reference to FIG. Fig. 10A to Fig. 12C described.

[0102] Fig. 10A to Fig. 12C shows one example of the operation screen of the display device 1 (specifically, the foreground image generation section 22, the background image generation section 23, and the composition processing section 24) according to this embodiment. Fig. 10A, Fig. 10B and Fig. 10C respectively show a foreground image 70a, a background image 70b, and an operation screen 70 in the case where the pointing device 10 (the detection section 12) does not detect the operation finger. In addition, the Fig. 11A, Fig. 12A, the Fig. 11B, Fig. 12B and the Fig. 11C, Fig. 12C show the foreground image 70a, the background image 70b, and the operation screen 70, respectively, in the case where the pointing device 10 (the detection section 12) detects the operation finger. In particular, Fig. 11A, Fig. 11B and Fig. 11C respectively the foreground image 70a, the background image 70b and the operation screen 70 at a time when a Fig. 10A, Fig. 10B and Fig. 10C (a state in which the pointing device 10 (the detection section 12) does not detect the operation finger) switches to a state in which the pointing device 10 (the detection section 12) detects the operation finger. Fig. 12A, Fig. 12B and Fig. 12C respectively show the foreground image 70a, the background image 70b and the operation screen 70 after the operation finger detected by the detection section 12 starts to move out from the Fig. 11A, Fig. 11B and Fig. 11C shown state.

[0103] A song list 71 includes a plurality of songs 71A to 71Z (26 songs), one example of a plurality of selection targets that are more in number than displayable songs (5 songs) on the operation screen 70.

[0104] First, the operation screen 70 is displayed with reference to the Fig. 10A to 10C in the case where the pointing device 10 (the detection section 12) does not detect the operating finger.

[0105] As in Fig. 10A, the foreground image 70a includes the songs 71A to 71E as part of the songs included in the song list 71 and a fixed cursor 76 that selects one of the songs 71A to 71Z included in the song list 71. When the scrolling operation of the song list 71 is performed to position one of the songs 71A to 71Z on the cursor 76, one of the songs 71A to 71Z can be played back from a specific audio source (for example, a CD or the like) in an audio device mounted on the vehicle. The songs 71A to 71E are sequentially arranged from an upper end to a lower end within a range in a lateral direction excluding a left end of the foreground image 70a (ie, the operation screen 70). In addition, the cursor 76 is fixed at the lower end within the range in the lateral direction excluding the left end of the foreground image 70a (ie, the operation screen 70). In the case shown in 。 Fig. 10A, the cursor 76 and the song 71E are located at the same position (the cursor 76 is located on the song 71E), and the song 71E is selected.

[0106] As in Fig. 10B, the background image 70b includes a graphic object group (hereinafter referred to as "object group") 77 containing a large number of granular graphic objects (hereinafter simply referred to as "grains"). The object group 77 is arranged such that the grains are arranged as components at equally spaced intervals from right to left and from top to bottom over a certain area on the background image 70b (i.e., the operation screen 70), specifically, in an entire area of the operation screen 70 in a vertical direction and the lateral direction. When the background image generation section 23 determines that the pointing device 10 (the recognition section 12) does not recognize the operation finger, it generates the background image 70b including the object group 77 in a predetermined display mode, i.e.,a display mode in which the grains in this example are arranged in the same color at equally spaced intervals from right to left and from top to bottom.

[0107] Then, the composition processing section 24 sets the foreground image 70a in Fig. 10A and the background image 70b in Fig. 10B to generate the operating screen 70. As in Fig. As shown in Fig. 10C, the operation screen 70 includes the songs 71A to 71E as the part of the song list 71 arranged sequentially in the vertical direction, and the cursor 76 arranged at the same position as the song 71E (ie, on the song 71E). The operation screen 70 also includes the object group 77, which includes the grains of the same color in a background arranged at the equally spaced intervals from right to left and from top to bottom over the entire area.

[0108] Next, the operation screen 70 is displayed with reference to the Fig. 11A to 11C at a time when the state in which the pointing device 10 (the detection section 12) does not detect the operation finger switches to the state in which the pointing device 10 (the detection section 12) detects the operation finger.

[0109] The direction of the scroll operation, ie the direction of movement of the operating finger on the operating section 11, is downward in this example.

[0110] As in Fig. 11A, scrolling in the song list 71 is not started if the Fig. 10C switches to the state in which the pointing device 10 (the detection section 12) detects the operating finger. Thus, the foreground image generation section 72 generates the same foreground image 50a as in Fig. 10A.

[0111] When the background image generating section 23 determines, based on the status signal from the status receiving section 21, that the pointing device 10 (the recognition section 12) recognizes the operating finger, as shown in Fig. 11B, it generates the background image 70b in which grains 77a forming part of the object group 77 are in a different display mode from the other grains of the object group 77. In particular, of the grains included in the object group 77, the grains 77a are grains in two rows located at the top of the operation screen 70 and have a different color from the other grains of the object group 77.

[0112] Then, the composition processing section 24 sets the foreground image 70a in Fig. 11A and the background image 70b in Fig. 11B to generate the operating screen 70. As in Fig. 11C, the operation screen 70 includes the songs 71A to 71E as the part of the song list 71 arranged sequentially in the vertical direction, and the cursor 76 arranged at the same position as the song 71E (i.e., on the song 71E). The operation screen 70 also includes the object group 77, which includes the grains arranged at equally spaced intervals from right to left and from top to bottom over the entire area, in the background, and the color of the grains 77a constituting the part of the object group 77 is displayed in the different mode from the color of the other grains of the object group 77. In this example, the grains 77a include the grains in the two rows located at the top of the operation screen 70.

[0113] In the case where the pointing device 10 (the detection section 12) detects the operation finger as described above, the part (the grains 77a) of the object group 77 is set to the display mode different from the other part (the other grains) of the object group 77. In this way, the user can easily understand the operation state of the operation screen. Specifically, the user recognizes a color change of the grains 77a as the part of the object group 77, and thus can easily determine whether the operation screen 50 can be operated by the pointing device 10 (specifically, whether the operable state continues). Furthermore, by recognizing the color change of the grains 77a as the part of the object group 77, the user can easily understand that the song list 71 is scrolled according to the operation using the pointing device 10.

[0114] Next, the operation screen 70 is displayed with reference to the Fig. 12A to 12C after the operating finger detected by the detection section 12 starts to move.

[0115] As in Fig. 12A, the songs that can be displayed are selected from the Fig. 11A is updated according to a movement of the operating finger on the operating section 11. In particular, when the song list 71 is scrolled downwards, the foreground image 70a no longer contains the songs 71A, 71B, but instead contains the songs 71Y, 71Z. In a Fig. 12A, the cursor 76 and the song 71C are located at the same position (the cursor 76 is located on the song 71C), and the song 71C is selected.

[0116] As in Fig. 12B, the background image generating section 23 generates the background image 70b in which grains 77b constituting a part of the object group 77 are in a different display mode from the other grains of the object group 77. At this time, the grains as the part of the object group 77 in the different color are shifted from the grains 77a to the grains 77b according to the movement of the operating finger on the operating section 11. Specifically, the grains 77b are arranged on a lower side of the grains 77a, and the grains constituting the part of the object group 77 in the different color are shifted downward along the movement direction of the operating finger on the operating section 11.

[0117] Then, the composition processing section 24 sets the foreground image 70a in Fig. 12A and the background image 70b in Fig. 12B to generate the operating screen 70. As in Fig. 12C, the operation screen includes the songs 71Y, 71Z, and the songs 71A to 71C as the part of the song list 71 arranged sequentially in the vertical direction, and the cursor 76 arranged at the same position as the song 71C (ie, on the song 71C). The operation screen 70 also includes the object group 77, which includes the grains arranged at the equally spaced intervals from right to left and from top to bottom over the entire area, in the background, and the color of the grains 77b constituting the part of the object group 77 is displayed in the different mode from the color of the other grains of the object group 77. In this example, the grains 77b include the grains in the two rows of the object group 77 located near the center in the vertical direction of the operation screen 70.

[0118] When the operating finger recognized by the pointing device 10 (the recognition section 12) is moved on the operating section 11, the grains (the grains 77a, 77b) constituting the part of the object group 77 displayed in the different display mode (the mode of the color different from the other grains) are moved along the moving direction (the downward direction) of the operating finger (an outlined arrow in Fig.12C). Accordingly, the user can visually recognize whether the part of the object group 77 in the different display mode is moved (slid) along the movement direction of the operation finger, and thus can easily determine whether the scrolling operation of the song list 71 by the pointing device 10 can be properly continued. Furthermore, the user recognizes a direction in which the grains constituting the part displayed in the different display mode of the object group 77 displayed as the background are moved, and thus can easily understand the scrolling operation on the operation screen 70 according to the operation using the pointing device 10.

[0119] In this example, the object group 77 containing a number of grains arranged at equally spaced intervals from right to left and from top to bottom is used; however, for example, a planar graphic object (a layer object) covering an area where the grains of the object group 77 are arranged may be used. In such a case, when the state in which the pointing device 10 (the detection section 12) does not detect the operation finger switches to the state in which the pointing device 10 (the detection section 12) detects the operation finger, a color of a part of the layer object corresponding to the grains 77a is changed. According to the movement of the operation finger detected by the pointing device 10, the part of the layer object in the different color is shifted from the part corresponding to the grains 77a to a part corresponding to the grains 77b.Even with such a modified example, the same operational effects as in this example can be realized.

[0120] Furthermore, in this example, the graphic object group arranged in a plane on the operation screen is used. However, a virtual three-dimensional space may be set in the background image 70b, and a graphic object or graphic object group arranged in this three-dimensional space may be used. Even in such a modified example, the same operational effects as in this example can be realized.

[0121] Embodiments of the invention have been described. However, the invention is not limited to a particular embodiment, and various modifications and changes may be made within the scope of the invention as defined in the claims.

Claims

[1] Vehicle display device with a display section (30), a pointing device (10) which includes an operating section (11) and a detection section (12) which detects an operating finger operating the operating section (11), and a display control section (20) for displaying an operation screen (40; 50; 60; 70) on the display section (30), the operation screen containing a plurality of selection targets (41, 42, 43, 44, 45; 51 - 55; 61 - 67; 71A - 71Z) whose selection operation can be performed using the pointing device (10), wherein the display control section (20) is arranged to Generating an image in which a cursor (46) is moved to select a selection target from the plurality of selection targets (41, 42, 43, 44, 45; 51-55; 61-67; 71A-71Z), and / or an image in which the plurality of selection targets (41, 42, 43, 44, 45; 51-55; 61-67; 71A-71Z) arranged along a specific axis on the operation screen (40; 50; 60; 70) are scrolled on the operation section (11) in accordance with a position of the operation finger detected by the detection section (12), Generating an image containing an object or group of objects (47; 57; 69; 77) over a specific area on the operating screen (40; 50; 60; 70), and when the operating finger is detected by the detection section (12), executing processing of changing a shape of a part (47a, 47b; 57a, 57b; 69a, 69b; 77a, 77b) of the object or object group (47; 57; 69; 77) with respect to the shape of the part (47a, 47b; 57a, 57b; 69a, 69b; 77a, 77b) of the object or object group (47; 57; 69; 77) in a case where the operating finger is not detected by the detection section (12). [2] The vehicle display device according to claim 1, wherein the display control section (20) is configured to generate an image in which a changed part of the object or object group (47; 57; 69; 77) is moved along a moving direction of the operating finger when the operating finger detected by the detection section (12) is moved with respect to the operating section (11), the changed part representing a part in which the shape of the part (47a, 47b; 57a, 57b; 69a, 69b; 77a, 77b) of the object or object group (47; 57; 69; 77) is changed. [3] The vehicle display device according to claim 1 or 2, wherein the display control section (20) is configured to generate the operation screen (40; 50; 60; 70) by composing a foreground image (40a; 50a; 70a) containing the plurality of selection targets (41, 42, 43, 44, 45; 51-55; 61-67; 71A-71Z) and a background image (40b; 50b; 70b) containing the object or object group (47; 57; 69; 77). [4] A vehicle display device according to claim 1 or 2, wherein the display control section (20) is arranged to determine the shape of the part (47a, 47b; 57a, 57b; 69a, 69b; 77a, 77b) of the object or object group (47; 57; 69; 77) based on a position on the operation screen (40; 50; 60; 70), the position corresponding to a state signal from the pointing device (10). [5] Vehicle display device according to claim 1 or 2, wherein the plurality of selection targets (41, 42, 43, 44, 45; 61 - 67) are arranged in a row in a first direction on the operating screen (40; 60), the object or group of objects (47; 69) is arranged along the first direction, and the display control section (20) is arranged to generate an image in which the shape of the part (47a, 47b; 69a, 69b) of the object or object group (47; 69) is offset in a second direction intersecting the first direction with respect to a shape of the other part of the object or object group (47; 69) when the operating finger is detected by the detection section (12). [6] Vehicle display device according to claim 5, wherein the shape of the part (47a, 47b; 69a, 69b) of the object group (47; 69) contains a plurality of grains, and a displacement amount of the plurality of grains in the second direction is reduced as the plurality of grains move away from a position on the operation screen (40; 50; 60; 70) in the first direction, the position corresponding to a status signal from the pointing device (10). [7] Vehicle display device according to claim 1 or 2, wherein the object or group of objects (69) is arranged in a virtual three-dimensional space, and the display control section (20) is arranged to generate an image in which the shape of the part (69a, 69b) of the object or the object group (69) is raised in a certain direction in the virtual three-dimensional space with respect to a shape of the other part of the object or the object group (69) when the operating finger is detected by the detection section (12).

Images