Information processing apparatus and information processing method

Abstract

To improve the visibility of the details of display in information processing of controlling a second display provided on at least one face of an operating unit arranged with a first display.SOLUTION: An information processing apparatus comprises: a first display; an operating unit that is arranged with the first display; a second display that is provided on at least one face of the operating unit; and a control unit that moves the details of display displayed on the second display from the second display and displays the details on the first display.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present invention relates to an information processing device and an information processing method. [Background technology]

[0002] The operating device of Patent Document 1 includes a dial knob and a detection unit that detects a flick operation on the tip surface of the dial knob, and based on the detected flick operation, selects one of a number of change targets related to the vehicle air conditioning, and causes an image showing the state of the selected change target to be displayed on a display unit. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-34675 A Summary of the Invention [Problem to be solved by the invention]

[0004] The device in Patent Document 1 has a problem in that the display area is limited to a small space on the top surface of the dial knob, making it difficult for the user to recognize the displayed content. In particular, when operating the dial knob, the display is often covered by the user's hand, making it difficult to recognize the displayed content while operating the dial knob.

[0005] One aspect of the disclosed embodiment is that in information processing that controls a second display provided on at least one surface of an operating unit arranged alongside a first display, the display content shown on the second display is moved and displayed on the first display, whose display space is not limited by the size of the operating unit, thereby improving the visibility of the display content. [Means for solving the problem]

[0006] An embodiment of the present disclosure is exemplified by an information processing device including a first display, an operation unit arranged alongside the first display, a second display provided on at least one surface of the operation unit, and a control unit that moves display content displayed on the second display from the second display to the first display. Effect of the Invention

[0007] The control unit of the present disclosure can improve the visibility of the display content in information processing that controls the second display provided on at least one surface of the operation unit arranged alongside the first display. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is an external perspective view illustrating a display device serving as a CID and an operation unit of an in-vehicle system according to a first embodiment. [Diagram 2] FIG. 2 is a diagram showing a schematic configuration of the display device and the operation unit. [Diagram 3] FIG. 3 is an exploded perspective view illustrating the structure of the operation unit. [Figure 4] FIG. 4 is a diagram illustrating the effect of a bundle of optical fibers in the operation unit. [Diagram 5] FIG. 5 illustrates an effect obtained by a modification of the operation unit illustrated in FIG. [Figure 6] FIG. 6 illustrates a hardware configuration of the in-vehicle system. [Figure 7] FIG. 7 shows an example of a personified character that represents the display content provided by the in-vehicle system via the operation unit. [Figure 8] FIG. 8 shows an example of the display content provided by the operation unit of the in-vehicle system. [Figure 9] FIG. 9 is a diagram illustrating a display control process in the in-vehicle system. [Figure 10] FIG. 10 is a diagram showing an example of changing the display mode of the display contents. [Figure 11]FIG. 11 is a diagram showing a modified example of the operation unit. [Figure 12] FIG. 12 is a diagram illustrating a drive control process of the operation unit in the in-vehicle system. [Figure 13] FIG. 13 is a diagram illustrating an example of a hardware configuration of an in-vehicle system according to the second embodiment. [Figure 14] FIG. 14 is a diagram illustrating a display control process in the in-vehicle system according to the second embodiment. [Figure 15] FIG. 15 is a diagram illustrating an example of a hardware configuration of an in-vehicle system according to the third embodiment. [Figure 16] FIG. 16 is a diagram illustrating a display control process in the in-vehicle system according to the third embodiment. [Figure 17] FIG. 17 is a diagram showing an example of a process for moving the display content to a position where a touch operation has been performed. [Figure 18] FIG. 18 is a diagram showing an example of a function selection menu. [Figure 19] FIG. 19 is a diagram showing a state in which the function icons in FIG. 18 have been moved. [Figure 20] FIG. 20 is a diagram illustrating an example of a hardware configuration of an in-vehicle system according to the fourth embodiment. [Figure 21] FIG. 21 is a diagram showing the arrangement of display units in a vehicle. [Figure 22] FIG. 22 is a diagram showing an example in which a character 61 moves between display units arranged on seats aligned to the left and right. [Figure 23] FIG. 23 is a diagram showing an example in which a character 61 moves between display units arranged on seats aligned in front and behind each other. [Figure 24] FIG. 24 is a diagram showing an example in which a display unit on the driver's seat side and a display unit on the passenger's seat side are disposed on pillars. [Diagram 25] FIG. 25 is a diagram showing an example in which a display device is provided on the ceiling surface. [Figure 26] FIG. 26 is a diagram showing an example in which a projection device for projecting an image onto the floor surface is provided. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] First Embodiment Hereinafter, with reference to the drawings, an in-vehicle system 100 according to an embodiment and an information processing method executed by the in-vehicle system 100 will be described. The in-vehicle system 100 has, for example, a Center Information Display (CID) disposed in front of the driver's seat of the vehicle, and executes processing as an interactive system that displays and outputs to a user in response to the user's operation of an operation unit.

[0010] <Configuration> Fig. 1 is an external perspective view illustrating a display device 14 and an operation unit 20 which are CIDs of an in-vehicle system 100, and Fig. 2 is a diagram showing a schematic configuration of the display device 14 and the operation unit 20, where A in Fig. 2 shows a front view and B in Fig. 2 shows a side view. In the example of Figs. 1 and 2, the display device 14 is embedded in an instrument panel 40 of a vehicle.

[0011] The operation unit 20 has a housing 201 and a support mechanism 30 that supports the housing 201. The housing 201 is, for example, cylindrical and has two end faces perpendicular to the central axis. The housing 201 has openings formed on the two end faces, and the edges of the openings and the cross section of the housing 201 are ring-shaped. The operation unit 20 may be cylindrical or may be rectangular. However, the shape of the operation unit 20 is not limited to a cylindrical shape, cylindrical shape, rectangular shape, etc., and may be tapered, sloped, etc. The operation unit 20 may have various shapes, such as a cylindrical shape, a rectangular tube shape, a polyhedron, a shape including a quadratic curved surface, etc. The operation unit 20 is arranged side by side with the display device 14 on the lower side of the display device 14 as the first display. In the example of FIG. 1, the operation unit 20 is arranged in the center in the left-right direction on the lower side of the display device 14, the upper part of the operation unit 20 is located above the lower side of the display device 14, and the lower part of the operation unit 20 is located below the lower side of the display device 14. Not limited to this, the operation unit 20 may be located entirely above the lower side of the display device 14, or entirely below the lower side of the display device 14. In addition, the operation unit 20 of this embodiment is provided adjacent to the display device 14, but is not limited to this, and may be provided at a distance from the display device 14 as long as it is located at a position where the user can view it together with the display device 14. Here, the back surface of the housing 201 facing the display device 14 is called the facing surface, and the opposite surface of the facing surface is called the display surface 202. In the housing 201, a bundle of optical fibers 203 (see Figs. 3 to 5) is provided in a direction connecting an opening of the display surface 202 and an opening of the opposing surface (rear surface), forming a light guide path that projects light from the display element 14A of the display device 14 onto the display surface 202. Therefore, in the operation unit 20, the display surface 202 functions as a second display, and a part of the display element 14A facing the opposing surface (rear surface) that is the opposite surface of the display surface 202 can be projected onto the display surface 202. The operation unit 20 is not limited to a configuration that guides the display of the display device 14 to the display surface 202, and the operation unit 20 itself may have a function as a display device. Specifically, a display device such as a liquid crystal display or an Organic Light Emitting (OLED) display may be provided in the housing 201, and the display may be wired via a support mechanism 30 or the like.

[0012] The operation unit 20 is supported from below in the vertical direction by the support mechanism 30. As described above, the display device 14 of the present embodiment is embedded in the instrument panel 40 of the vehicle, and therefore the left-right direction of the display element 14A is the left-right direction with respect to the traveling direction of the vehicle.

[0013] FIG. 3 is an exploded perspective view illustrating the structure of the operation unit 20. As described in FIG. 1 and FIG. 2, the operation unit 20 is attached with its back surface facing the display device 14. As shown in FIG. 3, the operation unit 20 has a housing 201, a fingerprint sensor 22, a touch sensor 23, a dial switch 25, a push switch 26, and an optical fiber 203. The operation unit 20 displays the processing state of the in-vehicle system 100 on a display surface 202 as a second display, and receives user input via the fingerprint sensor 22, the touch sensor 23, the dial switch 25, the push switch 26, and the like. In addition to the configuration illustrated in FIG. 3, the operation unit 20 also has a thermosensor 24A, an infrared sensor 24B, and an illuminance sensor 27 (FIG. 6).

[0014] The housing 201 is cylindrical, forms the side surface of the ring-shaped cross section of the operation unit 20, and forms ring-shaped edges at both ends of the cylinder. This forms an outer shell with ring-shaped openings at both ends. The housing 201 may be decorated, and is also called a decorative portion.

[0015] A flat circular touch sensor 23 is provided on the display surface 202 of the operation unit 20. The touch sensor 23 has the same configuration as a touch panel provided on a normal mobile device, for example, a smartphone. For example, the touch sensor 23 has a first electrode row arranged parallel to a first direction (defined as the X-axis direction) on a plane, and a second electrode row arranged parallel to a second direction (defined as the Y-axis direction) perpendicular to the first direction. The touch sensor 23 detects the movement of the user's finger in terms of coordinate values ​​of the X-axis and Y-axis based on a change in capacitance from the user's finger with the two electrode rows. However, the touch sensor 23 is not limited to a capacitance type sensor, and may be, for example, a pressure-sensitive sensor. Although omitted in FIG. 3, a transparent protective film may be provided on the upper surface (display surface 202 side) of the touch sensor 23.

[0016] The fingerprint sensor 22 is, for example, a capacitance sensor. The fingerprint sensor 22 measures capacitance based on the unevenness of a fingerprint that touches the front surface (display surface 202) of the operation unit 20, and forms a two-dimensional image to form an image of the fingerprint. However, the fingerprint sensor 22 is not limited to a capacitance sensor. For example, the fingerprint sensor 22 may be a Charge Coupled Device (C It may be an array of CCD (Crystal-Diode) or Metal-Oxide-Semiconductor (MOS) image sensors. If fingerprint sensor 22 is an array of image sensors, it operates in a similar manner to camera 21 to produce an image of the fingerprint.

[0017] When the user rotates the housing 201 clockwise or counterclockwise, the dial switch 25 outputs a physical quantity or an electrical signal corresponding to the amount of rotation. The in-vehicle system 100 acquires the value of the physical quantity or the value of the electrical signal intensity corresponding to the amount of rotation from the dial switch 25. The dial switch 25 has a cylindrical shape that is inscribed slidably on the cylindrical inner surface of the housing 201.

[0018] When the user rotates the housing 201 clockwise or counterclockwise, the contact point between the inner surface of the housing 201 and the dial switch 25 changes, and the electric resistance value, voltage value, etc. between the terminals drawn from the dial switch 25 change. The in-vehicle system 100 detects the amount of rotation and the direction of rotation of the housing 201 from the change in the electric resistance value, voltage value, etc. between the terminals of the dial switch 25. For example, when the housing 201 rotates clockwise, the electric resistance value, voltage value, etc. increase (or decrease) from the initial value. Also, for example, when the housing 201 rotates counterclockwise, the electric resistance value, voltage value, etc. decrease (or increase) from the initial value. Also, the in-vehicle system 100 may determine that the switch is turned off when the change in the electric resistance value, voltage value, etc. between the terminals output from the dial switch 25 is within a reference value, and that the switch is turned on when the change exceeds the reference value. As described above, the operation unit 20 can accept a rotation operation by the user. That is, the operation unit 20 may be physically rotatable. However, the dial switch 25 may be omitted. When the dial switch 25 is omitted, the housing 201 of the operation unit 20 does not have to rotate. When the housing 201 of the operation unit 20 does not rotate, the operation unit 20 may detect a rotation operation by the user's hand by the touch sensor 23.

[0019] The dial switch 25 is not limited to a switch that detects changes in electrical resistance, voltage, etc. between terminals. For example, a general rotary encoder may be used as the dial switch 25. For example, the dial switch 25 may be a switch that has a plurality of windows provided in a circumferential direction on its cylindrical side wall, and that judges whether or not light is transmitted between the inside and outside (the inner surface side of the housing 201) of the cylindrical side wall of the dial switch 25. For example, a light emitting unit such as an LED may be provided on the inner surface side of the housing 201, and a light receiving unit such as a photodiode that receives light from the light emitting unit that has passed through the window may be provided on the inside of the cylindrical side wall of the dial switch 25. By changing and arranging the size of the window stepwise, such as large, medium, and small, the amount of light received by the light receiving unit or the light receiving time changes. The direction of rotation can also be determined depending on whether the direction of this change is large, medium, or small.

[0020] When the push switch 26 is pressed, the contacts are closed and an electric signal flows between the contacts, thereby detecting the user's operation. In this embodiment, four push switches 26 are provided between the dial switch 25 and the ring 204 provided below it. The four push switches 26 are provided in the respective arc regions that divide the ring 204 into four, for example, at the center positions of the sectorial arcs that divide the circular ring 204 at the center of the ring 204 by 90 degrees. Therefore, when the user presses the housing 201 evenly, the four push switches 26 can be simultaneously turned from off to on. Also, when the user presses a position on the housing 201 that corresponds to the vicinity of the middle of one of the ring portions that are divided into two from the ring 204, the two push switches 26 can be turned from off to on. Furthermore, when the user presses a position on the housing 201 that corresponds to the 90-degree sector portion that is divided into four from the ring 204, the two push switches 26 can be simultaneously turned from off to on. When the push switch 26 is pressed, only one of the push switches 26 can be turned on from off. The number of the push switches 26 is not limited to four, but may be one to three, or may be five or more. When the number of the push switches 26 is less than four, the in-vehicle system 100 can construct a simple user interface. When the number of the push switches 26 is five or more, the in-vehicle system 100 can detect the user's operation in a detailed manner.

[0021] The optical fibers 203 are inserted as a bundle into the cylindrical space inside the housing 201 between an opening on the back surface of the operation unit 20 and an opening on the display surface 202. The bundle of optical fibers 203 projects the part of the display element 14A facing the back surface of the operation unit 20 onto the display surface 202 of the operation unit 20.

[0022] 4 is a diagram illustrating the effect of the bundle of optical fibers 203 in the operation unit 20. As described in FIG. 3, the portion of the display element 14A facing the rear surface of the operation unit 20 is projected onto the display surface 202 of the operation unit 20 by the optical fibers 203.

[0023] FIG. 5 illustrates an effect of the modified example of the operation unit 20 illustrated in FIG. 4. That is, in FIG. 5, the bundle of optical fibers 203 gradually expands in the range from the back surface of the operation unit 20 to the display surface 202, forming a tapered shape. That is, in the bundle of optical fibers 203, the gaps between the optical fibers 203 are narrow and dense in the portion close to the back surface of the operation unit 20, and the gaps between the optical fibers 203 are widened in the portion close to the display surface 202 of the operation unit 20. Alternatively, the outer diameter of each optical fiber 203 may be narrow in the portion close to the back surface of the operation unit 20, and may be widened in the portion close to the display surface 202 of the operation unit 20. With such a bundle of optical fibers 203, the number of pixels per unit area of ​​the image on the display surface 202 is lower than that of the display element 14A of the display device 14, and the fineness (resolution) is reduced, but the entire image is enlarged. That is, the portion of the display element 14A facing the back surface of the operation unit 20 is enlarged and projected onto the display surface 202. In this case, even if the display element 14A has a typical rectangular shape, the lower side of the display surface 202 may be disposed so as to protrude downward beyond the lower side of the display device 14.

[0024] 6 illustrates an example of a hardware configuration of the in-vehicle system 100. The in-vehicle system 100 includes a control unit 10 and external devices connected to the control unit 10 via an interface (I / F). The control unit 10 includes a CPU 11 and a main memory device 12. The external devices include an external memory device 13, a display device 14, an input unit 15, a communication device 16, a microphone 17, a camera 21, a speaker 28, an operation unit 20, and a car navigation system 50.

[0025] The CPU 11 executes a computer program that has been loaded in an executable manner in the main storage device 12, and provides the functions of the in-vehicle system 100. The CPU 11 is also called a processor. However, the CPU 11 is not limited to a single processor, and may have a multi-processor configuration. The CPU 11 may also be a single processor connected via a single socket, and may have a multi-core configuration. Furthermore, at least a part of the processing of the in-vehicle system 100 may be provided by a dedicated processor such as a Digital Signal Processor (DSP), a Graphics Processing Unit (GPU), a numerical calculation processor, a vector processor, or an image processing processor, an Application Specific Integrated Circuit (ASIC), or the like. In addition, the in-vehicle At least a part of the system 100 may be a dedicated large scale integration (LSI) such as a Field-Programmable Gate Array (FPGA) or other digital circuit. At least a portion of the system 100 may include analog circuitry.

[0026] The main memory device 12, which is also simply called memory, stores computer programs executed by the CPU 11, data processed by the CPU 11, etc. The main memory device 12 may be a Dynamic Random Access Memory (DRAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), or a ory (ROM), etc.

[0027] The external storage device 13 is used, for example, as a storage area that supplements the main storage device 12, and stores computer programs executed by the CPU 11, data processed by the CPU 11, etc. The external storage device 13 may be a hard disk drive, a solid state drive (SSD), etc. Furthermore, a drive device for a removable storage medium may be provided in the in-vehicle system 100. The removable storage medium may be, for example, a Blu-ray disc, a Digital Versatile Disc (DVD), a Compact Disc (CD), a flash memory card, or the like.

[0028] The display device 14 is, for example, a liquid crystal display, an electroluminescence panel, etc. The input unit 15 is, for example, a keyboard, a pointing device, etc. In this embodiment, a touch panel is exemplified as the pointing device. The communication device 16 transmits and receives data to and from other devices on a wireless network. The wireless network is, for example, a mobile phone network. The microphone 17 acquires the user's voice. That is, the microphone 17 functions as a speech detection means for detecting the user's speech. The number of microphones 17 is not limited to one, and multiple microphones may be provided. The microphone 17 may be provided within the operation unit 20.

[0029] Camera 21 may be disposed near operation unit 20, or may be disposed anywhere in the vehicle cabin. Camera 21 has, for example, an array of CCD or MOS image sensors, and transmits an image, which is pixel array data, to CPU 11. CPU 11 acquires video from camera 21 at a predetermined frame rate, for example, 10 frames / second to 120 frames / second. In this embodiment, an image including a series of multiple frames is referred to as a video. Also, what is included in one frame is simply referred to as an image.

[0030] The speaker 28 has, for example, a piezoelectric element. An electrical signal is input to the speaker 28, which converts the electrical signal into mechanical vibrations of the piezoelectric element (piezo element) to generate sound on the front side of the display device 14. The speaker 28 may be disposed near the operation unit 20. The speaker 28 may also be disposed at any location inside the vehicle. The car navigation system 50 has map information and has a function of determining the vehicle's position and displaying the vehicle's position on a map, as well as a function of providing route guidance to a set destination.

[0031] The operation unit 20 has a touch sensor 23, a thermosensor 24A, an infrared sensor 24B, a dial switch 25, a push switch 26, and an illuminance sensor 27. Each of these components of the operation unit 20 is connected to the CPU 11 via an interface I / F. Among these, the touch sensor 23, the dial switch 25, and the push switch 26 are the same as those described in FIG. 3.

[0032] The thermosensor 24A is also called a thermal camera. The thermosensor 24A measures the temperature distribution of the object to be photographed from the intensity of infrared or far-infrared rays in an image taken by infrared or far-infrared rays. The thermosensor 24A outputs infrared image data, for example, by an array of pixel sensors. The pixel sensors are made of, for example, InGaAs, GaAs, AL GaAs, InAs, GaSb, etc. The pixel sensor is an element such as a photodiode or phototransistor made of a compound semiconductor or a laminate thereof. However, instead of an array of compound semiconductor elements, an array of thermocouples (thermopiles) may be used as the pixel sensor. The thermosensor 24A may have a filter that selects infrared rays in a specific wavelength range from reflected light from an object. The thermosensor 24A may be disposed on the edge of the operation unit 20 (housing 201). The thermosensor 24A may be separated from the operation unit 20 and disposed near the operation unit 20. The thermosensor 24A may be disposed at an arbitrary location in the vehicle cabin, away from the operation unit 20.

[0033] The infrared sensor 24B measures the temperature of the object based on the intensity of infrared rays from the object. However, unlike the thermosensor 24A, the infrared sensor 24B measures the temperature of the object based on, for example, the amount of infrared rays from a specific position on the object, rather than the temperature distribution of the object. The infrared sensor 24B has, for example, a thermopile. However, the infrared sensor 24B may be one that utilizes the pyroelectric effect of ferroelectric ceramics. The infrared sensor 24B may also be one that uses elements such as photodiodes and phototransistors made of compound semiconductors similar to those of the thermosensor 24A. The infrared sensor 24B may be disposed on the edge of the operation unit 20 (casing 201). The infrared sensor 24B may also be separated from the operation unit 20 and disposed near the operation unit 20. The infrared sensor 24B may also be disposed at any location in the vehicle cabin, away from the operation unit 20.

[0034] The illuminance sensor 27 uses elements such as a photodiode, a phototransistor, a photoresistor, etc., and measures the illuminance near the display surface 202 of the operation unit 20. The illuminance sensor 27 may be disposed on an edge of the operation unit 20 (housing 201).

[0035] <Display example> 7 and 8, examples of display contents in the in-vehicle system 100 will be described. FIG. 7 shows an example of a character 61 which anthropomorphizes the display contents provided by the in-vehicle system 100 through the operation unit 20. Here, the character 61 is a symbolic graphic expression for expressing the state of the operation unit 20. The in-vehicle system 100 moves and displays the character 61 displayed on the display surface 202 of the operation unit 20 to the display unit 14 using a movement trigger (opportunity for movement) such as an operation of the operation unit 20, detection of a specific utterance, or a touch operation on the display unit 14.

[0036] A of FIG. 7 shows a state in which the character 61 is displayed on the display surface 202, which is the second display. B of FIG. 7 shows the process in which the character 61 moves from the display surface 202 to the display device 14. In B of FIG. 7, the character 61 is deformed so as to pull the tail 61a, but the character 61 may be moved without being deformed. C of FIG. 7 shows a state in which the character 61 is displayed on the display device 14, which is the first display. In C of FIG. 7, the tail 61a is torn off and droplets 61b are displayed on the trajectory of the character's movement, but the droplets 61b do not necessarily have to be displayed. In addition, the droplets 61b may be continuously displayed while the character 61 is displayed on the display device 14, or may be erased over time, for example, starting from the droplets farthest from the character 61. In this way, the in-vehicle system 100 can display the character 61 enlarged and improve the visibility of the character 61 by moving and displaying the character 61 displayed on the display surface 202 to the display device 14, whose display space is not limited by the size of the operation unit 20. Furthermore, the in-vehicle system 100 can more visually indicate that the character 61 is moving by displaying the character 61 as if it were trailing a tail 61a or leaving droplets 61b when the character 61 moves.

[0037] FIG. 8 is an example of display content 62 provided by the in-vehicle system 100 through the operation unit 20. Here, the display content 62 is a graphic representation showing numerical values, icons, etc. for expressing the state of the operation unit 20. In the example of FIG. 8, the user selects the air conditioner temperature setting function for the in-vehicle system 100, and the display content 62 is the set temperature of the air conditioner. The in-vehicle system 100 moves and displays the display content 62 displayed on the display surface 202 of the operation unit 20 to the display device 14 using, as a movement trigger, an operation on the operation unit 20, detection of a specific utterance, a touch operation on the display device 14, etc. A in FIG. 8 shows a state in which the display content 62 is displayed on the display surface 202, which is the second display. B in FIG. 8 shows a process in which the display content 62 is being moved from the display surface 202 to the display device 14. 8C shows a state in which the display content 62 is displayed on the display device 14, which is the first display. In this way, the in-vehicle system 100 moves and displays the display content 62 displayed on the display surface 202 on the display device 14, whose display space is not limited by the size of the operation unit 20, thereby making it possible to display the display content 62 in a larger size and improving the visibility of the display content 62. Also, in the example of FIG. 8, when the display content 62 is moved, The movement of the display content 62 can be more visually understood by applying a display effect such as deforming the display so as to leave a trail when the display content 62 is moved, or leaving droplets along the path of the movement of the display content 62. Note that the example of Fig. 8 is a case where the user selects the temperature setting function of the air conditioner for the in-vehicle system 100, and the display content 62 is a number representing the set temperature, but this embodiment can also be applied to functions other than the temperature setting function of the air conditioner, such as the air volume setting function of the air conditioner, the volume setting function of the audio, and the channel selection function of the radio or television, and in that case, the display content 62 may be a number representing the air volume or volume, or a number representing the radio frequency or television channel.

[0038] <Processing example> The control unit 10 of the in-vehicle system 100 executes the processing of FIG.

[0039] Fig. 9 is a diagram illustrating a display control process in the in-vehicle system 100. The control unit 10 of the in-vehicle system 100 periodically executes the display control process in Fig. 9, for example, after the in-vehicle system 100 is started up until it is stopped.

[0040] 9 is started, the in-vehicle system 100 judges whether or not an operation of the operation unit 20 by a user has been detected (S51). For example, the in-vehicle system 100 judges Yes if there is a change in the detection result by the touch sensor 23, the fingerprint sensor 22, the dial switch 25, or the push switch 26, and judges No if there is no change in the detection result.

[0041] If the answer is No in step S51, the in-vehicle system 100 determines whether a specific utterance has been detected via the microphone (S52). Here, the specific utterance is a predetermined phrase, and may be, for example, an utterance by the user calling out to a character, such as "Hey! XXX (XXX is the character's name)." The specific utterance may also be a phrase instructing an operation, such as "Turn down the temperature of the air conditioner," "Turn up the volume," or "Turn on the radio."

[0042] If the answer is No in step S52, the in-vehicle system 100 detects the user's movement from the image of the camera 21 and determines whether or not a specific gesture has been detected (S53). Here, the specific gesture is a predetermined action such as "waving a hand," "making a gesture as if operating the operation unit," or "pointing at the operation unit 20."

[0043] If the answer is No in step S53, the in-vehicle system 100 identifies the position of the user's eyes and the position of the operation unit 20 from the image from the camera 21, and determines whether an object such as a steering wheel or luggage is covering the operation unit 20 when viewed from the position of the user's eyes (S54).

[0044] If the answer is No in step S54, the in-vehicle system 100 determines whether the temperature inside the vehicle has reached a predetermined value for performing a push notification, based on the information from the thermosensor 24A (S55).

[0045] If the answer is No in step S55, the in-vehicle system 100 determines whether or not there is traffic information for which push notification is to be performed, based on the information from the car navigation system 50 (S56). If the answer is No in step S56, the in-vehicle system 100 ends the process of FIG.

[0046] If the answer is YES in steps S51 to S56, that is, if a movement trigger is detected, the in-vehicle system 100 performs a movement display process of the display contents as shown in FIG. 7 and FIG. 8 (S57). Then, the in-vehicle system 100 changes the display mode of the display contents based on the input information (S58). FIG. 10 is a diagram showing an example of changing the display mode of the display contents. For example, when the operation unit 20 is rotated by the user, the in-vehicle system 100 rotates the character 61 according to the amount of rotation of the operation unit 20 as shown in A of FIG. 10. Also, when the operation unit 20 is pressed by the user, the in-vehicle system 100 may perform a display such that the character 61 is temporarily moved to the back. Also, when the operation unit 20 is gripped by the user, the in-vehicle system 100 may perform a display such that the width of the character 61 is narrowed as shown in B of FIG. 10. By changing the display mode according to the operation in this way, it is possible to feed back to the user that the operation has been performed, and it is possible for the user to easily understand the operation.

[0047] In addition, when the volume or the set temperature is changed by the operation of the operation unit 20, the brightness or color of the display content 62 may be changed according to the changed value. For example, when the volume is increased, the brightness may be increased according to the amount of increase, and when the volume is decreased, the brightness may be decreased according to the amount of decrease. When the set temperature is increased, the display may be displayed in a warm color such as red, and when the set temperature is decreased, the display may be displayed in a cool color such as blue. In addition, the in-vehicle system 100 may display the same display content as that of FIG. 8C around the operation unit 20 as shown in FIG. 10C, and may rotate in conjunction with the rotation of the operation unit 20. In this case, the numerical value that has moved to the center of the display device 14 (directly above the operation unit 20) due to the rotation may be made effective as the set value. For example, when the display content indicates the temperature setting value of the air conditioner, if "24" representing 24°C is initially displayed in the center of the display device 14, and "23" representing 23°C moves to the center of the display device 14 in conjunction with the rotation of the operation unit 20, 23°C becomes effective as the temperature setting value of the air conditioner. Also, instead of rotating the display content, a cursor or highlighting attached to the display item may be moved. For example, when the cursor is initially placed on "24" representing 24°C in the display content or "24" is highlighted, and then the cursor or highlighting moves to "23" representing 23°C in conjunction with the rotation of the operation unit 20, 23°C becomes effective as the temperature setting value of the air conditioner. Furthermore, a scale 63 may be displayed and rotated together with the operation unit 20, and the amount of rotation of the operation unit 20 may be clearly indicated by the scale 63. It should be noted that, when the moving display control is executed without any operation by the user, such as a push notification, the change of the display mode in step S58 may be skipped.

[0048] After step S58, the in-vehicle system 100 determines whether or not a return condition for the display content is satisfied (S59), for example, when the user's line of sight is off the display device 14 or the display surface 202 for a predetermined period of time or more, or when no operation is performed on the in-vehicle system 100 for a predetermined period of time or more. The in-vehicle system 100 detects the user's line of sight from an image captured by the camera 21, and determines whether or not the line of sight is directed toward the display device 14 or the display surface 202. That is, the camera 21 also functions as a line of sight detection means. Note that the line of sight detection means is not limited to a camera, and may be a line of sight detection sensor.

[0049] If the answer is YES in step S59, the in-vehicle system 100 uses this as a return trigger and performs a process (return process) of returning the display content of the display device 14 to the display surface 202 (S60). In this case, the in-vehicle system 100 displays an operation in which the display content moves from the display device 14 to the display surface 202, which is the opposite of Figures 7 and 8. In this way, the in-vehicle system 100 of this embodiment performs display control so that the display content can move back and forth between the display device 14 and the display surface 202.

[0050] <Effects of the embodiment> As described above, the in-vehicle system 100 of this embodiment includes the control unit 10 that controls the display device (first display) 14 and the second display (display surface 202) provided on the outer surface of the operation unit 20 arranged alongside the display device 14. The control unit 10 moves the display content displayed on the second display from the second display and displays it on the display device 14. In this way, the in-vehicle system 100 of this embodiment moves and displays the display content on the first display, whose display space is not limited by the size of the operation unit 20, thereby improving the visibility of the display content.

[0051] When a user's operation on the operation unit 20 is detected, the control unit 10 moves the display content to the display device 14. As a result, even when the user operates the operation unit 20 and the display content is hidden by the user's hand, the in-vehicle system 100 of this embodiment can improve the visibility of the display content by moving the display content to the display device 14 and displaying it.

[0052] The in-vehicle system 100 of this embodiment further includes an utterance detection means for detecting an utterance by the user, and when a specific utterance by the user is detected, the control unit 10 moves the display content of the second display to the display device 14. This allows the in-vehicle system 100 to move and display the display content using the utterance by the user as a trigger.

[0053] The control unit 10 changes the display mode of the display content displayed on the second display and moves it to the first display, thereby allowing the in-vehicle system 100 to display the display content in an easily visible manner.

[0054] The control unit 10 moves a part of the display content displayed on the second display to the first display, thereby enabling the in-vehicle system 100 to indicate that the display content is moving.

[0055] In the in-vehicle system 100 of this embodiment, the display content may be a character that resembles a face, thereby enabling the in-vehicle system 100 to provide a friendly user interface. When the control unit 10 detects a user's speech to call out to the character 61, it moves the character 61 to the display device 14. This allows the user to operate the in-vehicle system 100 by speaking as if having a conversation with the character 61, and provides an easy-to-understand user interface.

[0056] <Modification> Fig. 11 is a diagram showing a modified example of the operation unit 20X. The operation unit 20X in Fig. 11 includes a display element 211 on the display surface 202 side, and display contents related to the operation of the operation unit 20X are displayed by the control unit 10. The support mechanism 30A supports the operation unit 20X so that it can move toward and away from the display surface 149 of the display device 14. The support mechanism 30A includes an actuator 301, and displaces the operation unit 20X between a state in which it protrudes from the display surface 149 of the display device 14 (first state) and a state in which it is stored in the display device 14 (second state) in response to the control of the control unit 10.

[0057] When the display device 14 is viewed from the front, if a part or the whole of the operation unit 20X is included in the inner area of ​​the display device 14, a notch or hole may be provided in the part of the display device 14 that interferes with the operation unit 20X, and the operation unit 20X may pass through this notch or hole. If the notch or hole in the display device 14 reaches the display element 14A, a notch or hole may be provided in the display element 14A itself. In this case, display is not possible in the part of the display element 14A that corresponds to the notch or hole, but the missing display area can be made up by the display element 211.

[0058] Fig. 12 is a diagram illustrating a drive control process of the operation unit 20X in the in-vehicle system 100. The control unit 10 of the in-vehicle system 100 periodically executes the process of Fig. 12 in parallel with the display control process of Fig. 9, for example, after the in-vehicle system 100 is started up, until it is stopped.

[0059] 12 is started, the in-vehicle system 100 acquires information (state information) indicating the state of the user from the camera 21 and the proximity sensor 29 (S71). For example, the in-vehicle system 100 detects the user's line of sight and the user's posture based on an image from the camera 21. The in-vehicle system 100 also detects, by the proximity sensor 29, that the user's hand has approached the operation unit 20X.

[0060] The in-vehicle system 100 determines whether or not the user is about to operate the operation unit 20X based on the user's state information acquired in step S71 (S72).

[0061] In step S72, if the user is about to operate the operation unit 20X (YES in S72), the in-vehicle system 100 judges whether the operation unit 20X is in the first state (S73). If the operation unit 20X is in the first state (YES in S73), the control unit 10 ends the process of FIG. 12, and if the operation unit 20X is not in the first state (NO in S73), the control unit 10 controls the support mechanism 30A to advance the operation unit 20X to the first state, and moves the display content displayed on the second display (display surface 202) to the first display (display device 14). (S74). Note that the amount by which the tip of the operation unit 20X protrudes from the display surface of the display device 14 when the operation unit 20X is in the first state (protrusion amount) may be preset to a desired protrusion amount depending on the thickness of the user's finger, the length of the nail, etc.

[0062] On the other hand, if the user is not trying to operate the operation unit 20X in step S72 (NO in S72), the in-vehicle system 100 judges whether the operation unit 20X is in the second state (S75). If the operation unit 20X is in the second state (YES in S75), the control unit 10 ends the process of Fig. 12, and if the operation unit 20X is not in the second state (NO in S75), the control unit 10 controls the support mechanism 30A to move the operation unit 20X backward to set it to the second state, and moves the display content displayed on the first display (display device 14) to the first display (display surface 202) (S76).

[0063] In this way, according to this modification, when the user does not perform an operation, the operation unit 20X is stored (second state), and only when the user is about to perform an operation, the operation unit 20X is protruded (first state), providing a smart system. When the user is about to perform an operation, the display content displayed on the second display (display surface 202) is moved to the first display (display device 14), so that the visibility of the display content can be improved even when the display content on the second display is hidden by the user's hand.

[0064] Second Embodiment In the first embodiment described above, the operation of the operation unit or the like is used as a movement trigger to perform the movement display processing of the display contents, but in this embodiment, the configuration is different from the first embodiment in that the movement display processing of the display contents is performed when the user's hand approaches the operation unit 20. Note that since the other configurations are the same as in the first embodiment, the same elements are given the same reference numerals and will not be described again.

[0065] 13 is a diagram illustrating a hardware configuration of an in-vehicle system 100A according to a second embodiment. In addition to the configuration of the first embodiment, the in-vehicle system 100A further includes a proximity sensor (proximity detection means) 29 that detects the approach of a user's hand to the operation unit 20. For example, the distance to the target object is calculated by calculating the relationship between the time required from when an ultrasonic signal is emitted to a detection target area until the signal reflected by the target object is received and the speed of sound. When the control unit 10 detects by the proximity sensor 29 that the user's hand approaches the operation unit 20 and the distance from the hand (target object) to the operation unit becomes less than a predetermined value, the control unit 10 performs a movement display process of the display content.

[0066] Fig. 14 is a diagram illustrating a display control process in the in-vehicle system 100A according to the second embodiment. When the display control process in Fig. 14 is started, the in-vehicle system 100A judges whether or not the user's hand approaches the operation unit 20 (S50). If the answer is Yes in step S50, the in-vehicle system 100A performs a moving display process (S57). The other processes are the same as those in the first embodiment described above.

[0067] The in-vehicle system 100A of this embodiment further includes a proximity detection means for detecting the approach of a user's hand to the operation unit 20, and moves the display content when the approach of the hand is detected. As a result, the in-vehicle system 100A executes a display control process of the display content when the user simply brings his / her hand close to the operation unit 20, and therefore the visibility of the display content can be easily improved.

[0068] <Third embodiment> This embodiment is different from the second embodiment described above in that a touch operation on the display device 14 is detected and the display content is moved to the position where the touch operation was performed. Since the other configurations are the same as those of the second embodiment, the same elements are denoted by the same reference numerals and will not be described again.

[0069] FIG. 15 is a diagram illustrating a hardware configuration of an in-vehicle system 100B according to a third embodiment. In addition to the configuration of the second embodiment, the in-vehicle system 100B further includes a touch panel 14B that detects a touch operation on the display device 14. The touch panel 14B is a transparent panel, and is provided so as to be superimposed on the display surface side (user side) of the display element 14A. When the user touches (touches) the touch panel 14B, the touch panel 14B inputs the touch position to the control unit 10. Note that the touch operation may be a single tap, a double tap, a long press, a flick, a pinch in, a pinch out, or the like, a predetermined number of touches, a predetermined time of touch, or an operation of moving multiple touch points along a predetermined trajectory.

[0070] Fig. 16 is a diagram illustrating a display control process in the in-vehicle system 100B according to the third embodiment. When the display control process in Fig. 16 is started, the in-vehicle system 100B judges whether or not a touch operation on the display device 14 is detected (S41). If the answer is No in step S41, the in-vehicle system 100B proceeds to step S50, and if the answer is YES in step S41, the in-vehicle system 100B moves the display content to the position where the touch operation was performed and displays it (S42). After step S42, the in-vehicle system 100B proceeds to step S58 and changes the display mode. Note that other processes are the same as those in the second embodiment described above.

[0071] Fig. 17 is a diagram showing an example of a process for moving display content to a position where a touch operation has been performed. In A of Fig. 17, a touch operation is performed at a position PX on the display device 14. In this case, the control unit 10 moves and displays the character 61 that has been displayed on the display surface 202 to a position PX on the display device 14 as shown in B of Fig. 17.

[0072] FIG. 18 is a diagram showing an example of a function selection menu, and FIG. 19 is a diagram showing a state in which the function icon in FIG. 18 has been moved. When a user operates the in-vehicle system 100B to display a selection menu, the control unit 10 causes the selection menu to be displayed on the display device 14 and the display surface 202 as shown in FIG. 18. In this case, the control unit 10 moves the character 61 that was displayed on the display surface 202 to the display device 14 and displays it there. In this selection menu, A plurality of functional icons (menu icons) displayed on display 14 and at least one functional icon displayed on display surface 202 are displayed so as to be aligned in a line in a predetermined movement direction. In this embodiment, functional icons F1 to F7 are aligned in the left-right direction of display device 14, and this left-right direction is the movement direction of functional icons F1 to F7. Of the plurality of functional icons F1 to F7, one arranged at the position of operation unit 20 is displayed on display surface 202 of operation unit 20. Note that, in this embodiment, the movement direction (arrangement direction) of functional icons F1 to F7 is a straight line facing left-right, but is not limited thereto, and they may be configured to move vertically or in a curved line.

[0073] When the user performs a slide operation on the functional icons F1 to F7, the control unit 10 moves and displays the functional icons F1 to F7 in response to the slide operation. At this time, one of the functional icons F1 to F7 that has been moved to the position of the operation unit 20 by the slide operation is displayed on the display surface 202, and the function indicated by the functional icon displayed on the display surface 202 is selected. In FIG. 18, the music icon F4 is displayed on the display surface 202, and the music function is selected. When these functional icons F1 to F7 are moved one to the right, the telephone icon F3 is displayed on the display surface 202, and the telephone function is selected. When each of the functional icons F1 to F7 is selected in this manner, the display mode of each of the functional icons F1 to F7 is changed in step S58 of FIG. 16 according to the selected function. In FIG. 18, the music function is selected, and the character 61 is changed to a form in which the character wears headphones so as to enjoy music. Also, in A of FIG. 19, the telephone function is selected, and the character 61 is changed to a form in which the character holds a receiver so as to make a call. Furthermore, in this state, when a touch operation is performed at a position PX on the display device 14, the changed character 61 is displayed at the position where the touch operation was performed, as shown in FIG. 19B.

[0074] In this embodiment, when the character 61 is not displayed or when the operation unit 20 is not provided, the control unit 10 may display a selection menu at the position where a touch operation is performed on the display device 14.

[0075] The in-vehicle system 100B of this embodiment further includes a touch panel (touch detection means) 14B that detects a touch operation on the display device 14 by the user, and moves the display content to the touched position on the display device 14. This allows the in-vehicle system 100B to display the display content at a position selected by the user, thereby improving convenience.

[0076] The in-vehicle system 100B of this embodiment is the same as the second embodiment except for the configuration in which the display content is moved to the position where the touch operation is performed, but may be the same as the first embodiment.

[0077] <Fourth embodiment> This embodiment differs from the third embodiment described above in that it is configured to include multiple sets (display units) of display devices 14 and display surfaces 202. The other configurations are the same as those of the first embodiment, so that the same elements are given the same symbols and will not be described again.

[0078] FIG. 20 is a diagram illustrating a hardware configuration of an in-vehicle system 100C according to a fourth embodiment. The in-vehicle system 100C is equipped with a plurality of display units 140. FIG. 21 is a diagram illustrating the arrangement of the display units in the vehicle. The display unit 140 is a user interface equipped with a set of a display device 14 and a display surface 202, and is provided at each seat in the vehicle at a position where the user can operate it. For example, the display unit 140A provided at the front seat (driver's seat) is embedded in the instrument panel 40 in front of the driver's seat, and the display unit 140B provided at the front seat (passenger's seat) is embedded in the instrument panel 40 in front of the passenger's seat. The display units 140A, 140B of each front seat are mounted on a pillar (A-pillar) in front of each front seat. ) may be embedded. The display unit 140C provided in the seat behind the driver's seat (rear seat) is embedded in the rear surface of the headrest or the rear surface of the seat back of the driver's seat, and the display unit 140D provided in the seat behind the passenger seat (rear seat) is embedded in the rear surface of the headrest or the rear surface of the seat back of the passenger seat. The displays in the rear seats may be embedded in pillars in front of the seats (B pillars in the center of the vehicle) or in the ceiling surface. In FIG. 21, the vehicle has a total of four seats, two seats in the front and two seats in the rear, but this is not limited to this, and the vehicle may have no rear seats or may have multiple rows of rear seats. The in-vehicle system 100C may have a configuration in which all of these seats are provided with the display unit 140, or a configuration in which only some of the seats are provided with the display unit 140. The vehicle is not limited to a passenger car, but may be a bus, a taxi, or the like.

[0079] Moreover, the display unit 140 of this embodiment includes a camera 21, a proximity sensor (approach detection means) 29, a microphone 17, and a speaker 28. Therefore, the user can operate the in-vehicle system 100C and the in-vehicle system 100C can output to the user at each seat in the same manner as in the above-described embodiment.

[0080] When the display unit 140 provided on each seat receives an operation by the user, for example, when it acquires an utterance for performing an operation such as "Hey, XX" via the microphone 17, when it acquires a user's gesture for performing an operation via the camera 21, or when it detects an operation on the operation unit 20, the in-vehicle system 100C provides feedback to the user that the operation has been detected. For example, the in-vehicle system 100C may provide feedback by outputting a voice message such as "An operation has been detected" or "Is there something I can help you with?" from the speaker 28, or by displaying the message by blinking the display surface 202, increasing the brightness of the display surface 202, or by displaying a message on the display device 14.

[0081] Furthermore, when the in-vehicle system 100C of this embodiment receives an operation by the user, the in-vehicle system 100C moves the character 61 to the display device (third display) 14 of the display unit 140 that received the operation, thereby providing feedback that the operation has been received. Note that the in-vehicle system 100C of this embodiment displays the character 61 only on one display surface 202 of the multiple display units 140. Then, when a movement flag is generated by detecting a user's operation on a display unit 140 on which the character 61 is not displayed, the control unit 10 of the in-vehicle system 100C moves the character 61 from the display unit 140 on which the character 61 is displayed to the display unit 140 on which the movement flag is generated. Note that the case where the movement flag is generated is not limited to the case where a user's operation is detected, and may be, for example, the case where the approach of the user's hand to the operation unit 20 is detected, the user's speech is detected, or a touch operation on the display device 14 is detected. That is, among the display units 140 at each seat, the character 61 is moved to the display device 14 of the display unit 140 for which the movement flag has been raised as the movement destination.

[0082] 22 is a diagram showing an example in which a character 61 moves between display units 140 arranged at seats arranged side by side. For example, when a character 61 is displayed on the display surface 202 of display unit 140B at the passenger seat, if a movement flag is generated at display unit 140A at the seat (driver's seat) arranged laterally of the vehicle relative to the passenger seat, control unit 10 moves character 61 from display unit 140B as the movement origin to display unit 140A as the movement destination. At this time, control unit 10 causes display device 14 to display an operation (animation) according to the arrangement of display units 140A and 140B, so that it is possible to know which display unit 140 the character 61 is moving to or from which display unit 140.

[0083] In the case of FIG. 22, the control unit 10 moves the character 61 from the display surface 202 of the display unit 140B, which is the source of the movement, to a predetermined position (directly above the display surface 202) 141 of the display device 14, and then causes the display device 14 to display an action of moving in a direction corresponding to the position of the display unit 140A, which is the destination of the movement. That is, the control unit 10 moves the character 61 from the predetermined position 141 of the display device 14 in the display unit 140B to the right, and causes the character 61 to disappear (move outward) from the side (right side) 14R in the direction in which the display device (third display) of the display unit 140A, which is the destination of the movement, is located. The control unit 10 also displays the character (display content) 61 on the display device 14 of the destination of the movement so that it appears from the side in the direction in which the display device 14 of the source of the movement is located. For example, since the display unit 140B, which is the movement source, is located on the left side of the display unit 140A, the control unit 10 causes the display unit 14 of the display unit 14 to display an action (cut-in) in which the character 61 appears from the left side 14L of the display unit 14 and moves in the direction (rightward) of the predetermined position 141. This allows the in-vehicle system 100C to inform the user in the passenger seat that the character 61 has moved to the display unit 140A of the driver's seat, and to inform the user in the driver's seat that the character 61 has moved from the passenger seat. In addition, the in-vehicle system 100C can feed back to the user in the driver's seat that an operation has been detected by the movement of the character 61. Note that when the character 61 is displayed on the display surface 202 of the display unit 140A of the driver's seat, if a movement flag is generated in the display unit 140B of the passenger seat, the control unit 10 causes the display to be performed so that the character 61 moves in the opposite direction to that shown in FIG. 22. Also, when a character 61 is displayed on the display surface 202 of the display unit 140D on the left side of the rear seat, if a movement flag occurs on the display unit 140C on the right side of the rear seat, the control unit 10 may move and display the character 61 with the display unit 140D as the source and the display unit 140C as the destination, as in FIG. 22.

[0084] 23 is a diagram showing an example in which a character 61 moves between display units 140 arranged in seats arranged in a row. For example, when a character 61 is displayed on the display surface 202 of display unit 140B in the passenger seat, if a movement flag is generated in display unit 140D in the seat arranged in the front-rear direction of the vehicle relative to the passenger seat (left rear seat), control unit 10 moves character 61 from display unit 140B as the movement origin to display unit 140D as the movement destination.

[0085] In the case of FIG. 23, the control unit 10 moves the character 61 from the display surface 202 of the display unit 140B, which is the source, to a predetermined position (directly above the display surface 202) 141 of the display device 14, and then causes the display device 14 of the destination display unit 140A (third display) to display an action of moving toward a side in a certain direction and disappearing. For example, the control unit 10 moves the character 61 upward from the predetermined position 141 of the display device 14 in the display unit 140B, and causes the character 61 to display an action (cutout) of moving toward a side (upper side) 14U corresponding to the position of the display unit 140D and disappearing (moving as if going out). The control unit 10 also displays the character (display content) 61 on the destination display device 14 (third display) so that it appears from a side in a direction in which the display device 14 of the source is located. For example, since the display unit 140B from which the character 61 moves is located in front of the display unit 140D, the control unit 10 causes the display device 14 of the display unit 14 to display an action (cut-in) in which the character 61 appears from the upper side 14U of the display device 14 and moves in the direction (downward) of the predetermined position 141. This enables the in-vehicle system 100C to inform the user in the passenger seat that the character 61 has moved to the display unit 140A on the left side of the rear seat, and to inform the user in the left side of the rear seat that the character 61 has moved from the passenger seat. Note that when the character 61 is displayed on the display surface 202 of the display unit 140D on the left side of the rear seat, When a movement flag is generated in the display unit 140B, the control unit 10 may move and display the character 61 in the same manner as in Fig. 23. Furthermore, when a movement flag is generated in the display unit 140C on the right side of the rear seat while the character 61 is being displayed on the display surface 202 in the display unit 140A on the driver's seat, or when a movement flag is generated in the display unit 140A on the driver's seat while the character 61 is being displayed on the display surface 202 in the display unit 140C on the right side of the rear seat, the control unit 10 may move and display the character 61 in the same manner as in Fig. 23.

[0086] In FIG. 23, when the central axis in the up-down direction on the display surface of the display device 14 is the up-down axis 143, the control unit 10 moves the character 61 from the predetermined position 141 straight up along the up-down axis 143, for example, when moving the character 61 from the display unit 140B to the display unit 140D. In contrast, when moving the character 61 from the display unit 140B to the display unit 140C, the control unit 10 may move the character 61 by tilting the moving direction obliquely upward and to the right from the up-down axis 143, depending on the position of the display unit 140C. Also, when moving the character 61 from the display unit 140C to the display unit 140B, the control unit 10 may move the character 61 from the right part of the upper side 14U obliquely downward and to the left, depending on the position of the display unit 140C, and move it to the predetermined position 141 to display it.

[0087] 23, the control unit 10 moves the character 61 to be cut out so as to go out from the upper side 14U, but the control unit 10 may once move the character 61 upward from the predetermined position 141, then make a U-turn and move downward, and display the character 61 to go out from the lower side 14D. The control unit 10 may also move the character 61 to be cut in upward from the lower side 14D, then make a U-turn and move downward, and display the character 61 to be moved to the predetermined position 141.

[0088] FIG. 24 is a diagram showing an example in which a display unit 140A on the driver's seat side and a display unit 140B on the passenger's seat side are arranged on a pillar 41. When the display units 140A and 140B are arranged on the pillar as shown in FIG. 24, as in the case where the display units 140A and 140B are arranged on the pillar, it is possible to accept an operation by a user (driver) in the driver's seat and provide feedback to the user, as well as to accept an operation by a user in the passenger's seat and provide feedback to the user, in the same manner as in the case where the display units 140A and 140B are arranged on the instrument panel 40 as described above. In addition to the display units 140A and 140B arranged on the instrument panel 40, an additional display device may be arranged on the pillar as in FIG. 24, and feedback that an operation has been detected may be displayed on the display device on the pillar. In this case, the control unit 10 may move the character 61 to the destination display device 14, and then move it further to the display device on the pillar and display it.

[0089] 25 is a diagram showing an example in which display devices 142A-142D are provided on the ceiling surface. The display devices 142A-142D are additional display devices provided on the ceiling surface of each seat separately from the display units 140A-140D of each seat. The display devices 142A-142D may also be projection devices that project a display image. When the control unit 10 detects an operation by a user at each seat, it causes the display devices 142A-142D of the seat to display an indication that the operation has been detected, and provides feedback to the user at each seat that the operation has been detected.

[0090] FIG. 26 is a diagram showing an example in which projection devices 143A-143D that project images onto a floor surface are provided. The projection devices 143A-143D are a form of additional display devices provided on the ceiling surface of each seat separately from the display units 140A-140D of each seat. The projection devices 143A-143D may also be projection devices that project display images. When the control unit 10 detects an operation by a user at each seat, it notifies the projection devices 143A-143D of the seat to display the operation. A display is displayed indicating that an operation has been detected, and feedback is provided to the user in each seat that an operation has been detected.

[0091] The in-vehicle system 100C of this embodiment is the same as the third embodiment except for the configuration including the multiple display units 140, but may be the same as the first or second embodiment. Also, in this embodiment, the movement of the character 61 has been mainly described, but the display content to be moved is not limited to the character 61, and may be any display content related to the operation of the operation unit 20, such as the set temperature of the air conditioner or an icon of a selected function.

[0092] As described above, the in-vehicle system 100C of this embodiment can improve the visibility of the display contents by moving the display contents of the operation unit 20 to the display device and displaying them. In addition, the in-vehicle system 100C of this embodiment can prevent discrepancies between the behavior of the in-vehicle system 100C that receives the operation and the recognition of the user who performed the operation by feeding back to the user that the operation by the user has been detected.

[0093] Furthermore, the in-vehicle system 100C of this embodiment can provide a user-friendly system in which a character 61 that is displayed on one of the multiple display units 140 can be moved to the display unit 140 on which the movement flag has been generated and displayed, so that a character 61 that lives somewhere on the multiple display units 140 can be summoned by the user and moves around the system.

[0094] In this embodiment, the character 61 is configured to be displayed on one of the plurality of display units 140, but the character 61 may be displayed on some or all of the plurality of display units 140. For example, the character 61 may be displayed on one of the display units 140A, 140B in the front seat and one of the display units 140C, 140D in the rear seat.

[0095] In addition, as shown in FIG. 11, when the support mechanism 30A supports the operation unit 20 so that it can move forward and backward, the operation unit 20 may be in a first state on the display unit 140 on which a character is displayed, and the operation unit 20 may be in a second state on the display unit 140 on which a character is displayed.

[0096] <Other> In the above embodiment, the in-vehicle system 100 is taken as an example to illustrate a computer system that executes a process of moving the display contents on the display surface 202 of the operation unit 20. However, the configuration of the above embodiment is not limited to an in-vehicle system. In other words, the configuration of the above embodiment can be applied to various systems that function as interactive systems, such as general personal computers, multimedia terminals in convenience stores, game machines, etc.

[0097] <Computer-readable recording medium> A program for causing a computer or other machine or device (hereinafter, referred to as a computer, etc.) to realize any of the above functions can be recorded on a recording medium readable by the computer, etc. Then, the computer, etc. can provide the function by reading and executing the program from the recording medium.

[0098] Here, a computer-readable recording medium refers to a recording medium that stores information such as data and programs electrically, magnetically, optically, mechanically, or chemically and can be read by a computer. Examples of such recording media that can be removed from a computer include flexible disks, magneto-optical disks, CD-ROMs, CD-R / Ws, DVDs, Blu-ray disks, DATs, 8mm tapes, flash drives, etc. Examples of recording media that are fixed to a computer include hard disks and ROMs (read-only memories). In addition, SSDs (Solid State Drives) are also used as recording media that can be removed from a computer. The present invention can also be used as a recording medium fixed to a computer. [Explanation of symbols]

[0099] 10: Control section 11: CPU 12: Main memory 13:External storage device 14:Display device 14A: Display element 14B: Touch panel 14D: Bottom 14L: Left side 14U: Top 14R: Right side 15: Input section 16: Communication equipment 17: Microphone 20,20X:Operation unit 21: Camera 22: Fingerprint sensor 23: Touch sensor 24A: Thermosensor 24B: Infrared sensor 25: Dial switch 26: Push switch 27: Illuminance sensor 28: Speaker 29: Proximity sensor 30,30A: Support mechanism 40: Instrument panel 41: Pillar 50: Car navigation system 61: Character 62:Display content 100, 100A, 100B, 100C: In-vehicle systems 140, 140A, 140B, 140C, 140D: Display unit 201: Housing 202:Display surface 203: Optical fiber 204: Ring 211: Display element 301: Actuator

Claims

1. a first display and an operation unit arranged alongside the first display; A second display provided on at least one surface of the operation unit; and a control unit that moves and displays the display content displayed on the second display from the second display to the first display.

2. The information processing apparatus according to claim 1 , wherein the control unit moves the display content from the second display to the first display when a user's operation on the operation unit is detected.

3. The device further includes an approach detection unit that detects an approach of a user's hand to the operation unit, The information processing device according to claim 1 , wherein the control unit moves the display content from the second display to the first display when the approach of the hand is detected.

4. The device further includes a speech detection means for detecting a speech by a user, The information processing apparatus according to claim 1 , wherein the control unit moves the display content from the second display to the first display when a specific utterance by the user is detected.

5. a touch detection unit configured to detect a touch operation on the first display by a user; The information processing device according to claim 1 , wherein the control unit moves the display content from the second display to a touched position on the first display.

6. The information processing device according to claim 5 , wherein the control unit causes a menu icon to be displayed as the display content at a touched position on the first display.

7. The information processing device according to claim 1 , wherein the control unit, when moving the display content displayed on the second display to the first display, changes the display mode of the display content and displays it on the first display.

8. The information processing apparatus according to claim 1 , wherein the control unit moves a part of the display content displayed on the second display to the first display.

9. 9. The information processing device according to claim 1, wherein the display content is a character resembling a face.

10. The information processing device according to claim 9 , wherein the control unit moves the character from the second display to the first display when an utterance of a user calling out to the character is detected.

11. The information processing device according to claim 9 , wherein the control unit, when a predetermined function is selected while the character is displayed on the first display, causes an icon related to the selected function to be displayed on the second display.

12. The information processing device according to claim 11 , wherein the control unit changes a display mode of the character displayed on the first display in accordance with a selected function.

13. The control unit is displaying a plurality of icons on the first display and at least one icon on the second display so as to be aligned in a line in a predetermined moving direction; moving the plurality of icons in the moving direction in response to a user's sliding operation on the icon; Selecting a function indicated by an icon displayed on the second display after the movement; The information processing apparatus according to claim 11 , wherein a display mode of the character moved to the first display is changed in accordance with a selected function.

14. Further comprising a third display; The information processing apparatus according to claim 1 , wherein the control unit moves the display content from the second display to the first display, and moves the display content from the first display to a third display.

15. the first display and the third display are provided to correspond to seats arranged in a lateral direction of the vehicle, The control unit is When the display content is moved, the display content is displayed on the first display so as to move toward a side in a direction in which the third display is located and disappear, and then the display content is displayed on the third display so as to appear from a side in a direction in which the first display is located. The information processing device according to claim 14.

16. the first display and the third display are provided to correspond to seats arranged in a front-rear direction of the vehicle, The control unit is When the display content is moved from the first display to the third display, the display content is displayed so that it moves toward an upper side of the first display and disappears, and then appears from an upper side of the third display. The information processing device according to claim 14.

17. The information processing device according to claim 15 , wherein the control unit moves the display content from the first display to the third display when a user operation on the third display is detected.

18. a control unit that controls a first display and a second display provided on at least one surface of an operation unit arranged alongside the first display; The control unit moves the display content displayed on the second display from the second display to the first display and displays the same thereon. Information processing device.

19. The information processing device of any one of claims 1 to 8, 14 to 16 and 18, wherein the control unit moves and displays the display content being displayed on the second display from the second display to the first display in response to detection of any one of a user's operation on the operation unit, the user's approach to the operation unit, a user's speech, or a user's touch operation on the first display.

20. a control unit that controls a first display and a second display provided on at least one surface of an operation unit arranged alongside the first display, The display contents displayed on the second display are displayed on the front side of the second display. An information processing method for moving and displaying the information on the first display.

Images