Information processing device and program

Abstract

To provide an information processing apparatus that improves operability in operation using a touch panel to improve convenience for a user.SOLUTION: An information processing apparatus 50 comprises: an operation detection unit 61 that, when detecting a plurality of touch positions on the basis of, coordinate information input from a touch panel 40, determines that it detects a first touch operation, calculates a length of a virtual line segment connecting the plurality of touch positions detected as the first touch operation or a rotation angle with respect to a reference direction of the virtual line segment, when the coordinate information is input from the touch panel 40 on the touch panel 40 displaying a second screen, determines whether there is a change in the length or the rotation angle of the virtual line segment on the basis of, the input coordinate information, and when determining that there is a change in the length or the rotation angle of the virtual line segment, determines that it detects a second touch operation; and an execution instruction unit 65 that instructs execution of processing associated with the change in the length of the virtual line segment or the direction of rotation of the virtual line segment.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device and a program. [Background technology]

[0002] 2. Description of the Related Art Conventionally, information processing devices that include a touch panel as a receiving unit that receives input from a user are known. For example, Patent Document 1 discloses an input device for a portable electronic device that includes a touch panel having an input function by touching the input surface, a display unit that is superimposed on the input surface of the touch panel and capable of displaying visible information related to the input operation, and an input control unit that instructs processing of the input signal based on the input signal from the touch panel. This input device is said to enable the user to operate the touch panel while viewing the visible information of the operation buttons on the touch panel, by having the display unit display operation buttons as visible information related to the input operation. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2012-150849 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in order to input multiple commands by operating a touch panel, the user needs to remember the correspondence between the touch panel operations and the commands input by these operations, and there is room for improvement in terms of convenience.

[0005] The present invention has been made in consideration of the above circumstances, and aims to provide an information processing device and a program that improve the operability of operations using a touch panel and improve user convenience. [Means for solving the problem]

[0006] In order to achieve the above object, an information processing device of the present invention includes an operation detection unit that detects, based on coordinate information indicating a touch position input from a touch panel, a first touch operation on the touch panel, the first touch operation being an operation of a user bringing two fingers into contact with the touch panel, and a second touch operation different from the first touch operation, the second touch operation being an operation of moving the two fingers touching the touch panel while keeping them in contact with the touch panel; a display control unit that displays, on the touch panel, a first screen that is a screen that is first displayed on the touch panel after a power supply is turned on, the first screen displaying operation items and operators including a button, switch, or slider for operating the operation items with one of the user's fingers, and when the first touch operation is detected by the operation detection unit, changes the display on the touch panel from the first screen to a second screen displaying a circular figure whose diameter indicates a first operation amount and whose pointer provided near the circumference indicates a second operation amount; and an execution instructing unit that, when the second touch operation is detected by the operation detection unit on the touch panel displaying the second screen, instructs to execute a process associated with the detected second touch operation, wherein the operation detection unit determines that the first touch operation has been detected when two contact points are detected on the touch panel displaying the first screen based on the coordinate information, calculates a length of a virtual line segment connecting the two contact points on the touch panel and a rotation angle of the virtual line segment with respect to a reference direction based on the coordinate information, displays the circular figure on the second screen based on the coordinate information, and determines that the second touch operation has been detected when it is determined that there is a change in the length or the rotation angle of the virtual line segment, and the circular figure is always displayed as the circular figure with the virtual line segment as its diameter even if the two contact points move on the touch panel, and the execution instructing unit changes the diameter of the circular figure in correspondence with the length of the virtual line segment, and Change in and instructs execution of a process corresponding to the first operation amount indicated by the rotation angle Change inThe position of the pointer is changed in response to the second amount of operation, and an instruction is given to execute a process corresponding to the second amount of operation indicated by the position of the pointer. Effect of the Invention

[0007] According to the present invention, the operability of operations using a touch panel can be improved, and user convenience can be improved. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing a configuration of an information processing device. [Diagram 2] FIG. 11 is a diagram showing an example of a first menu screen. [Diagram 3] 11A to 11C are diagrams illustrating a method of operating the first menu screen by a touch operation. [Figure 4] FIG. 13 is a diagram showing an example of a second menu screen. [Diagram 5] FIG. 13 is a diagram showing a state in which an occupant touches the touch panel with two fingers. [Figure 6] FIG. 13 is a diagram showing the positional relationship between a circular figure and a pointer. [Figure 7] FIG. 13 is a diagram showing a state in which two fingers are placed apart. [Figure 8] FIG. 13 is a diagram showing a state in which the distance between two fingers is narrowed. [Figure 9] FIG. 13 is a diagram showing the state in which the pointer has been rotated left. [Figure 10] FIG. 13 is a diagram showing the state in which the pointer has been rotated to the right. [Figure 11] FIG. 13 is a diagram showing a case where two fingers touch the touch panel diagonally, rather than parallel to the horizontal direction. [Figure 12] 4 is a flowchart showing an operation of the information processing device. [Figure 13] FIG. 13 is a diagram showing another example of the second menu screen. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, embodiments will be described with reference to the accompanying drawings. FIG. 1 is a diagram showing an example of a system configuration of the present embodiment. 1 is a system mounted on a vehicle equipped with a plurality of electrical components 80, and receives an operation by a passenger in the vehicle, and changes the settings of the electrical components corresponding to the received operation in response to the received operation. The electrical components 80 correspond to devices to be operated. In this embodiment, the vehicle is described as having a steering wheel provided on the right side of the vehicle.

[0010] The system 1 includes a camera 10, an operation unit 20, an audio output unit 30, a touch panel 40, and an information processing device 50. The information processing device 50 includes an input / output interface 51 and is connected to an electrical component 80 mounted on the vehicle via the input / output interface 51. In Fig. 1, the electrical components 80 are shown as electrical components 80A, 80B, and 80C, but the number of electrical components mounted on the vehicle and operated by the information processing device 50 is not limited to three, namely, electrical components 80A, 80B, and 80C. Hereinafter, the interface is abbreviated as I / F. Hereinafter, when the electrical components 80A, 80B, and 80C are collectively referred to, they will be referred to as electrical components 80. The information processing device 50 and the electrical equipment 80 are connected by a cable that complies with communication standards such as CAN (Controller Area Network) or Ethernet (registered trademark), and the input / output interface 51 is equipped with a connector and a processing circuit that are compatible with standards such as CAN and Ethernet.

[0011] The camera 10 is mounted in the passenger compartment of the vehicle and captures images of passengers in the vehicle. For example, the camera 10 is installed on either the left or right end of the windshield or the A-pillar, and captures images of the driver sitting in the driver's seat and the passenger sitting in the passenger seat. The camera 10 includes an image sensor such as a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal-Oxide-Semiconductor), and a data processing circuit that generates a captured image based on the light receiving state of the image sensor. The camera 10 outputs the captured image generated by capturing images to the information processing device 50.

[0012] The operation unit 20 functions as a reception unit that receives operations from the occupant. The operation unit 20 includes, for example, hardware buttons and switches.

[0013] The audio output unit 30 includes a DAC (Digital Analog Converter), an amplifier, and a speaker, and functions as a notification unit that executes a notification operation. The DAC, the amplifier, and the speaker are not shown in the drawings. The audio output unit 30 converts a digital audio signal input from the information processing device 50 into an analog signal, amplifies the converted analog signal using an amplifier, and outputs the amplified signal from a speaker.

[0014] The touch panel 40 includes a display panel 41 such as a liquid crystal panel or an organic EL (electro-luminescence) panel, and a touch sensor 43. The touch sensor 43 detects a touch operation on the touch panel 40 by a passenger. The touch sensor 43 detects the touch position where the occupant's finger touches the display panel 41, and outputs coordinate information indicating the touch position to the information processing device 50. A coordinate system is set in advance on the display panel 41, and the touch sensor 43 outputs coordinate information indicating the position on the display panel 41 where the occupant's finger touches to the information processing device 50. In this embodiment, a case will be described in which the operator that operates the touch panel is the occupant's finger, but the operator may also be a pointer such as a stick.

[0015] The information processing device 50 is a computer device including an input / output I / F 51, a memory 53, and a processor 60. The processor 60 operates in accordance with a control program 55 stored in the memory 53, thereby controlling the operation of the information processing device 50.

[0016] The memory 53 includes semiconductor memories such as a RAM (Random Access Memory) and a ROM (Read Only Memory). The RAM is used for temporary storage of various data, and the ROM stores a control program 55 for controlling the operation of the information processing device 50, various setting information, and the like.

[0017] The processor 60 is an arithmetic processing device composed of a CPU (Central Processing Unit) and an MPU (Micro Processing Unit). The processor 60 executes the control program 55 to control each part of the information processing device 50. The processor 60 may be composed of a single processor, or may be composed of multiple processors. The processor 60 may be composed of a SoC (System-on-a-chip) integrated with a part or all of the memory 53 and other circuits. The processor 60 may be composed of a combination of a CPU that executes a program and a DSP that executes a predetermined arithmetic processing. Furthermore, all of the functions of the processor 60 may be implemented in hardware, or may be composed of a programmable device.

[0018] The information processing device 50 includes, as functional components, an operation detection unit 61, a display control unit 63, and an execution instruction unit 65. These functional components are functions realized by the processor 60 performing calculations in accordance with the control program 55.

[0019] The operation detection unit 61 detects a touch operation based on coordinate information input from the touch panel 40. The touch operations detected by the operation detection unit 61 include a first touch operation and a second touch operation.

[0020] The first touch operation is an operation in which the passenger touches the touch panel 40 with two fingers. When the coordinates of two points are input from the touch panel 40 as coordinate information indicating the touch positions, the operation detection unit 61 determines that the first touch operation has been detected.

[0021] The second touch operation is an operation in which, while keeping the two fingers that were touched to the touch panel 40 by the first touch operation in a constant space between the two fingers, the two fingers are rotated around a center of rotation or the space between the two fingers is widened or narrowed. The operation detection unit 61 sets a virtual line segment connecting two points indicated by the coordinate information of the two points detected as the first touch operation. After that, when the operation detection unit 61 detects that the length of the virtual line segment has been changed or that the virtual line segment has been rotated based on the coordinate information of the two points input from the touch panel 40, it determines that a second touch operation has been detected.

[0022] The display control unit 63 generates image data and causes a menu screen, which is an image based on the generated image data, to be displayed on the touch panel 40. The menu screens that the information processing device 50 causes to be displayed on the touch panel 40 include a first menu screen 100 and a second menu screen 200. The first menu screen 100 corresponds to the first screen, and the second menu screen 200 corresponds to the second screen.

[0023] The execution instruction unit 65 instructs the target electrical component 80 to execute a process corresponding to the second touch operation detected by the operation detection unit 61.

[0024] FIG. 2 is a diagram showing an example of the first menu screen 100. As shown in FIG. When the occupant operates the ignition key and the accessory power of the vehicle is turned on, the display control unit 63 causes the touch panel 40 to display the first menu screen 100. The first menu screen 100 is the screen that is first displayed on the touch panel 40 after the accessory power is turned on, and therefore can also be called the initial screen.

[0025] The first menu screen 100 has a plurality of screens for each of the electrical components 80 whose settings can be changed by operating the first menu screen 100. For example, the first menu screen 100 has a plurality of screens, such as a first menu screen 100 for changing the settings of an air conditioner, a first menu screen 100 for changing the settings of an audio device, a first menu screen 100 for changing the settings of a car navigation device, and the like. The electrical components 80 whose settings can be changed by operating the first menu screen 100 include, in addition to those mentioned above, for example, an electric seat and the touch panel 40 itself. The driving mode of the vehicle may also be changed by the first menu screen 100. The driving modes include, for example, an eco mode that emphasizes low fuel consumption, a sport mode that emphasizes acceleration, and a normal mode that is intermediate between the eco mode and the sport mode. In addition, by operating the switches and buttons provided on the operation unit 20, the display of the first menu screen 100, that is, the electrical components 80 whose settings can be changed by operating the first menu screen 100, can be changed.

[0026] FIG. 2 shows a first menu screen 100 for changing the settings of the air conditioner. On the first menu screen 100 for manipulating the settings of the air conditioner, for example, a display for changing the volume of air output from the air conditioner, a display for changing the set temperature of the air conditioner, etc. are displayed. In addition, the first menu screen 100 for changing the settings of the air conditioner displays a button 107 for switching the power of the air conditioner on and off, an indication 109 of the set temperature of the air conditioner, an indication 111 of the air outlet for the air output from the air conditioner, and a button 113 for switching between internal air circulation and external air introduction. Moreover, on the first menu screen 100 for manipulating the settings of the audio device, for example, a song selection button, a volume adjustment button, a radio station selection button, and the like are displayed.

[0027] Operation items and controls for operating these operation items are displayed on the first menu screen 100. The controls include buttons, switches, slider bars, sliders, etc. The first menu screen 100 for changing the settings of the air conditioner shown in Fig. 2 displays set temperature 101A and air volume 101B of the air conditioner as operation items. Slider bar 103A and slider 105A are displayed as controls for changing set temperature 101A, and slider bar 103B and slider 105B are displayed as controls for changing air volume 101B.

[0028] FIG. 3 is a diagram showing a method of operating the first menu screen 100 by a touch operation. When the occupant wishes to change the setting of an operation item on the first menu screen 100, the occupant places one finger on the display position of the slider 105A or 105B of the operation item whose setting is to be changed, and moves the finger left and right along the slider bar 103A or 103B. The display control unit 63 changes the display position of the slider 105A or 105B to coordinates indicated by the coordinate information input from the touch panel 40. Furthermore, when the execution instruction unit 65 detects that the finger that was in contact with the touch panel 40 has been removed from the touch panel 40, it outputs an instruction to the air conditioner to change the setting temperature or air volume based on the position of the slider 105A or 105B at the time the finger was removed.

[0029] Fig. 4 is a diagram showing an example of the second menu screen 200. Fig. 5 is a diagram showing a state in which an occupant has two fingers in contact with the touch panel 40. The display control unit 63 changes the display on the touch panel 40 from the first menu screen 100 to the second menu screen 200 when the first touch operation is detected. The second menu screen 200 is also a screen for changing the settings of the electrical components 80. In this embodiment, a case will be described where the second menu screen 200 is an operation screen for changing the settings of the same electrical components 80 as the first menu screen 100 that was displayed before the second menu screen 200 was displayed. For example, if the first menu screen 100 was a screen for changing the settings of an air conditioner, the second menu screen 200 will also be a screen for changing the settings of the air conditioner. Furthermore, the second menu screen 200 may be an operation screen for always changing the settings of the electrical components 80 that have been previously set, regardless of the first menu screen 100.

[0030] An operation image 210 and additional information 230 are displayed on the second menu screen 200. The operation image 210 includes a circular figure 201, contact points 203A and 203B, a range figure 205, and a pointer 207. The incidental information 230 includes the operation items of the electrical equipment 80 whose settings are changed by operating the operation image 210, and the current settings thereof. For example, if the electrical equipment 80 is an air conditioner, the air volume and the set temperature of the air conditioner are displayed as the incidental information 230.

[0031] The contact points 203A and 203B are figures that indicate the positions of the fingers of the occupant who performed the first touch operation on the touch panel 40. The contact points 203A and 203B are displayed at the contact positions on the touch panel 40 where the occupant has made contact with two fingers.

[0032] Circular figure 201 is a circular figure whose diameter is the line segment connecting contact point 203A and contact point 203B. Even if the two fingers of an occupant touching touch panel 40 are moved on touch panel 40, circular figure 201 is always displayed as a circular figure whose diameter is the line segment connecting contact point 203A and contact point 203B, which are the contact positions of the two fingers.

[0033] The range graphic 205 is a graphic that indicates a setting range corresponding to the operation item of the electrical component 80 that is the target of operation in the operation image 210. The range graphic 205 is displayed above the circular graphic 201 at a position a certain distance away from the circular graphic 201. The range of the range graphic 205 on the right side as one faces the second menu screen 200 indicates the range for increasing the temperature, and the range of the range graphic 205 on the left side as one faces the second menu screen 200 indicates the range for decreasing the temperature. One end of the needle 207 is in contact with the circular figure 201, and when the circular figure 201 is rotated by an operation by the passenger, the needle 207 also rotates in conjunction with the rotation of the circular figure 201. The rotation of the needle 207 also changes the position of the range figure 205 indicated by the needle 207. When the needle 207 accepts a second touch operation, the position of the range figure 205 indicated by the needle 207 indicates the amount of the accepted second touch operation.

[0034] When the second menu screen 200 is displayed on the touch panel 40, the occupant performs a second touch operation while keeping two fingers touching the touch panel 40. The second touch operation includes an operation of rotating the two fingers around a rotation center while keeping the distance between the two fingers constant, and an operation of changing the distance between the two fingers to wider or narrower.

[0035] 6 is a diagram showing a virtual line segment 221 and an angle θ between the virtual line segment 221 and the horizontal lateral direction of the touch panel 40. In this embodiment, the angle θ is an angle in which a left rotation angle is positive and a right rotation angle is negative. In addition, a line segment 223 shown in FIG. 6 is a line segment that indicates the horizontal lateral direction of the touch panel 40. First, the operation detection unit 61 sets a virtual line segment 221 connecting two points indicated by the coordinate information detected as the first touch operation. The operation detection unit 61 calculates the length X of this virtual line segment 221 and a rotation angle θ from the horizontal direction of the touch panel 40. The length X of the virtual line segment 221 calculated from the coordinate information of the first touch operation is referred to as a first length X1, and the rotation angle θ of the virtual line segment 221 calculated from the coordinate information of the first touch operation is referred to as a first rotation angle θ1. Next, when the occupant performs a second touch operation and new coordinate information is input from the touch panel 40, the operation detection unit 61 calculates a new length X and rotation angle θ of the virtual line segment 221 based on the input coordinate information of the two points. The length X of the virtual line segment 221 calculated from the new coordinate information by the second touch operation is referred to as a second length X2, and the rotation angle θ of the virtual line segment 221 calculated from the new coordinate information by the second touch operation is referred to as a second rotation angle θ2. The operation detection unit 61 compares the first length X1 and the first rotation angle θ1 with the second length X2 and the second rotation angle θ2 to determine whether or not a second touch operation has been detected.

[0036] When the second length X2 of the virtual line segment 221 is longer than the first length X1, the operation detection unit 61 determines that an operation of spreading the space between two fingers has been detected as a second touch operation. In this case, the operation detection unit 61 calculates the difference between the second length X2 and the first length X1, and outputs the calculated difference to the execution instruction unit 65. Furthermore, when the second length X2 of the virtual line segment 221 is shorter than the first length X1, the operation detection unit 61 determines that an operation of narrowing the distance between two fingers has been detected as the second touch operation. In this case as well, the operation detection unit 61 calculates the difference between the second length X2 and the first length X1, and outputs the calculated difference to the execution instruction unit 65.

[0037] Furthermore, when the second rotation angle θ2 of the virtual line segment 221 is greater than the first rotation angle θ1, the operation detection unit 61 determines that an operation of rotating two fingers to the left has been detected as the second touch operation. In this case, the operation detection unit 61 calculates the difference between the second rotation angle θ2 and the first rotation angle θ1, and outputs the calculated difference to the display control unit 63 and the execution instruction unit 65. Furthermore, when the second rotation angle θ2 of the virtual line segment 221 is smaller than the first rotation angle θ1, the operation detection unit 61 determines that an operation of rotating two fingers to the right has been detected as the second touch operation. In this case, the operation detection unit 61 calculates the difference between the second rotation angle θ2 and the first rotation angle θ1, and outputs the calculated difference to the display control unit 63 and the execution instruction unit 65.

[0038] FIG. 7 shows a case where two fingers are placed in contact with the touch panel 40 with a wider gap between them, and FIG. 8 shows a case where the two fingers are placed in contact with the touch panel 40 with a narrower gap between them. The display control unit 63 changes the display positions of the contact points 203A and 203B based on the coordinate information detected as the second touch operation. When the occupant performs an operation to spread the distance between the two fingers, the diameter of the circular figure 201 is also changed to a larger value in accordance with the distance between the two fingers as shown in FIG. Furthermore, when the occupant performs an operation of narrowing the distance between the two fingers, the diameter of the circular figure 201 is also changed to be smaller in accordance with the narrowed distance between the two fingers as shown in FIG.

[0039] In this embodiment, it is assumed that the operation of spreading two fingers apart corresponds to an operation of increasing the airflow of the air conditioner, and the operation of narrowing the distance between two fingers corresponds to an operation of decreasing the airflow of the air conditioner. When the execution instruction unit 65 receives information indicating the difference between the second length X2 and the first length X1 from the operation detection unit 61, the execution instruction unit 65 calculates the ratio of the input difference to a preset reference value. The reference value is a value that associates the difference between the second length X2 and the first length X1 with the amount by which the air volume of the air conditioner is increased or decreased. The execution instruction unit 65 calculates the air volume to be increased or decreased for the air conditioner based on the ratio between the input difference and the reference value, and instructs the air conditioner to change to the calculated air volume.

[0040] FIG. 9 shows a state in which the pointer 207 has been rotated left as seen by the passenger through the second touch operation, and FIG. 10 shows a state in which the pointer 207 has been rotated right as seen by the passenger. In this embodiment, it is assumed that the operation of rotating two fingers to the left around the center of rotation while keeping the distance between the two fingers constant corresponds to the operation of lowering the temperature setting of the air conditioner, and the operation of rotating two fingers to the right around the center of rotation while keeping the distance between the two fingers constant corresponds to the operation of raising the temperature setting of the air conditioner.

[0041] When the difference between the second rotation angle θ2 and the first rotation angle θ1 is input from the operation detection unit 61, the display control unit 63 changes the display position of the pointer 207 relative to the range graphic 205 based on the input difference. When the second menu screen 200 is displayed on the touch panel 40 , the pointer 207 is displayed in the center of the range graphic 205 . The display control unit 63 changes the display position of the needle 207 relative to the range graphic 205 based on the difference between the second rotation angle θ2 and the first rotation angle θ1. When the difference between the second rotation angle θ2 and the first rotation angle θ1 is a positive value, the display control unit 63 rotates the display position of the needle 207 left based on the difference, as shown in FIG. Furthermore, when the difference between the second rotation angle θ2 and the first rotation angle θ1 is a negative value, the display control unit 63 rotates the display position of the needle 207 to the right based on the difference, as shown in FIG.

[0042] When information indicating the difference between the second rotation angle θ2 and the first rotation angle θ1 is input from the operation detection unit 61, the execution instruction unit 65 calculates the ratio of the input difference to a preset reference value. The reference value is a value that associates the difference between the second rotation angle θ2 and the first rotation angle θ1 with the amount by which the temperature setting of the air conditioner is to be increased or decreased. The execution instruction unit 65 calculates the temperature by which to increase or decrease the temperature setting of the air conditioner based on the ratio between the input difference and a reference value, and instructs the air conditioner to change to the calculated temperature.

[0043] FIG. 11 is a diagram showing a case where an occupant performs a first touch operation by touching the touch panel 40 with two fingers diagonally, rather than parallel to the horizontal direction. For example, when touch panel 40 is disposed in the center of the vehicle in the width direction and disposed at a position away from the driver in the driver's seat, as shown in Fig. 11, virtual line segment 221 connecting two touching fingers may be inclined with respect to the horizontal lateral direction of touch panel 40. Fig. 11 shows a state in which virtual line segment 221 connecting two touching fingers is inclined by angle θ with respect to the horizontal lateral direction of touch panel 40. In this case, the display control unit 63 displays the pointer 207 in a state inclined at an angle θ with respect to the vertical direction of the touch panel 40. This allows the state where the occupant first touches the touch panel 40 with two fingers to be the initial state, and the air volume and temperature settings can be changed in response to the subsequent movement of the two fingers.

[0044] Additionally, on the second menu screen 200, additional information 230 is displayed. For example, when the operation target of the operation image 210 is an air conditioner, information such as the set temperature and air volume is displayed as the incidental information 230. The display position of the incidental information 230 is preferably a position that does not overlap with the operation image 210 and is easily visible to the occupant operating the operation image 210.

[0045] For example, assume that the area of ​​the touch panel 40 seen from the passenger is divided into three areas, a left area, a right area, and a center area, and the passenger touches two fingers on the left area of ​​the touch panel 40. In this case, the operation image 210 is displayed in the left area and the auxiliary information 230 is displayed in the right area, with the coordinates where the touch of the two fingers is detected as the center. Also, assume that the occupant touches two fingers to the right region of the touch panel 40. In this case, an operation image 210 is displayed in the right region with the coordinates where the touch of the two fingers is detected as the center, and additional information 230 is displayed in the left region.

[0046] In addition, when displaying the supplementary information 230, the information processing device 50 may analyze the image captured by the camera 10 and determine whether the touch panel 40 is being operated by a passenger in the passenger seat or by the driver in the driver's seat. When the information processing device 50 determines that an occupant in the passenger seat is operating the touch panel 40 and the central region of the touch panel 40 is touched, the information processing device 50 displays the additional information 230 in the right region of the touch panel 40 on the driver's seat side. When the information processing device 50 determines that a driver in the driver's seat is operating the touch panel 40 and the central region of the touch panel 40 is touched, the information processing device 50 displays the additional information 230 in the left region of the touch panel 40 on the passenger seat side. This is to prevent the display on the touch panel 40 from being hidden by the hand or the like during operation. Alternatively, the central area of ​​touch panel 40 may be divided into an upper area and a lower area, and when coordinates of the upper central area of ​​touch panel 40 are detected as coordinates indicating the touch position, additional information 230 may be displayed in the lower central area. Alternatively, when coordinates of the lower central area of ​​touch panel 40 are detected as coordinates indicating the touch position, additional information 230 may be displayed in the upper central area.

[0047] FIG. 12 is a flowchart showing the operation of the information processing device 50. The operation of the information processing device 50 will be described with reference to the flowchart shown in FIG. First, the information processing device 50 determines whether or not coordinate information of a touch operation has been input from the touch panel 40 (step S1). If the coordinate information of a touch operation has not been input (step S1 / NO), the information processing device 50 does not proceed to the next step until the coordinate information of a touch operation is input. Step S1 corresponds to a detection step.

[0048] When the coordinate information of the touch operation is input from the touch panel 40 (step S1 / YES), the information processing device 50 determines whether or not a plurality of pieces of coordinate information indicating the touch position are input (step S2). Step S2 corresponds to a detection step. When a plurality of pieces of coordinate information indicating a touch position are not input (step S2 / NO), the information processing device 50 determines the operator displayed at the position of the coordinates indicated by the input coordinate information (step S3). When the information processing device 50 determines the operator, it determines whether or not an operation corresponding to the determined operator has been detected (step S4). For example, when the operator is the slider bar 103 and the slider 105, the information processing device 50 determines whether or not an operation of sliding the slider 105 has been detected as the operation corresponding to the operator. Furthermore, when the operator is a switch or a button, the information processing device 50 determines whether or not the coordinates included in the display position of the switch or button have been continuously detected for a certain period of time or more.

[0049] When an operation corresponding to the operator is detected (step S4), the information processing device 50 instructs the target electrical equipment 80 to execute a process corresponding to the detected operation (step S5). For example, when the electrical equipment 80 to be operated on the first menu screen 100 is an air conditioner, and the operator determined in step S4 is a slider bar and slider for temperature setting, the information processing device 50 changes the temperature setting of the air conditioner with the amount of operation received by operating the slider.

[0050] Furthermore, when a plurality of pieces of coordinate information indicating touch positions are input from the touch panel 40 (step S2 / YES), the information processing device 50 causes the touch panel 40 to display the second menu screen 200 (step S6). Step S6 corresponds to a display step. The information processing device 50 displays contact points 203A and 203B at the coordinates of the touch panel 40 indicated by the input coordinate information, and displays a circular figure 201 having a diameter including the two contact points 203A and 203B. The information processing device 50 also displays a pointer 207 in a direction parallel to a direction perpendicular to a virtual line segment 221 connecting the contact points 203A and 203B and at a position tangent to the circular figure 201, and displays a semicircular range figure 205 outside the circular figure 201.

[0051] Next, the information processing device 50 sets a virtual line segment 221 connecting the two contact points 203A and 203B (step S7), and calculates the length and rotation angle θ of the set virtual line segment 221. The rotation angle θ is, for example, the angle between the virtual line segment 221 and the horizontal right direction of the second menu screen 200.

[0052] Next, the information processing device 50 judges whether or not a change in the coordinates indicated by the coordinate information input from the touch panel 40 has been detected (step S10). If the information processing device 50 cannot detect a change in the coordinates (step S8 / NO), the information processing device 50 returns to the judgment of step S8. Alternatively, the information processing device 50 may proceed to the judgment of step S15 and judge whether or not a certain time has elapsed since the input of the coordinate information in step S1.

[0053] When the information processing device 50 detects a change in the coordinates (step S8 / YES), it calculates the length and rotation angle θ of the virtual line segment 221 connecting the two contact points 203A and 203B corresponding to the changed coordinates (step S9). The information processing device 50 determines whether or not there is a change in the calculated virtual line segment 221.

[0054] When the information processing device 50 determines that there is a change in the length of the virtual line segment 221 (step S10 / length), it calculates the airflow rate of the air conditioner to be changed based on the difference between the first length X1 of the virtual line segment 221 before the change in coordinates is detected and the second length X2 of the virtual line segment 221 after the change in coordinates is detected (step S11). When the second length X2 of the virtual line segment 221 is longer than the first length X1 before the change, the information processing device 50 increases the airflow rate of the air conditioner. Also, when the second length X2 of the virtual line segment 221 is shorter than the first length X1 of the virtual line segment 221 before the change, the information processing device 50 decreases the airflow rate of the air conditioner. Steps S9 to S11 correspond to a detection procedure. The information processing device 50 calculates the air volume to be instructed to change to the air conditioner based on the difference between the length of the virtual line segment 221 before and after the change. When the information processing device 50 calculates the air volume to be instructed to change to the air conditioner, it outputs an instruction to change to the calculated air volume to the air conditioner (step S12). Step S12 corresponds to an instruction step.

[0055] Furthermore, when the information processing device 50 determines that there is a change in the rotation angle θ of the virtual line segment 221 (step S10 / rotation), it calculates the airflow of the air conditioner to be changed based on the difference between the first rotation angle θ1 of the virtual line segment 221 before the change in coordinates is detected and the second rotation angle θ2 of the virtual line segment 221 after the change in coordinates is detected (step S13). Steps S9, S10, and S13 correspond to the detection procedure. The information processing device 50 lowers the set temperature of the air conditioner when the second rotation angle θ2 of the virtual line segment 221 becomes larger than the first rotation angle θ1 before the coordinates are changed. Also, the information processing device 50 raises the set temperature of the air conditioner when the second rotation angle θ2 of the virtual line segment 221 becomes smaller than the first rotation angle θ1 before the coordinates are changed. The information processing device 50 calculates the set temperature to be instructed to the air conditioner to change based on the difference between the rotation angle θ before and after the change of the virtual line segment 221. After calculating the set temperature, the information processing device 50 outputs an instruction to the air conditioner to change to the calculated set temperature (step S14). Step S14 corresponds to the instruction step.

[0056] Next, the information processing device 50 determines whether or not a certain time has elapsed since the instruction was output to the air conditioner (step S15). If the certain time has not elapsed (step S15), the information processing device 50 waits until the certain time has elapsed (step S15). If the certain time has elapsed (step S15 / YES), the information processing device 50 causes the audio output unit 30 to output a predetermined sound and executes an announcing operation (step S16). After that, the information processing device 50 determines whether or not the input of coordinate information from the touch panel 40 has stopped (step S17). That is, it determines whether or not the occupant has released his / her finger from the touch panel 40. If the coordinate information continues to be input from the touch panel 40 (step S17 / NO), the information processing device 50 proceeds to the determination of step S8. If the input of the coordinate information from the touch panel 40 is stopped (step S17 / YES), the information processing device 50 ends this processing flow.

[0057] In the flowchart shown in FIG. 12, the information processing device 50 detects either a change in the length of the virtual line segment or a rotation of the virtual line segment in the judgment of step S10, but may detect both a change in the length of the virtual line segment and a rotation of the virtual line segment. When the information processing device 50 detects both a change in the length of the virtual line segment and a rotation of the virtual line segment, it changes the airflow of the air conditioning device based on the detected change in the length of the virtual line segment, and changes the temperature setting of the air conditioning device based on the detected amount of rotation of the virtual line segment.

[0058] As described above, the information processing device of this embodiment includes the operation detection unit 61, the display control unit 63, and the execution instruction unit 65. Based on coordinate information indicating a touch position input from touch panel 40, operation detection unit 61 detects a first touch operation on touch panel 40 and a second touch operation different from the first touch operation. The display control unit 63 causes the first menu screen 100 to be displayed on the touch panel 40, and changes the display on the touch panel 40 from the first menu screen 100 to the second menu screen 200 when the operation detection unit detects the first touch operation. When the operation detection unit 61 detects a second touch operation on the touch panel 40 on which the second menu screen 200 is displayed, the execution instruction unit 65 instructs the execution of a process associated with the detected second touch operation. When a plurality of touch positions are detected based on the coordinate information, the operation detection unit 61 determines that a first touch operation has been detected. Furthermore, the operation detection unit 61 calculates the length of an imaginary line segment 221 connecting the multiple touch positions detected as the first touch operation, or the rotation angle of the imaginary line segment 221 with respect to a reference direction. Furthermore, when coordinate information is input from the touch panel 40 on which the second screen is displayed, the operation detection unit 61 determines whether or not there has been a change in the length or rotation angle of the virtual line segment 221 based on the input coordinate information, and if it determines that there has been a change in the length or rotation angle of the virtual line segment 221, it determines that a second touch operation has been detected. The execution instruction unit 65 instructs the execution of a process associated with a change in the length of the virtual line segment 221 or a rotation direction of the virtual line segment 221 . Therefore, the occupant can intuitively perform a first touch operation by narrowing, widening, or rotating the spacing of multiple fingers touching the touch panel 40, thereby instructing the execution of a process associated with each of these operations. This improves the operability of operations using the touch panel, and improves user convenience.

[0059] In addition, based on the multiple touch positions detected as the first touch operation, the display control unit 63 displays a range graphic 205 indicating the range within which the settings of the electrical equipment 80 to be operated can be changed, and an indicator 207 indicating the position of the range graphic 205. The display control unit 63 detects the amount of change in the rotation angle of the virtual line segment 221 based on the coordinate information input from the touch panel 40, and changes the position of the range figure 205 indicated by the needle 207 in accordance with the detected amount of change. Therefore, it is possible for the occupant to visually recognize the amount of operation when multiple fingers are touching the touch panel 40 and rotating them simultaneously while keeping the distance between the multiple fingers fixed.

[0060] The display control unit 63 displays a circular figure 201 having a diameter equal to the two touch positions detected as the first touch operation. The display control unit 63 detects a change in the length of the virtual line segment 221 based on the coordinate information input from the touch panel 40, and changes the diameter of the circular figure 201 in accordance with the detected change in length. Therefore, the passenger can recognize the amount of operation based on the diameter of the circular figure 201 displayed on the touch panel 40.

[0061] The above-described embodiment is merely an example of one aspect of the present invention, and any modification and application can be made without departing from the spirit and scope of the present invention. For example, in the above-described embodiment, the first touch operation and the second touch operation are performed with two fingers, but the first touch operation and the second touch operation can also be performed with three fingers. For example, when the first touch operation and the second touch operation are performed with three fingers, namely the thumb, the index finger, and the middle finger, the information processing device 50 may select the two fingers that are farthest from each other as the fingers to be operated, and may set the contact points of these two fingers on the touch panel 40 as the contact points 203A and 203B, and may set the line segment connecting these contact points 203A and 203B as the virtual line segment 221.

[0062] Further, the operation of spreading two fingers apart may be associated with an operation of increasing the temperature setting of the air conditioner, and the operation of narrowing the distance between two fingers may be associated with an operation of decreasing the temperature setting of the air conditioner. Furthermore, the operation of rotating two fingers to the left around the center of rotation while keeping the distance between the two fingers constant may be associated with an operation of decreasing the airflow of the air conditioner, and the operation of rotating two fingers to the right around the center of rotation may be associated with an operation of increasing the airflow of the air conditioner.

[0063] In addition, in the above-described embodiment, the case has been described in which the target of operation on the second menu screen 200 is an air conditioner. However, the electrical equipment 80 that is the target of operation on the second menu screen 200 is not limited to an air conditioner, and may be an audio device, an electric seat, or a navigation device. In addition, the driving mode of the vehicle may be changed by operating the second menu screen 200.

[0064] Further, the second menu screen 200 is not limited to the display shown in Fig. 4. For example, the second menu screen 200 may display arrows pointing in each of the up, down, left and right directions of the second menu screen 200, and setting items whose settings can be changed when a finger is moved in the direction indicated by the arrows. It is also possible to display a plurality of range graphics 205 and pointers 207. A circular graphic 201, range graphics 205A and 205B, and pointers 207A and 207B are displayed on the second menu screen 200 shown in FIG. The range graphic 205A is displayed above the circular graphic 201 at a fixed distance from the circular graphic 201. The range of the range graphic 205A on the right side as one faces the second menu screen 200 indicates the range for increasing the temperature, and the range of the range graphic 205A on the left side as one faces the second menu screen 200 indicates the range for decreasing the temperature. The range graphic 205B is displayed below the circular graphic 201 at a position a certain distance away from the circular graphic 201. The range of the range graphic 205B on the right side as one faces the second menu screen 200 indicates the range for decreasing the temperature, and the range of the range graphic 205B on the left side as one faces the second menu screen 200 indicates the range for increasing the temperature. One end of the pointer 207A is in contact with the circular figure 201, and when the circular figure 201 is rotated by an operation of the passenger, the pointer 207A also rotates in conjunction with the rotation of the circular figure 201. In addition, as the pointer 207A rotates, the position of the range figure 205A indicated by the pointer 207 also changes. One end of the needle 207B is in contact with the circular figure 201, and when the circular figure 201 is rotated by an operation of the passenger, the needle 207B also rotates in conjunction with the rotation of the circular figure 201. In addition, as the needle 207B rotates, the position of the range figure 205B indicated by the needle 207 is also changed. Furthermore, since both pointer 207A and pointer 207B rotate in conjunction with the rotation of circular figure 201, the amount of change in the position of range figure 205A indicated by pointer 207A before and after rotation is the same as the amount of change in the position of range figure 205B indicated by pointer 207B before and after rotation.

[0065] 1 is a schematic diagram showing functions of the information processing device 50 classified according to main processing contents, and can be further divided into more blocks according to processing contents. Furthermore, the processing of each block may be executed by one piece of hardware or by multiple pieces of hardware. Furthermore, the processing of each block may be realized by one program or by multiple programs.

[0066] 12 is divided according to the main processing contents in order to facilitate understanding of the processing of the information processing device 50, and the present invention is not limited by the manner in which the processing units are divided or the names of the processing units. Furthermore, the processing of the information processing device 50 can be divided into more processing units according to the processing contents, or one processing unit can be divided so as to include more processes. Furthermore, the processing order of the above flowchart is not limited to the example shown in the figure.

[0067] In addition, when the program of the present invention is realized by a computer, the program executed by the computer can be configured in the form of a recording medium or a transmission medium for transmitting the program. The recording medium can be a magnetic or optical recording medium or a semiconductor memory device. Specifically, the recording medium can be a flexible disk, a hard disk drive (HDD), a compact disk read only memory (CD-ROM), a DVD, a Blu-ray (registered trademark) disc, or a magneto-optical disk. In addition, the recording medium can be a portable or fixed recording medium such as a flash memory or a card-type recording medium. In addition, the recording medium can be a non-volatile storage device such as a RAM, a ROM, or a HDD, which is an internal storage device provided in the display device. [Explanation of symbols]

[0068] 1 System 10. Camera 20 Control section 30 Audio output section 40 Touch Panel 43 Touch Sensor 50 Information processing device 51 Input / Output Interface 53 Memory 55 Control Program 60 processors 61 Operation detection unit 65 Execution instruction section 80, 80A, 80B, 80C Electrical equipment 100 First menu screen 101A set temperature 101B Air volume 103, 103A, 103B Slider Bar 105, 105A, 105B slider 107 Power button 200 Second menu screen 201 Circular Shapes 203A Contact point 203B Contact point 205 Range Shape 207 Guidelines 210 Operation Image 230 Additional Information

Claims

1. an operation detection unit that detects, based on coordinate information indicating a touch position input from a touch panel, a first touch operation on the touch panel, the first touch operation being an operation of a user touching two fingers on the touch panel, and a second touch operation different from the first touch operation, the second touch operation being an operation of moving the two fingers touching the touch panel while keeping them in contact with the touch panel; a display control unit that displays on the touch panel a first screen which is a screen that is displayed first on the touch panel after the power is turned on, the first screen displaying operation items and operators including a button, switch, or slider for operating the operation items with one finger of a user, and when the operation detection unit detects the first touch operation, changes the display on the touch panel from the first screen to a second screen displaying a circular figure whose diameter indicates a first operation amount and whose pointer provided near the circumference indicates a second operation amount; an execution instructing unit that, when the second touch operation is detected by the operation detecting unit on the touch panel on which the second screen is displayed, instructs execution of a process associated with the detected second touch operation, The operation detection unit determining that the first touch operation has been detected when two contact points on the touch panel displaying the first screen are detected based on the coordinate information; Calculating a length of a virtual line segment connecting two contact points on the touch panel and a rotation angle of the virtual line segment with respect to a reference direction based on the coordinate information; displaying, on the second screen, the circular figure having a diameter equal to a virtual line segment connecting the two contact points based on the coordinate information; When it is determined that there is a change in the length or the rotation angle of the virtual line segment, it is determined that the second touch operation has been detected; The circular figure is always displayed as a circular figure having a diameter equal to the virtual line segment even if the two contact points move on the touch panel; the execution instruction unit changes a diameter of the circular figure in response to a length of the virtual line segment, and instructs execution of a process corresponding to the first operation amount indicated by an amount of change in the diameter, and changes a position of the pointer in response to an amount of change in the rotation angle, and instructs execution of a process corresponding to the second operation amount indicated by the position.

23. An information processing apparatus comprising:

2. 2. The information processing device according to claim 1, wherein the display control unit causes a range graphic indicating a changeable range of the pointer indicating the second operation amount to be displayed near a circumference of the circular graphic on the second screen.

3. On the computer, a detection step of detecting, based on coordinate information indicating a touch position input from a touch panel, a first touch operation on the touch panel, the first touch operation being an operation of a user touching two fingers on the touch panel, and a second touch operation different from the first touch operation, the second touch operation being an operation of moving the two fingers touching the touch panel while keeping them in contact with the touch panel; a display step of displaying on the touch panel a first screen which is a screen that is displayed first on the touch panel after the power is turned on, the first screen displaying operation items and operators including a button, a switch, or a slider for operating the operation items with one finger of a user, and when the first touch operation is detected by the detection step, changing the display on the touch panel from the first screen to a second screen displaying a circular figure whose diameter indicates a first operation amount and whose pointer provided near the circumference indicates a second operation amount; when the second touch operation is detected on the touch panel on which the second screen is displayed by the detection step, executing an instruction step of instructing execution of a process corresponding to the detected second touch operation; the detecting step includes determining that the first touch operation has been detected when two contact points on the touch panel displaying the first screen are detected based on the coordinate information; Calculating a length of a virtual line segment connecting two contact points on the touch panel and a rotation angle of the virtual line segment with respect to a reference direction based on the coordinate information; displaying, on the second screen, the circular figure having a diameter equal to a virtual line segment connecting the two contact points based on the coordinate information; When it is determined that there is a change in the length or the rotation angle of the virtual line segment, it is determined that the second touch operation has been detected; The circular figure is always displayed as a circular figure having a diameter equal to the virtual line segment even if the two contact points move on the touch panel; the instruction step includes changing a diameter of the circular figure in response to a length of the virtual line segment, instructing execution of a process corresponding to the first operation amount indicated by an amount of change in the diameter, changing a position of the pointer in response to an amount of change in the rotation angle, and instructing execution of a process corresponding to the second operation amount indicated by the position of the pointer. A program characterized by:

Images