Information processing apparatus and image display method

Abstract

To display an image desired by a user in a simple operation method and through fewer operation procedures.SOLUTION: An information processing apparatus includes a display device, a touch sensor unit, a storage device, and a control unit which causes a screen of the display device to display one or more objects (21a, 21b, 21c, 22, 23, 24) corresponding to one or more functions, and modifies the content of the displayed objects on the basis of an output signal from a touch sensor unit. When a first pinch-out operation for one object is detected, the control unit enlarges a display region of the object and outputs, to the display device, a control signal for displaying an image through a corresponding function on the enlarged display region. When detecting a second pinch-out operation for the object of which the display region has been enlarged, the control unit outputs, to the display unit, a control signal for enlarging an image displayed in the object around the region to which the second pinch-out operation has been performed.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to an information processing apparatus and an image display method, and particularly to an information processing apparatus suitable for displaying an image for assisting driving mounted on a vehicle and an image display method thereof.

Background Art

[0002] Conventionally, an information processing apparatus that displays a camera image or the like mounted on a vehicle on a display screen is known. Generally, a screen called a home screen is first displayed on the display screen, and a plurality of functions available to the driver are presented. By selecting and activating a desired function from these functions, for example, a camera image can be displayed on the display screen and used for safety confirmation when starting or parking the vehicle.

[0003] For example, Patent Document 1 discloses a vehicle start assistance apparatus aimed at accurately providing the driver with the surrounding situation when starting the vehicle.

Prior Art Documents

Patent Documents

[0004]

Patent Document

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in conventional information processing devices with display functions installed in vehicles, including the vehicle starting support device disclosed in Patent Document 1, the home screen is composed of a uniform layout according to the manufacturer's convenience, and generally, multiple functions available to the driver are displayed in the form of a function menu. In such conventional information processing devices, in order for the driver to use the camera image display function, etc., it was necessary to select the desired function on the home screen and activate that function. Furthermore, the camera image displayed after activation is an image according to predetermined settings, and if the driver wishes to adjust the displayed image (for example, enlarge or reduce the image, or change the specified range of the image), more complicated operations are required. In other words, with the conventional technology, when the driver wants to display camera images etc. from the home screen when starting or parking the vehicle, complicated operations are required, which may cause delays and inconvenience before driving can be performed. In addition, because there are many necessary operating procedures, there is a possibility that the driver may make careless operations such as errors.

[0006] This invention was made in consideration of the above points, and proposes an information processing device and an image display method that can display images desired by the user with fewer operating steps and a simpler operating method than conventional methods. [Means for solving the problem]

[0007] To solve the above problems, the present invention provides, for example, an information processing device capable of displaying images according to a function, comprising: a display device having a screen that displays an image and accepts touch operations; a touch sensor unit that detects contact of an object on the screen of the display device and outputs position information of the contact point; a storage device that stores a program and data; and a control unit that executes a program to control the display of one or more objects corresponding to one or more of the above functions on the screen of the display device, and controls the display content of the objects based on an output signal from the touch sensor unit, wherein the control unit, based on an output signal from the touch sensor unit, detects a first pinch-out operation on one of the objects, outputs a control signal to the display device that enlarges the display area of ​​the object and displays an image corresponding to the function of the object in the enlarged display area; and the control unit, based on an output signal from the touch sensor unit, detects a second pinch-out operation on the enlarged display area of ​​the object, outputs a control signal to the display device that enlarges the display image within the object, centering on the area where the second pinch-out operation was performed. [Effects of the Invention]

[0008] According to the present invention, the user can display the desired image with fewer operating steps and a simpler operating method than in the conventional method. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram showing an example configuration of an information processing device 1 according to one embodiment of the present invention. [Figure 2] This flowchart shows an example of the processing procedure performed by the information processing device 1. [Figure 3] This figure shows an example of the configuration of the home screen 20. [Figure 4] This diagram illustrates an example of the display transitions on the home screen 20. [Figure 5]This diagram illustrates the cropping and angle adjustment of the displayed image when using three-finger gestures. [Modes for carrying out the invention]

[0010] One embodiment of the present invention will be described in detail below with reference to the drawings.

[0011] The following descriptions and drawings are illustrative examples for explaining the present invention, and have been omitted and simplified as appropriate for clarity of explanation. Furthermore, not all combinations of features described in the embodiments are necessarily essential to the solution of the invention. The present invention is not limited to the embodiments, and any application that aligns with the spirit of the invention falls within its technical scope. Those skilled in the art can make various additions and modifications within the scope of the present invention. The present invention can also be implemented in various other forms. Unless otherwise specified, each component may be singular or plural.

[0012] Furthermore, in the following explanation, when describing similar elements without distinction, the common part of the reference code (excluding the subscript and subscript) including subscripts and subscripts will be used. When describing similar elements with distinction, the reference code including subscripts and subscripts may be used. For example, when describing home screens without distinction, it will be written as "Home Screen 20," whereas when describing home screen 20 with distinction based on the transition state, it will be written as "Home Screen 20A," "Home Screen 20B," "Home Screen 20C," and so on.

[0013] Figure 1 is a block diagram showing an example configuration of an information processing device 1 according to one embodiment of the present invention. The information processing device 1 is an in-vehicle device mounted on a vehicle such as an automobile, and is typically a computer configured with a processor, memory, storage device, and various interfaces. As shown in Figure 1, the information processing device 1 includes a control unit 11, a storage device 12, a display device 13, and a touch sensor unit 14. The information processing device 1 is also connected to an external camera 15, but this camera 15 may be part of the configuration of the information processing device 1. The following describes the case in which the information processing device 1 is mounted on a vehicle.

[0014] The control unit 11 controls the entire information processing device 1. The control unit 11 is, for example, a processor, and by reading and executing programs held in the storage device 12, it realizes various controls on each component of the information processing device 1 (display device 13 and touch sensor unit 14). In the following description, for the sake of simplicity, the description of control by the control unit 11 may be omitted. Specifically, for example, the display of an image on the display screen of the display device 13 is realized based on the display signal that controls the image display being sent from the control unit 11 to the display device 13, but this may be described in a simplified way such as "the display device 13 displays".

[0015] The storage device 12 consists of a main memory and an auxiliary storage device, which stores programs and various data.

[0016] Furthermore, in the information processing device 1, the main entity performing the processing by executing a program is not limited to the processor, but may also include hardware circuits that perform part or all of the processing. In addition, the program executed in the information processing device 1 is not limited to those stored in the storage device 12, but may be installed and executed in the storage device 12 of the information processing device 1 from a program source (not shown) such as a program distribution server or a storage medium readable by the information processing device 1.

[0017] The display device 13 is a display device having a screen capable of receiving touch operations. The display device 13 displays various images on the screen for the user (for example, the driver or passenger of a vehicle) according to the display signal from the control unit 11. When a user makes a touch operation (touch operation) on the screen of the display device 13 with their finger or the like, the touch sensor unit 14 detects the contact point and outputs a signal indicating the position information of the contact point to the control unit 11 in real time. The images displayed on the display device 13 may be still images or moving images. In addition, the information processing device 1 may be equipped with output devices other than the display device 13, such as a speaker for audio output.

[0018] Camera 15 is a camera having an image sensor such as a CMOS (Complementary Metal Oxide Semiconductor) sensor or a CCD (Charge Coupled Device) sensor. When the information processing device 1 is an in-vehicle device, multiple cameras 15 are mounted on the vehicle to capture images of the area outside the vehicle, and each camera captures a designated area. As will be described in detail later, the information processing device 1 generates an ambient image of the surroundings of the vehicle based on the images captured by one or more cameras 15, and displays the ambient image on the display device 13. The information used to generate the ambient image is not limited to the image data captured by the camera 15, but may also be combined with the above image data to generate the ambient image. In this case, the sensors such as LiDAR (Light Detection And Ranging) and sonar mounted on the vehicle.

[0019] Figure 2 is a flowchart showing an example of the processing procedure performed by the information processing device 1. The processing shown in Figure 2 is executed after the vehicle's accessory (ACC) power is turned ON and the information processing device 1 starts up. Figure 2 shows an example of the processing for typical display transitions among the control related to the display on the display device 13.

[0020] According to FIG. 2, first, when the information processing apparatus 1 is activated, the control unit 11 acquires the situation around the vehicle from the camera 15 or other sensors, and based on the acquired information, generates a peripheral environment video representing the surrounding situation in various directions of the vehicle (step S101). In this example, a plurality of peripheral environment videos are generated for each predetermined direction (left rear of the vehicle, right rear of the vehicle, front of the vehicle). Note that after step S101, the control unit 11 continuously generates the latest peripheral environment video.

[0021] Next, the control unit 11 starts displaying the home screen 20 on the display screen (screen) of the display device 13 (step S102). On the home screen 20, a plurality of objects corresponding to functions available to the user are arranged in a form that can be touched by the user's finger. The details of the home screen 20 will be described later while referring to FIG. 3 and the like.

[0022] On the home screen 20 displayed immediately after the start in step S102, a plurality of objects are displayed in button form, and for example, within the area of each button, a character string (which may also be an icon or the like) representing the corresponding function is shown. For the buttons displayed on the home screen 20, the user can perform a "two-finger operation".

[0023] The "two-finger operation" is an operation performed by touching the display screen (screen) with two fingertips. The two fingertips mainly assume the thumb and the index finger, but other fingers may also be used. In this embodiment, the two-finger operation can be further divided into a plurality of operations according to the movement of the fingertips after touching. Specifically, performing a pinch-out operation of widening the width of the fingertips after touching with two fingertips is called a "pinch-out operation with two fingers", and performing a pinch-in operation of narrowing the width of the fingertips after touching with two fingertips is called a "pinch-in operation with two fingers". In addition, a "drag operation with two fingers" of moving without changing the width of the fingertips after touching with two fingertips may also be accepted.

[0024] If the user initiates a two-finger operation on any button area of ​​the home screen 20 displayed in step S102, the touch sensor unit 14 transmits a signal to the control unit 11 indicating the position information of the two detected contact points (step S103). The touch sensor unit 14 continues to detect changes in the contact points until the fingertips leave the screen and the detection of contact points ends, and transmits the position information to the control unit 11. Based on the reception of such signals, the control unit 11 detects which function area (button) was touched with two fingers, and further identifies (detects) whether a pinch-out, pinch-in, or drag operation was performed depending on the subsequent fingertip movement.

[0025] If the two-finger operation initiated in step S103 is a pinch-out operation and the amount of movement of the pinch-out operation exceeds a predetermined amount, the control unit 11 changes the display mode of the touched object from a first display attribute (e.g., a button) to a second display attribute (e.g., a widget) (step S104).

[0026] The buttons that are compatible with the two-finger pinch-out operation described above are those among the multiple buttons displayed on the home screen 20 that correspond to the function of displaying an image. In the case of Figure 3, which will be described later, this would include buttons 21a, 21b, and 21c that correspond to the function of displaying ambient video, or button 23 that corresponds to the function of displaying map information. For example, if a two-finger pinch-out operation is detected on button 21c, which corresponds to the function of displaying the right mirror image (details will be described later), the control unit 11 expands the display area of ​​button 21a to the size of the widget. The size of the widget is larger than the size of the button, but its size may be seamlessly changeable, or it may be set to expand to one or more predetermined sizes.

[0027] Then, when the fingertips leave the screen and the detection of the two-finger pinch-out operation is complete, the control unit 11 displays information about the corresponding function in the display area, which has been enlarged to the size of the widget (step S105). For example, Figures 4(A) and 4(B), described later, show that when a two-finger pinch-out operation is performed on button 21c, which is assigned the function of displaying the right mirror image, the display area of ​​button 21c is enlarged and changed to widget 25B, and the right mirror image is displayed within widget 25B. When transitioning from a button to a widget, the layout of other buttons is appropriately changed so as not to overlap with the enlarged display area, specifically by moving their position or reducing their size (see buttons 21a, 21b, and 22 in Figure 4(B)).

[0028] Next, on the home screen 20 where the widget is displayed, the user can perform further finger operations on the widget area. In this embodiment, in addition to the two-finger pinch-in or pinch-out operation described above, several types of three-finger operations are available as effective finger operations for the widget.

[0029] "Three-finger operation" refers to an operation performed by touching the display screen with three fingertips. The three fingertips are mainly assumed to be the thumb, index finger, and middle finger, but other fingers may also be used. In this embodiment, three-finger operation can be divided into several types depending on the movement of the fingertips after touching. Specifically, after touching with three fingertips, a pinch-out operation is performed to increase the distance between the remaining contact point (e.g., the fingertip of the thumb) while maintaining the distance between two adjacent contact points (e.g., the fingertips of the index and middle fingers). This is called a "three-finger pinch-out operation." On the other hand, after touching with three fingertips, a pinch-in operation is performed to decrease the distance between the remaining contact point while maintaining the distance between two adjacent contact points. Furthermore, after touching with three fingertips, a drag operation is performed to move the three contact points in any direction while maintaining the relative distances between them. This is called a "three-finger drag operation." Note that in Figure 2, for simplicity, the display control when a three-finger pinch-out gesture is performed is omitted.

[0030] When the control unit 11 detects that a user has initiated a three-finger operation on the widget area on the home screen 20 where the widget is displayed (step S106), it determines the type of three-finger operation detected (step S107) and adjusts the image (e.g., ambient image) displayed on the widget according to that type.

[0031] When a three-finger pinch-out operation is performed on the widget area, the control unit 11 crops the image displayed on the widget in the positive direction according to the pinch-out magnification ratio, centered on a reference point determined from the position of the three fingertips (step S108). Although not shown in Figure 2, when a three-finger pinch-in operation is performed on the widget area, the control unit 11 crops the image displayed on the widget in the negative direction according to the pinch-in reduction ratio, centered on a reference point determined from the contact points of the three fingers. The determination of the reference point and cropping are explained in detail in Figure 5.

[0032] On the other hand, if a three-finger drag operation is performed on the widget area, the control unit 11 changes the angle of the image displayed on the widget according to the direction and distance of the drag (step S109). The angle change will be explained in detail in Figure 5.

[0033] Since the processing in steps S108 and S109 is performed in real time in response to the three-finger operation, when at least one fingertip of the three fingers is lifted, the control unit 11 detects the end of the three-finger operation and terminates the display control in steps S108 and S109 (step S110).

[0034] After that, a three-finger gesture may be performed on the widget again, but this is omitted in Figure 2. Also omitted in Figure 2, if a two-finger pinch-out gesture is performed on the widget area, the control unit 11 controls the display to enlarge the widget within a predetermined maximum expandable area on the home screen 20. At this time, the image displayed within the widget (for example, a right mirror image) is expanded only in its display range, without changing its magnification or angle.

[0035] Figure 2 shows the processing that occurs when a two-finger pinch-in operation is performed on the widget area after step S108 or step S109. In this case, when the control unit 11 detects that the amount of pinch-in movement exceeds a predetermined amount, it changes the display mode of the area from the current display attribute (widget) to the original display attribute (button) (step S111). As a result, the display on the display device 13 returns to the home screen 20 in step S102.

[0036] As described above, the display control shown in Figure 2 continues until any object displayed on the home screen 20 is selected and the function of that object is displayed in full screen on the screen, or until the accessory power of the vehicle equipped with the information processing device 1 is turned off. In other words, from the user's perspective, with conventional home screens, it was necessary to select the desired display function, expand that function to full screen, and then perform operations such as cropping or changing the angle of the image. However, with the home screen 20 of the information processing device 1 according to this embodiment, the user can display the image of the desired display function within a widget on the home screen 20, which displays multiple objects, without having to expand individual functions to full screen. Furthermore, the user can perform operations such as cropping or changing the angle of the displayed image. In addition, since these operations are performed with two or three fingers, the user can easily operate them with one hand.

[0037] Furthermore, while the above explanation regarding Figure 2 described display control for an object of a single display function, "right mirror image display," the control unit 11 can perform similar display control in parallel for multiple display functions displayed on the home screen 20. Specifically, for example, it is possible to change the right mirror image display button 21c to a widget display, and then change the left mirror image display button 21a to a widget display as well. By performing such display control, users can use multiple functions on the home screen 20 simultaneously.

[0038] Below, we will explain in more detail the various display controls for the home screen 20 mentioned above, referring to Figures 3 to 5.

[0039] Figure 3 shows an example of the configuration of the home screen 20. The home screen 20A shown in Figure 3 is an example of the home screen 20 that is first displayed after the information processing device 1 is started, and multiple objects are displayed as buttons 21a, 21b, 21c, 22, 23, and 24. Button 22 directly displays the date and time, which is its corresponding function, but the other buttons display a string (or icon, etc.) that represents the corresponding function, so that the user can easily determine the function provided when the button is operated. The individual buttons will be described below.

[0040] Buttons 21a to 21c are objects that correspond to the function of displaying the surrounding environment video of the vehicle captured by camera 15. To explain them individually, button 21a corresponds to the function of displaying the left mirror image, button 21c corresponds to the function of displaying the right mirror image, and button 21b corresponds to the function of displaying the front camera image. The left mirror image is the surrounding environment video of the left rear of the vehicle, and is mainly generated from video captured by camera 15 installed around the left side mirror of the vehicle. The right mirror image is the surrounding environment video of the right rear of the vehicle, and is mainly generated from video captured by camera 15 installed around the right side mirror of the vehicle. The front camera image is the surrounding environment video of the front lower part of the vehicle, and is mainly generated from video captured by camera 15 installed on the front of the vehicle. As described above, buttons 21a to 21c are all objects that correspond to the function of displaying the surrounding environment video, and are therefore sometimes collectively referred to as button 21.

[0041] Button 22 corresponds to the function that displays the current date and time. Button 23 corresponds to the function that displays map information. Button 24 corresponds to the telephone function.

[0042] When the user touches and drags any one of the buttons 21-24 above with two or three fingers, the control unit 11 controls the arrangement of the buttons on the home screen 20 (layout change). This control of layout change is a well-known technique, and a detailed explanation is omitted.

[0043] Furthermore, of the above buttons 21 to 24, at least button 21, which corresponds to the function of displaying ambient video, and button 23, which corresponds to the function of displaying map information, will have their display mode switched from a first display attribute (button) to a second display attribute (widget) when the user performs a two-finger pinch-out operation beyond a specified amount on the area of ​​the button, as explained in steps S103 to S105 of Figure 2, and the corresponding function will be provided within the widget. Then, as explained in step S111 of Figure 2, after switching to a widget, when a two-finger pinch-in operation beyond a specified amount is performed on the area of ​​the widget, the display mode will be switched from a second display attribute (widget) to a first display attribute (button).

[0044] Figure 4 is a diagram illustrating examples of display transitions for the home screen 20. Figure 4(A) shows home screen 20A as an example of the home screen 20 that is initially displayed after the information processing device 1 is started. Figure 4(B) shows home screen 20B as an example of the home screen 20 when it is switched from button display to widget display. Figure 4(C) shows home screen 20C as an example of the home screen 20 when a three-finger pinch-out operation is performed on the widget display.

[0045] The home screen 20A shown in Figure 4(A) is the same as the home screen 20A shown in Figure 3. In the home screen 20A, the button 21c displays the string "R Mirror," suggesting that it is an object corresponding to the function of displaying the right mirror image.

[0046] When user 30 performs a pinch-out operation with two fingers exceeding a specified amount on the area of ​​button 21c on home screen 20A, the screen transitions to home screen 20B as shown in Figure 4(B).

[0047] As shown in Figure 4(B), on the home screen 20B, the button 21c corresponding to the right mirror image display function is changed to a widget 25B, and the right mirror image is displayed within the widget 25B. Comparing Figure 4(A) and Figure 4(B), it is clear that the widget 25B is displayed in a larger area than the button 21c. Also, on the home screen 20B, in order to secure the display area of ​​the widget 25B, the display area of ​​the button 22 is reduced, and the placement of buttons 21b and 22 is changed.

[0048] When user 30 performs a pinch-in operation with two fingers exceeding a specified amount on the widget 25B area of ​​home screen 20B, the system transitions to home screen 20A as shown in Figure 4(A), and the widget display reverts to a button display.

[0049] Furthermore, when user 30 performs a three-finger pinch-out operation on the widget 25B area of ​​home screen 20B, the screen transitions to home screen 20C as shown in Figure 4(C).

[0050] The right mirror image displayed in widget 25C on the home screen 20C is cropped in the positive direction according to the pinch-out magnification, centered on a reference point determined by the position of the three fingertips used in the pinch-out operation, as explained in step S108 of Figure 2.

[0051] Although not shown in Figure 4, when a user 30 performs a three-finger pinch-in operation on the widget 25B area of ​​the home screen 20B, a right mirror image is displayed, which is cropped in the negative direction according to the pinch-in reduction ratio, centered on a reference point determined by the position of the fingertips of the three fingers used for the pinch-in operation.

[0052] Furthermore, although not shown in Figure 4, if user 30 performs a three-finger drag operation on the widget 25B area of ​​the home screen 20B, a right mirror image with an angle changed according to the direction and distance of the drag will be displayed on the transitioned widget 25C, as explained in step S109 of Figure 2.

[0053] Then, if user 30 performs a pinch-in operation with two fingers exceeding a specified amount on the widget 25C area of ​​the home screen 20C, the system transitions to the home screen 20A shown in Figure 4(A), and the widget display reverts to a button display, similar to when user 30 performs a pinch-in operation with two fingers exceeding a specified amount on the widget 25B area.

[0054] The following section will explain in detail how to crop and change the angle of the displayed image, referring to Figure 5.

[0055] Figure 5 illustrates the cropping and angle adjustment of the displayed image when using a three-finger gesture.

[0056] Figure 5(A) shows a model of user 30's hand when performing a three-finger operation. When user 30 touches the screen with the tips of their thumb, index finger, and middle finger, the point of contact by the index finger is denoted as point A, the point of contact by the middle finger as point B, and the point of contact by the thumb as point C. The distance between the midpoint of line segment AB and point C is denoted as distance D, and the midpoint between the midpoint of line segment AB and point C is denoted as point E. This point E corresponds to the reference point described in the explanation of step S108 in Figure 2.

[0057] When A to E are defined as described above, when a three-finger operation is performed on the widget area, the touch sensor unit 14 detects the coordinate information of points A to C, which are the contact points, and transmits it to the control unit 11. The control unit 11 then uses the received coordinate information of points A to C to calculate the distance D and the coordinate of point E.

[0058] When a three-finger operation is performed on the widget area, if the coordinates of point E move without the distance D changing (a change within a predetermined range may be permitted), the control unit 11 determines that a three-finger drag operation is being performed and changes the angle of the image displayed on the widget according to the direction and amount of movement of point E.

[0059] Figures 5(B) and (C) show examples of how the widget's displayed image changes when point E moves while the distance D remains unchanged. Specifically, the right mirror image displayed in widget 25A in Figure 5(B) shows only a small portion of the vehicle behind, indicating that the camera angle of 15 is not ideal. In contrast, in widget 25B in Figure 5(C), after a three-finger drag operation, the angle has been changed according to the movement of point E, and almost the entire vehicle behind is displayed. This is an improved display that makes it easier for the user 30 to check for safety to the right rear.

[0060] On the other hand, when a three-finger operation is performed on the widget area, if the distance D increases (expands), the control unit 11 determines that a three-finger pinch-out operation is being performed and, using point E as the reference point, crops the image displayed on the widget in the positive direction according to the expansion rate of the distance D. Conversely, if the distance D decreases (shrinks), the control unit 11 determines that a three-finger pinch-in operation is being performed and, using point E as the reference point, crops the image displayed on the widget in the negative direction according to the reduction rate of the distance D.

[0061] Figures 5(D) and 5(E) show examples of how the widget's displayed image changes when the distance D is increased. Specifically, the right mirror image displayed in widget 25B in Figure 5(D) is the same as the right mirror displayed in widget 25B in Figure 5(C), but the size of the vehicle behind is slightly smaller. In contrast, in widget 25C in Figure 5(E), after a three-finger pinch-out operation, the vehicle behind is displayed larger due to positive cropping, making it easier for the user 30 to carefully check the safety of the right rear.

[0062] As described above, according to the information processing device 1 of this embodiment, when the device is started, a home screen 20 showing objects corresponding to multiple functions is automatically displayed on the screen of the display device 13 by program processing by the control unit 11. The user (for example, a vehicle driver) can switch to widget display and display an image by performing a two-finger pinch-out operation with one hand on the object of the desired display function on the home screen 20, without having to select any display function from the home screen 20 and start the camera. In this embodiment, the two-finger pinch-out or pinch-in operation on an object is an operation to change the display size of the object (button, widget).

[0063] Furthermore, according to the information processing device 1 of this embodiment, the image range displayed on an object (widget) can be changed by performing a predetermined three-finger operation on a widget-type object that displays an image. More specifically, when a pinch-out or pinch-in three-finger operation is performed, the displayed image can be cropped, and when a drag three-finger operation is performed, the angle of the displayed image can be changed. By enabling such adjustment of the displayed image with a three-finger operation, users can view the image they want to see with simple one-handed operation.

[0064] Therefore, the information processing device 1 according to this embodiment can display images of the vehicle's surrounding environment with fewer operating steps and simpler operation methods compared to a conventional home screen. Specifically, the user can activate a widget that displays camera images that are expected to require confirmation, such as blind spots that cannot be seen with the naked eye, with a two-finger operation on one hand. Furthermore, the user can then perform operations such as cropping and zooming in / out within the widget with a three-finger operation on one hand, without switching screens or operation modes. As a result, the information processing device 1 according to this embodiment can prompt the driver to check for safety without having to activate the camera function from the home screen 20 each time the vehicle starts or stops.

[0065] The widget display images exemplified in Figures 4 and 5 were two-dimensional images, known as a 2D view. However, the images that can be displayed in the widget in this embodiment are not limited to these, and may also be three-dimensional images, known as a 3D view. Furthermore, a function to switch between 2D and 3D images may be added. When using a 3D image, the control unit 11 uses known image processing techniques to generate a 3D virtual space based on data acquired by the camera 15, sensors, etc., and displays the image obtained when the virtual camera captures this virtual space as a surrounding environment image within the widget. When a 3D image is displayed in the widget, if a pinch-out or pinch-in operation is performed using a three-finger gesture, the image obtained by zooming in or out of the virtual space without changing the position and orientation of the virtual camera is displayed within the widget. Also, when a drag operation is performed using a three-finger gesture while a 3D image is displayed in the widget, the virtual camera is moved around the vehicle according to the amount and distance of movement of point E as explained in Figure 5, and the image obtained by capturing the virtual space from the moved virtual camera is displayed within the widget.

[0066] Furthermore, the home screen 20 allows for the simultaneous display of widgets for multiple objects. As a result, users can arrange multiple image displays on the home screen 20 within the limited display space of the display device 13 without switching to individual display function screens, allowing them to easily and conveniently view multiple ambient images and other information simultaneously, thus enabling safer driving.

[0067] Furthermore, on the home screen 20, the layout of objects can be easily changed with one-handed operation (two-finger drag operation) to suit the user's preferences. As a result, accidental operations such as user errors can be prevented.

[0068] Furthermore, the information processing device 1 may save the layout of the home screen 20 that has been changed by the user and reproduce it the next time the home screen 20 is displayed. Specifically, in this case, before the vehicle's accessory power is turned OFF, the control unit 11 stores the final layout information of the home screen 20 in the storage device 12. Then, when the accessory power is turned ON again, the control unit 11 can display the home screen 20 with a layout according to the user's preference by referring to the layout information stored in the storage device 12.

[0069] Furthermore, regarding the saving of the home screen 20 layout settings described above, it may be possible to save not only the placement of each object, but also the display attributes of each object (i.e., whether it is in button format or widget format). In addition, in the case of widget format, it may be possible to save the display settings for the image displayed by the widget (i.e., the results of cropping and angle adjustments). Moreover, it may be possible to change the layout settings of the home screen 20 on a per-user basis. By adding these functions, it is possible to provide the home screen 20 in a form that better suits the user's preferences, thereby preventing unintentional operations by the user and improving usability and convenience.

[0070] It should be noted that the present invention is not limited to the embodiments described above, and various modifications are included. For example, the two-finger operation and the three-finger operation that are effective on the home screen 20 may be swapped. Furthermore, the embodiments described above are described in detail to make the present invention easier to understand, and are not necessarily limited to those having all the described configurations. It is also possible to replace a part of the configuration of one embodiment with a modified configuration, and it is also possible to add a modified configuration to the configuration of an embodiment. In addition, it is possible to add, delete, or replace a part of the configuration of an embodiment with other configurations. [Explanation of Symbols]

[0071] 1. Information Processing Device 11 Control Unit 12 Storage device 13 Display device 14 Touch sensor section 15 Cameras 20 (20A~20C) Home Screen Buttons 21 (21a~21c), 22, 23, 24 25 (25A~25C) Widgets 30 users

Claims

1. An information processing device capable of displaying an image according to a function, A display device having a screen for displaying an image and accepting a touch operation; a touch sensor unit that detects a contact of an object with a screen of the display device and outputs position information of the contact point; A storage device for storing programs and data; a control unit that controls displaying one or more objects corresponding to one or more of the functions on a screen of the display device by executing a program, and controls changing display content of the object based on an output signal from the touch sensor unit; When the control unit detects a first pinch out operation on one of the objects displayed with a first display attribute based on an output signal from the touch sensor unit, the control unit outputs to the display device a control signal for enlarging a display area of ​​the object and displaying an image in the enlarged display area using a function corresponding to the object with a second display attribute.

23. An information processing apparatus comprising:

2. When the control unit detects a second pinch-out operation on the object that has enlarged the display area based on the output signal of the touch sensor unit, the control unit outputs a control signal to the display device to enlarge and display a display image in the object centered on the area where the second pinch-out operation has been performed.

2. The information processing apparatus according to claim 1,

3. The first pinch out operation is an operation of touching the screen with two points and increasing the distance between the two contact points; The second pinch-out operation is an operation in which three points are touched on the screen, and the distance between two adjacent contact points among the three contact points increases from the remaining contact point.

2. The information processing apparatus according to claim 1,

4. The first pinch-in operation is an operation of touching the screen with two points and narrowing the distance between the two contact points, When the control unit detects the first pinch-in operation on the object displayed with the second display attribute, the control unit outputs to the display device a control signal for reducing a display area of ​​the object and displaying it with the first display attribute.

2. The information processing apparatus according to claim 1,

5. The second pinch-in operation is an operation of touching the screen with three points, and narrowing a distance between two adjacent contact points and the remaining contact point among the three contact points, When the control unit detects the second pinch-in operation on the object with the display area enlarged, the control unit outputs to the display device a control signal for reducing and displaying a display image in the object with the area where the second pinch-out operation is performed as a center.

2. The information processing apparatus according to claim 1,

6. The first drag operation is an operation of touching the screen at two points and moving the two touch points while maintaining a distance between the two touch points, When the control unit detects the first drag operation on one of the objects based on the output signal of the touch sensor unit, the control unit moves a display position of the object on the screen in accordance with a movement amount and a movement direction of the first drag operation, and when the movement of the object causes it to overlap with a display area of ​​another object, outputs a control signal to the display device for further changing the display position or display area of ​​the other object.

2. The information processing apparatus according to claim 1,

7. The second drag operation is an operation of touching the screen at three points and moving the three touch points while maintaining the mutual distance between the three touch points, When the control unit detects the second drag operation on the object that has enlarged the display area based on the output signal of the touch sensor unit, the control unit outputs to the display device a control signal for changing an angle of an image to be displayed on the object in accordance with a movement amount and a movement direction in the second drag operation.

2. The information processing apparatus according to claim 1,

8. The information processing device is an in-vehicle device to which a camera for capturing images of the outside of a vehicle is connected, The control unit generates a surrounding environment image representing a situation around the vehicle based on the photographed data of the camera, When a first pinch out operation on one of the objects is detected based on an output signal from the touch sensor unit, the control unit outputs to the display device a control signal for enlarging a display area of ​​the object and displaying the surrounding environment image according to a function corresponding to the object in the enlarged display area.

2. The information processing apparatus according to claim 1,

9. An image display method by an information processing device capable of displaying an image according to a function, comprising: The information processing device includes: A display device having a screen for displaying an image and accepting a touch operation; a touch sensor unit that detects a contact of an object with a screen of the display device and outputs position information of the contact point; A storage device for storing programs and data; a control unit that controls displaying one or more objects corresponding to one or more of the functions on a screen of the display device by executing a program, and controls changing display content of the object based on an output signal from the touch sensor unit, a display area enlarging step in which, when the control unit detects a first pinch out operation on one of the objects displayed with a first display attribute based on an output signal from the touch sensor unit, the control unit outputs to the display device a control signal for enlarging a display area of ​​the object and displaying an image in the enlarged display area using a function corresponding to the object with a second display attribute.

13. An image display method comprising:

Images