View display device and view display method

Abstract

To provide a view display device capable of easily calling a desired view.SOLUTION: A screen generation unit generates a screen including display regions corresponding to each of multiple regions associated with a touch screen, in response to detecting a specific input operation. Each of the multiple regions is associated with one of predetermined multiple viewpoints. A view generation unit is configured to generate a view from a viewpoint associated with one region, to be displayed in a display region corresponding to the one region, when the screen is displayed on the touch screen and a touch operation on the one of the multiple regions is detected.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] The present invention generally relates to a technique for calling a view from a predetermined viewpoint generated based on an image signal output from an imaging unit that images the surroundings of a vehicle.

Background Art

[0002] There is known a technique of displaying on a monitor a composite image (view), such as a stereoscopic image or an overhead view of the vehicle surroundings, obtained by synthesizing images captured by a plurality of cameras attached to a vehicle (see Patent Document 1). According to the view, by using three-dimensional coordinates as a projection plane, the vehicle and its surrounding images can be represented three-dimensionally.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional view calling, when the menu has multiple levels, it is necessary to perform operations such as touching and swiping many times to call the view. And, in order to confirm the environment (surrounding environment) around the vehicle that one is concerned about in the view, after calling the view, an operation to adjust the viewpoint of the view is required, and for this adjustment, one has to gaze at the screen. In addition to the complexity of using the view, the need to gaze at the screen makes it cumbersome to use the view, and the opportunity to use the view is impaired at the start of driving, when parking, during driving, etc.

[0005] The present invention has been made in consideration of the above points, and aims to propose a view display device or the like that can easily call a target view.

Means for Solving the Problems

[0006] In order to solve such problems, in the present invention, there is provided a view display device mounted on a vehicle and configured to display a view related to the vehicle on a touch screen, the view display device including: a view generation unit configured to generate a view from a predetermined viewpoint based on an image signal output from an imaging unit that images the surroundings of the vehicle; and a screen generation unit configured to generate a screen including a display area for displaying the view generated by the view generation unit. The screen generation unit generates a screen including a display area corresponding to each of a plurality of regions associated with the touch screen, triggered by detection of a specific input operation. Each of the plurality of regions is associated with any one of a plurality of predetermined viewpoints. The view generation unit generates a view from the viewpoint associated with one of the plurality of regions for display in the display area corresponding to the one region, triggered by detection of a touch operation on one of the plurality of regions when the screen is displayed on the touch screen.

[0007] In the above configuration, when a region associated with any one of a plurality of predetermined viewpoints is touched, a view from the viewpoint associated with the region is displayed. According to the above configuration, for example, when the user wants to check a concerned surrounding environment in a view, the user can call a view close to the purpose with fewer operations from among the specified views.

Advantages of the Invention

[0008] According to the present invention, a view display device with high convenience can be realized.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Mode for Carrying Out the Invention

[0010] (I) Embodiment Hereinafter, an embodiment of the present invention will be described in detail. However, the present invention is not limited to the embodiment. The view display device of this embodiment synthesizes an image around the vehicle captured by a plurality of cameras installed outside the vehicle into a view from a specific point selected by the user and displays it on the touch screen. For example, this view display device executes the following steps A) to E).

[0011] A) A step of displaying a plurality of regions on the touch screen using a specific touch operation as a trigger B) A step of detecting a touch operation in which the user designates one of the plurality of regions on the touch screen C) A step of displaying a view from a specific point pre-corresponding to the designated region within the selected region of the touch screen D) A step of detecting that a predetermined touch operation has been performed by the user on the touch screen E) A step of enlarging and displaying a view from a specific point pre-corresponding to the touched region in response to the detection of the predetermined touch operation

[0012] The above specific touch operation is characterized in that it is an operation of touching any position on the touch screen with two fingers. The plurality of regions displayed on the touch screen are characterized in that they are arranged imitating each aspect around the vehicle as the center. The above predetermined touch operation is characterized in that it is an operation of lifting the finger from the touch screen. The enlarged display of the view from a specific point pre-corresponding to the touched region is characterized in that it is a full-screen display.

[0013] Thus, the view display device is characterized in that it uses a two-finger touch operation as a trigger, sets an operation range starting from an arbitrary touch coordinate, and performs a pre-display according to the touch coordinate during the touch operation.

[0014] According to the above configuration, the user can immediately check the surrounding environment that he / she is concerned about with a simple operation at the start of driving, parking, during driving, etc. Also, with the pre-display, the user can efficiently select a view closer to his / her intention. For example, when the user is driving on a narrow road and an oncoming vehicle is approaching and he / she has to get as close as possible, or when he / she is driving with heightened attention, the user can instantaneously call up the view to check the surrounding environment.

[0015] Next, embodiments of the present invention will be described with reference to the drawings. The following description and drawings are examples for explaining the present invention, and for the sake of clarity of explanation, appropriate omissions and simplifications have been made. The present invention can also be implemented in various other forms. Unless otherwise particularly limited, each component may be singular or plural.

[0016] In the following description, in the drawings, the same elements are denoted by the same numbers, and the description thereof will be omitted as appropriate. Also, when explaining elements of the same kind without distinction, the common part (the part excluding the branch number) of the reference signs including the branch number is used, and when explaining elements of the same kind while distinguishing them, reference signs including the branch number may be used. For example, when explaining the camera without particularly distinguishing it, it may be described as "camera 102", and when explaining each individual camera while distinguishing them, it may be described as "front camera 102-1".

[0017] In this specification and the like, notations such as "first", "second", "third", etc. are attached to identify components, and do not necessarily limit numbers or order. Also, numbers for identifying components are used for each context, and the numbers used in one context do not necessarily indicate the same configuration in other contexts. Further, it does not prevent a component identified by a certain number from also having the functions of a component identified by another number.

[0018] FIG. 1 shows an example of the overall configuration of the driving support system 100. The driving support system 100 is mounted on the vehicle 101 and supports the driving of the vehicle 101.

[0019] Cameras 102 (an example of an imaging unit), such as wide-angle lenses and fish-eye lenses capable of observing a wide range, are provided on the front, rear, left, and right of the vehicle 101. These cameras 102 can observe (photograph) without omission the area including the road surface around the vehicle 101.

[0020] For example, the driving support system 100 includes a front camera 102-1, a rear camera 102-2, a left-side camera 102-3, and a right-side camera 102-4.

[0021] The front camera 102-1 is a camera 102 provided facing the front of the vehicle 101, and is provided on the front bumper, front grille, etc., which are parts in front of the vehicle 101. The rear camera 102-2 is a camera 102 provided facing the rear of the vehicle 101, and is provided on the rear bumper, rear garnish, etc., which are parts behind the vehicle 101. The left-side camera 102-3 is a camera 102 provided facing the left side of the vehicle 101, and is provided on the left door mirror, etc., which are parts on the left side of the vehicle 101. The right-side camera 102-4 is a camera 102 provided facing the right side of the vehicle 101, and is provided on the right door mirror, etc., which are parts on the right side of the vehicle 101.

[0022] The driving support system 100 generates a composite image from a predetermined viewpoint based on an image of the surroundings of the vehicle 101 captured by the camera 102, and presents the generated composite image to a user such as a driver. The composite images include a perspective view image (stereoscopic image) and a top view image (bird's-eye view image). The perspective view image is a perspective display image from a viewpoint other than directly above the vehicle 101, and is an image including a background image of the surroundings of the vehicle 101 viewed obliquely upward or from the side and an icon representing the vehicle 101. The top view image is an image from a viewpoint directly above the vehicle 101, and is an image including a background image of the surroundings of the vehicle 101 viewed from above and an image of the upper surface of the vehicle 101. Hereinafter, the perspective view image or its video will be referred to as "perspective view". The top view image or its video will be referred to as "top view". When the distinction between the perspective view and the top view is not required, it will be referred to as "view".

[0023] FIG. 2 shows an example of the hardware configuration of the driving support system 100. The driving support system 100 includes an image processing device 210, a navigation device 220, a wheel speed sensor 230, and a steering angle sensor 240.

[0024] The image processing device 210 is, for example, a camera ECU (Electronic Control Unit), and is mainly configured by a microcontroller including a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The image processing device 210 (an example of a view generation unit) controls the camera 102 that images the surroundings of the vehicle 101, and generates a view from a predetermined viewpoint (for example, the virtual viewpoint 411 described later) based on the image signal output from the camera 102. In addition, the image processing device 210 (an example of a screen generation unit) generates a screen including a display area for displaying the generated view. The functions of the image processing device 210 will be described later with reference to FIG. 3.

[0025] The navigation device 220 is an example of a touch screen that displays views, screens, etc. generated by the image processing device 210 and accepts input operations from the user. More specifically, the navigation device 220 includes a monitor 221 that displays images and a touch panel 222 that the user operates for input. The touch panel 222 is superimposed on the display surface of the monitor 221 where images and the like are displayed. The touch panel 222 generates virtual camera information in response to the user's input operation. Note that the touch sensor of the touch panel 222 only needs to be able to acquire the coordinates (touch coordinates) touched on the touch panel 222, and it can be any type of sensor such as a pressure sensor, an electrostatic sensor, or an optical sensor. The virtual camera information is information regarding a virtual camera that is virtually installed in a predetermined space model for projecting a view. The space model will be described later with reference to FIG. 4.

[0026] In addition to the touch panel 222, various operation buttons, rotary dials, joysticks, etc. provided around the monitor 221 may be used as the operation unit capable of input operations. Additionally, the navigation device 220 stores map data for route guidance and the like, and based on this map data and the current position data of the vehicle 101 detected by a GPS device (not shown) etc., it may perform route guidance to the target point set by the user.

[0027] The wheel speed sensor 230 is a sensor that detects the wheel speed of the vehicle 101. The detection information (wheel speed) detected by the wheel speed sensor 230 is input to the vehicle control ECU 250. The steering angle sensor 240 detects the steering angle of the steering of the vehicle 101. The steering angle sensor 240 sets the steering angle when the vehicle 101 is traveling in a straight-ahead state as the neutral position (0 degrees), and detects the rotation angle from the neutral position as the steering angle. The detection information (steering angle) detected by the steering angle sensor 240 is input to the vehicle control ECU 250.

[0028] Furthermore, the driving assistance system 100 includes a vehicle control ECU 250, a steering control unit 260, a throttle control unit 270, and a brake control unit 280.

[0029] The vehicle control ECU 250 is mainly composed of a microcontroller including a CPU, a ROM, a RAM, etc. The vehicle control ECU 250 drives and controls a power steering actuator 261, a throttle actuator 271, and a brake actuator 281 based on each piece of information input from an image processing device 210, a navigation device 220, a wheel speed sensor 230, and a steering angle sensor 240, and executes various processes for assisting the driving of the vehicle 101.

[0030] The steering control unit 260 controls the steering angle of the vehicle 101 by driving the power steering actuator 261 based on the vehicle control information determined by the vehicle control ECU 250. The throttle control unit 270 controls the throttle of the vehicle 101 by driving the throttle actuator 271 based on the vehicle control information determined by the vehicle control ECU 250. The brake control unit 280 controls the brake of the vehicle 101 by driving the brake actuator 281 based on the vehicle control information determined by the vehicle control ECU 250.

[0031] Note that between the image processing device 210, the navigation device 220, the wheel speed sensor 230, and the steering angle sensor 240 and the vehicle control ECU 250, they are connected by an in-vehicle LAN (Local Area Network) 290-1 for communicating sensor information. Also, between the steering control unit 260, the throttle control unit 270, and the brake control unit 280 and the vehicle control ECU 250, they are connected by an in-vehicle LAN 290-2 for communicating vehicle information.

[0032] In this embodiment, regarding the view display device 200 related to the display of a view, a case where it is configured to include an image processing device 210 and a navigation device 220 will be described as an example. That is, the view display device 200 includes an image processing device 210, a touch panel 222, and a monitor 221 to which the touch panel 222 is attached, and is mounted on the vehicle 101. Note that the view display device 200 may or may not include a camera 102. Further, the view display device 200 may be configured to include an image processing device 210 and the monitor 221 and the touch panel 222 are communicably connected.

[0033] FIG. 3 shows an example of the functional configuration of the view display device 200.

[0034] The image processing device 210 includes a control unit 310 and a storage unit 320. The control unit 310 controls the entire image processing device 210. The control unit 310 may be an arithmetic element typified by a programmable logic device such as a CPU, an FPGA, or a GPU (Graphics Processing Unit), or an integrated circuit such as an ASIC. In this case, in order to perform 3D graphic image processing at high speed, it is preferable that the control unit 310 includes a display control unit such as a GPU specialized for image processing or an arithmetic element capable of high-speed operation (for example, an FPGA).

[0035] The storage unit 320 stores coordinate conversion information used for coordinate conversion processing, image processing information used for image processing, and setting information 321 such as parameters. Further, the storage unit 320 may store a control program (image processing program). By executing the control program when the image processing device 210 is activated, the functions of the image processing device 210 (acquisition unit 311, drawing control unit 312, processing unit 313, synthesis unit 314, etc.) are realized.

[0036] The acquisition unit 311 acquires virtual camera information from the input unit 331 which is a function of the touch panel 222. The acquisition unit 311 transmits the acquired virtual camera information to the drawing control unit 312. The drawing control unit 312 acquires from the storage unit 320 the coordinate data of all vertices of the projection plane of the spatial model, or the coordinate data of the vertices of the projection plane that constitutes the projection destination (projection area) of the view, and transmits it to the processing unit 313 together with the GPU setting values. The GPU setting values include information on matrices (rotation matrix, scaling matrix, reduction matrix, translation matrix) used for coordinate transformation and drawing conditions (settings of light such as intensity and direction, transmittance setting, texture of the object).

[0037] The processing unit 313 performs coordinate transformation processing on the coordinate data input from the drawing control unit 312. The processing unit 313 combines the received matrices to calculate and use a model transformation matrix, a view transformation matrix, and a projection transformation matrix. The model transformation matrix is a transformation matrix responsible for the movement, rotation, and scaling of the spatial model itself. The view transformation matrix is a matrix that transforms the coordinates so that the position of the virtual camera is at the origin. The projection transformation matrix is a transformation matrix that gives a sense of perspective. The processing unit 313 performs coordinate transformation by multiplying the coordinate data of each vertex by these matrices. Note that the coordinate data and the coordinate transformation information (including the transformation formula) are associated with each other by a table or the like.

[0038] The composition unit 314 acquires the coordinate data coordinate-transformed by the processing unit 313 and the image signal (captured image) from the camera 102, and composes a view for display on the monitor 221 based on these. When performing view composition, the composition unit 314 can use known techniques such as texture mapping, for example. The composition unit 314 transmits a display control signal for causing the navigation device 220 to display a screen set according to an input operation (touch operation, touch coordinates, sliding direction of the slide operation, etc.) on the touch panel 222 for the composed view. The output unit 332 of the navigation device 220 displays a screen (view) on the monitor 221 based on the display control signal from the image processing device 210.

[0039] FIG. 4 shows an example of a spatial model (spatial model 400) for generating a perspective view.

[0040] The spatial model 400 includes a hemispherical (bowl-shaped) projection surface 410 that extends hemispherically centered on a vehicle model 401 showing the vehicle 101. On the projection surface 410, the center of the hemisphere is the zenith, the vertical direction passing through the zenith is the Y-axis, one axis orthogonal to the Y-axis is the X-axis, and the axis orthogonal to the X-axis and Y-axis is the Z-axis.

[0041] In the spatial model 400, a virtual camera that is virtually installed can be set at an arbitrary position in the three-dimensional space of the projection surface 410. The user can set the position of a desired virtual camera and the desired line-of-sight direction (display direction) when displaying a perspective view on the monitor 221 by operating the touch panel 222. For example, by operating the touch panel 222 in the up, down, left, or right direction (touching, sliding, dragging, swiping, flicking, etc.) or operating a rotary dial or the like, the position of the virtual camera can be specified or the desired line-of-sight direction can be selected. The touch panel 222 detects the operation by the user and generates virtual camera information and transmits it to the acquisition unit 311.

[0042] The virtual camera information includes the position (x, y, z) of the virtual camera on the spatial model 400, the line-of-sight direction of the virtual camera, the viewing angle of the virtual camera, etc. The line-of-sight direction is set as a vector of the azimuth calculated from the position and angle (roll angle, pitch angle, yaw angle) of the camera 102, or as a vector obtained from the position of the virtual camera and the fixation point of the virtual camera. Note that the angle and the fixation point are obtained from the operation direction and operation amount of the touch panel 222. The fixation point of the virtual camera is the coordinates indicating the place where the user wants to look.

[0043] Here, at the start of driving, when parking, during driving, etc., it is required that the surrounding environment of concern can be immediately confirmed with a simple operation. In the view display device 200, a preset virtual viewpoint 411 is provided, and by switching to the desired virtual viewpoint 411 in a single touch operation, the surrounding environment of concern can be confirmed.

[0044] The virtual viewpoint 411 is information that can set an image captured by a virtual camera, and includes information on the position (x, y, z) of the virtual camera and the line-of-sight direction of the virtual camera. Note that since the viewing angle of the virtual camera is determined in advance by the camera 102 to be used, it may be defined separately or may be included in the virtual viewpoint 411.

[0045] Note that for the top view, the main difference is that the spatial model is a plate-shaped projection plane, but it is basically the same as the perspective view, so the description thereof is omitted.

[0046] FIG. 5 shows an example of a region (setting region 500) associated with a touch panel 222 for a user to set (select) the virtual viewpoint 411. The setting region 500 is a region capable of detecting a touch operation for setting the virtual viewpoint 411.

[0047] On the touch panel 222 provided to overlap the monitor 221, when a setting screen 510 is displayed on the monitor 221, a plurality of predetermined setting regions 500 are associated with the touch panel 222 so as to overlap the setting screen 510. Each of the plurality of setting regions 500 is associated with one predetermined virtual viewpoint 411. Note that on the setting screen 510, a display region for displaying a view from the virtual viewpoint 411 associated with the setting region 500 is provided for each of the plurality of setting regions 500. When a touch operation is detected in the setting region 500, a view from the virtual viewpoint 411 associated with the setting region 500 is displayed in the display region corresponding to the setting region 500.

[0048] In this embodiment, as the setting area 500, setting areas 500-1 to 500-6 are defined. The setting area 500-1 is an area for setting a virtual viewpoint 411-1 capable of generating a view (front view) with a part in front of the vehicle 101 as the target of the virtual camera. For example, when a touch operation is detected in the setting area 500-1, the front view is displayed in the display area corresponding to the setting area 500-1. The setting area 500-2 is an area for setting a virtual viewpoint 411-2 capable of generating a view (right front view) with a part in the right front of the vehicle 101 as the target of the virtual camera. The setting area 500-3 is an area for setting a virtual viewpoint 411-3 capable of generating a view (right rear view) with a part in the right rear of the vehicle 101 as the target of the virtual camera. The setting area 500-4 is an area for setting a virtual viewpoint 411-4 capable of generating a view (rear view) with a part in the rear of the vehicle 101 as the target of the virtual camera. The setting area 500-5 is an area for setting a virtual viewpoint 411-5 capable of generating a view (left rear view) with a part in the left rear of the vehicle 101 as the target of the virtual camera. The setting area 500-6 is an area for setting a virtual viewpoint 411-6 capable of generating a view (left front view) with a part in the left front of the vehicle 101 as the target of the virtual camera.

[0049] In FIG. 5, the case where six areas are associated with the setting screen 510 is described as an example, but the number of divisions (two, three, four, five, seven, eight, ···, etc.) at which the view can be seen can be appropriately adopted.

[0050] In this way, the image processing device 210 associates each of the plurality of setting areas 500 with the touch panel 222 corresponding to each aspect around the vehicle 101 defined in advance with respect to the vehicle 101. A virtual viewpoint 411 for generating a view of each aspect is associated with each of the plurality of setting areas 500. According to the above configuration, for example, the user can call up the view of the desired aspect by performing a slide operation on the desired aspect, so that the view close to the purpose can be easily called up.

[0051] FIG. 6 shows an example of view call processing. The view call processing starts when an operation to display a predetermined screen is received in order to utilize a predetermined function in the driving support system 100. The predetermined screen may be any screen such as a map screen, a road guidance screen, an audio screen, a menu screen, etc.

[0052] In S601, the navigation device 220 displays a predetermined screen on the monitor 221.

[0053] In S602, the image processing device 210 determines whether or not a first operation has been detected on the touch panel 222. If the image processing device 210 determines that the first operation has been detected, the process proceeds to S603. If the image processing device 210 determines that the first operation has not been detected, the process returns to S602.

[0054] The first operation is a specific input operation on the touch panel 222. The first operation is an operation of touching the touch panel 222 with multiple fingers, an operation of pressing the touch panel 222, an operation of long-pressing the touch panel 222, etc.

[0055] In S603, the image processing device 210 generates a display control signal for displaying a setting screen 510 for setting a view, the setting screen 510 including display areas corresponding to each of a plurality of setting areas 500, and transmits the signal to the navigation device 220. The navigation device 220 displays the setting screen 510 based on the received display control signal. The setting screen 510 may be provided with a vehicle icon indicating the vehicle 101 and a boundary line indicating a display area around the vehicle icon.

[0056] Further, the image processing apparatus 210 associates a plurality of setting areas 500 associated with any one of the predetermined virtual viewpoints 411 with the touch panel 222, triggered by the detection of a first operation on the touch panel 222. Note that the association between the virtual viewpoint 411 and the setting area 500 may be realized by information such as a table, or may be described (coded) in a program source as a branch process or the like and realized by the branch process or the like.

[0057] In S604, the image processing apparatus 210 determines whether the touch coordinates are within the setting area 500. When the image processing apparatus 210 determines that the touch coordinates are within the setting area 500, the process proceeds to S605. When the image processing apparatus 210 determines that the touch coordinates are not within the setting area 500, the process proceeds to S606.

[0058] In S605, the image processing apparatus 210 generates a display control signal for displaying a view from the virtual viewpoint 411 associated with the setting area 500 corresponding to the touch coordinates in the display area corresponding to the setting area 500 and transmits it to the navigation apparatus 220. The navigation apparatus 220 displays a view from the virtual viewpoint 411 associated with the setting area 500 in the display area corresponding to the setting area 500 based on the received display control signal.

[0059] In this way, the image processing apparatus 210 generates a view from the virtual viewpoint 411 associated with the one setting area 500 for display in the display area corresponding to the one setting area 500, triggered by the fact that the generated setting screen 510 is displayed on the monitor 221 and a touch operation on one of the plurality of setting areas 500 is detected. According to the above configuration, for example, when the user wants to check the surrounding environment of interest in a view, the user can call a view close to the purpose with fewer steps from among the specified views.

[0060] In S606, the image processing apparatus 210 instructs the navigation apparatus 220 to clear the views of all display areas.

[0061] In S607, the image processing apparatus 210 determines whether or not the second operation has been detected on the touch panel 222 or the like. If the image processing apparatus 210 determines that the second operation has been detected, the process proceeds to S608. If the image processing apparatus 210 determines that the second operation has not been detected, the process proceeds to S604.

[0062] The second operation is a predetermined input operation on the setting screen 510. The second operation is, for example, an operation to confirm the first operation, an operation to lift a finger from the touch panel 222, an operation to turn a rotary dial to switch ON / OFF, or the like.

[0063] In S608, the image processing apparatus 210 determines whether or not the touch coordinates immediately before the second operation was detected are within the setting area 500. If the image processing apparatus 210 determines that the touch coordinates are within the setting area 500, the process proceeds to S609. If the image processing apparatus 210 determines that the touch coordinates are not within the setting area 500, the view call process ends.

[0064] In S609, the image processing apparatus 210 generates a display control signal for enlarging and displaying the view from the virtual viewpoint 411 associated with the setting area 500 at the touch coordinates immediately before the second operation was detected, and transmits the signal to the navigation apparatus 220. The navigation apparatus 220 enlarges and displays the view on the monitor 221. After the view is displayed, the user can specify the position of the virtual camera or select a desired line-of-sight direction.

[0065] In this way, when a second operation on the setting screen 510 is detected, the image processing apparatus 210 generates a screen (such as screen 760, screen 770, etc., to be described later) that includes a second display area larger than the first display area corresponding to the setting area 500 selected by the user. The image processing apparatus 210 changes the viewpoint of the view in order to expand and display the view displayed in the first display area in the second display area. According to the above configuration, when the second operation is performed, the view displayed in the first display area is expanded and displayed in the second display area. Thus, for example, the user can check the surrounding environment of concern in detail.

[0066] Further, the second display area may be provided across the entire screen. According to the above configuration, when the second operation is performed, the view displayed in the first display area is expanded and displayed across the entire screen. Thus, for example, the user can check the surrounding environment of concern in more detail.

[0067] FIG. 7 shows an example of screen transition in the call of a view.

[0068] Screen 710 is an example of a predetermined screen and shows a map screen in this example. In screen 710, when a touch operation (first operation) is performed with two fingers 701, the screen transitions from screen 710 to screen 720.

[0069] In this way, the first operation may be an operation of touching the touch panel 222 with a plurality of fingers. In the above configuration, since the first operation is performed by touching the touch panel 222 with a plurality of fingers, for example, the user does not need to shift their line of sight from the monitor 221 to search for an input device such as a rotary dial, and safer driving can be achieved.

[0070] Screen 720 is an example of the setting screen 510. On screen 720, a vehicle icon 721 is displayed at the center of the screen 720, and a boundary line 723 indicating a display area 722 is displayed around the vehicle icon 721. For example, when a slide operation reaches the display area 722-1 (setting area 500-1) of the front view, the screen transitions from screen 720 to screen 730.

[0071] Screen 730 is an example of the setting screen 510. Screen 730 is the screen when the touch coordinates are located in the display area 722-1 of the front view by a slide operation. When the touch coordinates are within the display area 722-1, a front view from the virtual viewpoint 411-1 associated with the display area 722-1 is displayed in the display area 722-1.

[0072] Also, when the slide operation reaches the display area 722-5 (setting area 500-5) of the left rear view via the display area 722-6 (setting area 500-6) of the left front view, the screen transitions from screen 730 to screen 740. Screen 740 is an example of the setting screen 510. On screen 740, when the touch coordinates are within the display area 722-5, a left rear view from the virtual viewpoint 411-5 associated with the display area 722-5 is displayed in the display area 722-5.

[0073] In this way, by performing a pre-display in the display area 722 corresponding to each aspect of the vehicle icon 721, the user can quickly confirm whether the final display is the intended display. Also, since the user can switch the pre-display by a slide operation, the presence or absence of surrounding obstacles can be quickly confirmed.

[0074] On the other hand, on the screen 730, when two fingers leave the touch panel 222 (when a second operation is detected), after the screen transitions from the screen 730 to the screen 750, the screen further transitions from the screen 750 to the screen 760. On the screen 750, a front view with an enlarged front portion is displayed. For example, on the screen 750, a part (front portion) of the front view from virtual viewpoints 411-9, 411-8, 411-7, etc., whose coordinates are closer to the vehicle model 401 than the coordinates of the virtual viewpoint 411-1 used in the front view displayed in the display area 722-1, is displayed. On the screen 760, the entire front view is displayed across the entire screen.

[0075] As described above, the second operation may be an operation of lifting a finger from the touch panel 222. According to the above configuration, for example, the user can call up a view closer to the target by a series of touch operations without lifting a finger from the touch panel 222 from the first operation to the second operation.

[0076] As described above, the image processing apparatus 210 generates a setting screen 510 with a vehicle icon 721 indicating the vehicle 101 at the center, including a display area 722 in the shape of each side around the vehicle 101 arranged to surround the vehicle icon 721, and provided with a boundary line 723 indicating each display area 722. Also, in each of the plurality of setting areas 500, a virtual viewpoint 411 targeting the end of the vehicle 101 on the side facing each side from the vehicle 101 is associated. Then, the image processing apparatus 210 generates a view triggered by detecting a touch operation on one setting area 500 (selection area) among the plurality of setting areas 500. This view is a view from the virtual viewpoint 411 targeting the end of the vehicle 101 on the side facing the aspect of the selection area, which is associated with the selection area and is for display in the display area 722 corresponding to the selection area. In the above configuration, since the vehicle icon 721 and the boundary line 723 indicating the display area 722 are displayed, for example, the user can easily call up a view targeting the end of the vehicle 101 on the relevant side by performing a touch operation on the display area 722 of the aspect to be confirmed.

[0077] According to the present embodiment, unlike the conventional case, multiple operations such as displaying a screen with a hard key (menu button), selecting a menu for calling a view with a soft key, and adjusting the view to a desired viewpoint are not required, so that the target view can be easily called.

[0078] (II) Supplementary Note The above-described embodiment includes, for example, the following contents.

[0079] In the above-described embodiment, the case where the present invention is applied to a view display device has been described. However, the present invention is not limited to this, and can be widely applied to various other systems, devices, methods, and programs.

[0080] Also, in the above-described embodiment, part or all of the program may be installed from a program source into a device such as a computer that realizes the view display device 200. The program source may be, for example, a program distribution server connected by a network or a computer-readable recording medium (for example, a non-temporary recording medium). Further, in the above description, two or more programs may be realized as one program, or one program may be realized as two or more programs.

[0081] Also, in the above-described embodiment, the illustrated and described screens are examples, and any design may be used as long as the received information is the same.

[0082] Also, in the above-described embodiment, the illustrated and described screens are examples, and any design may be used as long as the presented information is the same.

[0083] In the above description, information such as programs, tables, files, etc. that implement each function can be placed in a memory, a storage device such as a hard disk or an SSD (Solid State Drive), or a recording medium such as an IC card, an SD card, or a DVD.

[0084] Regarding the above-described configuration, within a range not exceeding the gist of the present invention, it may be appropriately changed, recombined, combined, or omitted.

Explanation of Reference Numerals

[0085] 200... view display device, 210... image processing device, 221... monitor, 222... touch panel.

Claims

1. A view display device that displays a view related to a vehicle on a touch screen, comprising: a view generation unit that generates a view from a predetermined viewpoint based on an image signal output from an imaging unit that images the periphery of the vehicle; a screen generation unit that generates a screen including a display area for displaying the view generated by the view generation unit; wherein the screen generation unit: when an operation of touching the touch screen with a finger is detected, displays a screen including a plurality of first display areas associated with a plurality of aspects around the vehicle; when it is detected that the finger has been slid to one of the plurality of first display areas while the finger remains touching the touch screen, displays, in the one first display area, a view from a viewpoint in the aspect around the vehicle that is associated with the one first display area; when it is detected that an operation of separating the finger from the one first display area has been performed, enlarges and displays, in a second display area larger than the one first display area, a view from a viewpoint in the aspect associated with the one first display area; a view display device.

2. The screen generation unit provides a screen having a vehicle icon indicating the vehicle at the center, and includes a first display area in the shape of each aspect around the vehicle arranged so as to surround the vehicle icon; the screen generation unit generates a screen provided with a boundary line indicating each first display area; in each of the plurality of first display areas, a viewpoint targeting an end portion of the vehicle on the side facing each aspect from the vehicle is associated; The view display device according to claim 1.

3. The finger is a plurality of fingers. The view display device according to claim 1.

4. The second display area is provided over the entire screen. The view display device according to claim 1.

5. The operation of touching with the finger is an operation on the touch screen on which an arbitrary screen is displayed. The view display device according to any one of claims 1 to 4.

6. A view display method executed by a view display device that generates a view from a predetermined viewpoint based on an image signal output from an imaging unit that images the periphery of a vehicle and displays a screen including a display area for displaying the generated view on a touch screen, When an operation of touching the touch screen with a finger is detected, a screen including a plurality of first display areas associated with a plurality of aspects around the vehicle is displayed. When it is detected that the finger has been slid to one of the plurality of first display areas while the finger remains touching the touch screen, a view from the perspective of the aspect associated with the one first display area among the surroundings of the vehicle is displayed in the one first display area. When it is detected that an operation of separating the finger from the one first display area has been performed, the view from the perspective of the aspect associated with the one first display area is enlarged and displayed in a second display area larger than the one first display area. View display method.

Images