Processing device and processing method

Abstract

To utilize a surround view to create an added value.SOLUTION: A processing device, which is a processing device of a vehicle, includes a control unit that executes display control for displaying a surround view 20, which is a composite image obtained by looking down the vehicle from outside, and the control unit executes superposition control for superposing a virtual object 21 on the surround view 20 to carry out performance according to an operation for the surround view 20 by a user.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present invention relates to a processing apparatus and a processing method. [Background technology]

[0002] 2. Description of the Related Art In recent years, as disclosed in, for example, Patent Document 1, a technique relating to display control for assisting a driver of a vehicle in driving has been proposed. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] US Patent Application Publication No. 2015 / 0097860 Summary of the Invention [Problem to be solved by the invention]

[0004] Display control that displays a surround view may be used to assist a driver in parking or other driving situations. The surround view is a composite image that shows a bird's-eye view of the vehicle from the outside. For example, a driver can smoothly park the vehicle as if he or she were looking down on the vehicle from the outside by driving the vehicle while viewing the surround view. Here, it is desired to create added value by utilizing the surround view.

[0005] In view of the above, an object of the present invention is to provide a processing device and a processing method capable of creating added value by utilizing surround view. [Means for solving the problem]

[0006] In order to solve the above problems, the processing device is a processing device of a vehicle and is equipped with a control unit that performs display control to display a surround view, which is a synthetic image of the vehicle as if it were viewed from the outside, and the control unit performs superimposition control in the display control to superimpose a virtual object on the surround view and perform an effect according to a user's operation on the surround view.

[0007] In order to solve the above problems, the processing method is a vehicle processing method, in which a control unit of the processing device performs display control to display a surround view, which is a synthetic image of the vehicle as if it were viewed from the outside, and the control unit performs superimposition control in the display control to superimpose a virtual object on the surround view and perform a presentation in response to a user's operation on the surround view. Effect of the Invention

[0008] According to the present invention, it is possible to create added value by utilizing the surround view. [Brief description of the drawings]

[0009] [Figure 1] 1 is a schematic diagram showing a schematic configuration of a vehicle according to an embodiment of the present invention; [Diagram 2] 1 is a block diagram showing an example of a functional configuration of a processing device according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a diagram showing an example of a surround view according to an embodiment of the present invention. [Figure 4] 5 is a flowchart showing an example of a process flow relating to superimposition control performed by the processing device according to the embodiment of the present invention. [Diagram 5] 3A to 3C are diagrams illustrating an example of a surround view in a first example of superimposition control according to the embodiment of the present invention. [Figure 6] 3A to 3C are diagrams illustrating an example of a surround view in a first example of superimposition control according to the embodiment of the present invention. [Figure 7] 3A to 3C are diagrams illustrating an example of a surround view in a first example of superimposition control according to the embodiment of the present invention. [Figure 8] 11A to 11C are diagrams illustrating an example of a surround view in a second example of superimposition control according to the embodiment of the present invention. [Figure 9] 11A to 11C are diagrams illustrating an example of a surround view in a second example of superimposition control according to the embodiment of the present invention. [Figure 10] 11A to 11C are diagrams illustrating an example of a surround view in a second example of superimposition control according to the embodiment of the present invention. [Figure 11] 11A to 11C are diagrams illustrating an example of a surround view in a second example of superimposition control according to the embodiment of the present invention. [Figure 12] 11A to 11C are diagrams illustrating an example of a surround view in a second example of superimposition control according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] A preferred embodiment of the present invention will be described in detail below with reference to the accompanying drawings. The dimensions, materials, and other specific values ​​shown in the embodiment are merely examples for facilitating understanding of the invention, and do not limit the present invention unless otherwise specified. In this specification and drawings, elements having substantially the same functions and configurations are given the same reference numerals to avoid repeated explanations, and elements not directly related to the present invention are not shown.

[0011] <Vehicle configuration> The configuration of a vehicle 10 according to an embodiment of the present invention will be described with reference to FIGS.

[0012] 1 is a schematic diagram showing a schematic configuration of a vehicle 10. As shown in FIG. 1, the vehicle 10 includes a display device 11, a plurality of cameras 12, and a processing device 13.

[0013] There is no particular limitation on the drive source of the vehicle 10. For example, the vehicle 10 may be equipped with only a motor as a drive source, may be equipped with only an engine as a drive source, or may be equipped with both a motor and an engine as drive sources.

[0014] The display device 11 is a device that visually displays various information. An example of the display device 11 is a liquid crystal display. The display device 11 is provided with an input device such as a touch panel.

[0015] The cameras 12 capture images of the surroundings of the vehicle 10. In the example of FIG. 1, four cameras 12 are provided at the front, rear, left side, and right side of the vehicle 10, respectively. The front camera 12 captures images in front of the vehicle 10. The rear camera 12 captures images behind the vehicle 10. The left camera 12 captures images on the left side of the vehicle 10. The right camera 12 captures images on the right side of the vehicle 10. Images obtained by the multiple cameras 12 are used to display a surround view, as described later. However, the number and arrangement of the cameras 12 are not limited to the example of FIG. 1. For example, the number of cameras 12 may be greater than that of the example of FIG. 1.

[0016] The processing device 13 includes a CPU (Central Processing Unit) which is an arithmetic processing device, a ROM (Read Only Memory) which is a storage element for storing programs and arithmetic parameters used by the CPU, and a RAM (Random Access Memory) which is a storage element for temporarily storing parameters that change appropriately during execution of the CPU. The processing device 13 may be, for example, one unit, or may be divided into multiple units. When the processing device 13 is divided into multiple units, various functions described below are shared among the multiple units, so that, for example, some functions of the control unit 13b described below and other functions may be shared by different units.

[0017] Fig. 2 is a block diagram showing an example of a functional configuration of the processing device 13. As shown in Fig. 2, the processing device 13 includes, for example, an acquisition unit 13a, a control unit 13b, and a storage unit 13c.

[0018] The acquisition unit 13a acquires information from each device in the vehicle 10. For example, the acquisition unit 13a acquires information from the camera 12. In this specification, the acquisition of information may include extraction or generation (e.g., calculation) of information. The acquisition unit 13a may also acquire information from an external device by wirelessly communicating with the external device of the vehicle 10.

[0019] The control unit 13b controls the operation of each device in the vehicle 10. In particular, the control unit 13b controls the display device 11 to perform control related to various displays.

[0020] The storage unit 13c stores various types of information. The information stored in the storage unit 13c is used in the processing performed by the control unit 13b.

[0021] <Operation of the processing device> The operation of the processing device 13 according to the embodiment of the present invention will be described with reference to FIGS.

[0022] As described above, the control unit 13b of the processing device 13 performs control regarding various displays by controlling the display device 11. Here, the control unit 13b executes display control for displaying a surround view.

[0023] FIG. 3 is a diagram showing an example of the surround view 20. The surround view 20 is a composite image in which the vehicle 10 is viewed from the outside. In other words, the surround view 20 is a composite image showing a model of the vehicle 10 (specifically, a host vehicle object 21a) and its surroundings from a viewpoint in which the vehicle 10 is viewed from the outside in a pseudo-broadcast manner. As shown in FIG. 3, in the surround view 20, a host vehicle object 21a, which is a virtual object 21 representing the vehicle 10, is superimposed on an image showing the surroundings of the vehicle 10. In the example of FIG. 3, in the surround view 20, the host vehicle object 21a is displayed on a road 22 on which the vehicle 10 runs. The road 22 has a left lane L1 and a right lane L2. The left lane L1 is a driving lane on which the vehicle 10 runs, and the right lane L2 is an opposing lane on which an oncoming vehicle runs against the vehicle 10. Therefore, the host vehicle object 21a is displayed on the lane L1.

[0024] Specifically, the control unit 13b performs a synthesis process of joining together images obtained by the cameras 12, and processes the planar image into a bowl-shaped three-dimensional image in the virtual space. Furthermore, the control unit 13b generates an image representing the surroundings including the front, rear, left side, and right side of the vehicle 10 by setting the viewpoint in the virtual space to face the vicinity of the center of the three-dimensional image. Then, the control unit 13b displays the host vehicle object 21a consisting of a model such as a 3D model representing the vehicle 10 in the image thus generated. In the example of FIG. 3, a 3D model is used as the host vehicle object 21a. The host vehicle object 21a is stored in advance in, for example, the storage unit 13c.

[0025] The surround view 20 is mainly used for the purpose of assisting the driver in parking. Therefore, the control unit 13b mainly executes display control during parking of the vehicle 10. For example, when a specific operation is performed by the driver, the control unit 13b executes display control and causes the display device 11 to display the surround view 20. Examples of the specific operation include an operation using a specific operation unit in the vehicle 10 (for example, pressing a button), or an operation of switching the shift position to a position where the vehicle 10 moves backward (for example, R range). The driver can smoothly perform parking by performing driving operations while visually checking the surround view 20. For example, the driver can change the viewpoint of the surround view 20 (i.e., the direction in which the vehicle 10 is viewed from the surround view 20) during the execution of the display control. Therefore, the driver can perform parking while switching the viewpoint of the surround view 20 to a desired viewpoint.

[0026] In the present embodiment, the control unit 13b executes superimposition control in which, in display control for displaying the surround view 20, the virtual object 21 is superimposed on the surround view 20, and a performance is performed in response to a user's operation on the surround view 20. This makes it possible to create added value by utilizing the surround view 20, as described later.

[0027] Here, as described above, the display control for displaying the surround view 20 is mainly used during parking, but can also be used in situations other than parking. In particular, in the display control executed in situations other than parking, the above-mentioned superimposition control is executed, which makes it easier to create added value by utilizing the surround view 20, as described later.

[0028] Fig. 4 is a flowchart showing an example of a process flow related to superimposition control performed by the processing device 13. Step S101 in Fig. 4 corresponds to the start of the control flow shown in Fig. 4. Step S105 in Fig. 4 corresponds to the end of the control flow shown in Fig. 4. The control flow shown in Fig. 4 is repeatedly executed, for example, at predetermined time intervals.

[0029] When the control flow shown in FIG. 4 starts, in step S102, the control unit 13b determines whether or not the vehicle 10 is being parked.

[0030] For example, when the shift position is a position (eg, R range) where the vehicle 10 moves backward, the control unit 13b can determine that the vehicle 10 is being parked.

[0031] If it is determined that the vehicle 10 is not being parked (step S102 / NO), the process proceeds to step S103, where the control unit 13b permits the superimposition control. On the other hand, if it is determined that the vehicle 10 is being parked (step S102 / YES), the process proceeds to step S104, where the control unit 13b prohibits the superimposition control.

[0032] After step S103 or step S104, the control flow shown in FIG. 4 ends.

[0033] As described above, when the vehicle 10 is parked (specifically, when a parking operation is being performed), the control unit 13b prohibits the superimposition control during the execution of display control to display the surround view 20. On the other hand, when the vehicle 10 is not parked (specifically, when a parking operation is not being performed), the control unit 13b permits the superimposition control during the execution of display control to display the surround view 20.

[0034] As described above, the superimposition control executed in the display control is a control that performs a presentation in response to a user's operation on the surround view 20. The purpose of this superimposition control is to enhance entertainment value by using a game or the like that utilizes the surround view 20 in situations other than when the vehicle 10 is parked (for example, when driving on a highway, etc.). Therefore, the control unit 13b prohibits the superimposition control when the vehicle 10 is parked and driving assistance for parking is required, which is the original purpose of the surround view 20, and on the other hand, permits the superimposition control in situations other than when the vehicle 10 is parked and driving assistance for parking is not required.

[0035] A user of the vehicle 10 (for example, a passenger of the vehicle 10 excluding the driver) can cause the control unit 13b to start the display control by performing an operation to start the display control. After the display control has started, if the superimposition control is permitted, the user of the vehicle 10 can cause the control unit 13b to start the superimposition control by performing an operation to start the superimposition control. Below, a first example and a second example will be described in order as examples of the superimposition control.

[0036] 5 to 7 are diagrams showing an example of a surround view 20 in a first example of superimposition control. In the first example, a user can play a game in which points can be earned by selecting a virtual object 21 displayed in the surround view 20.

[0037] In the example of FIG. 5, another vehicle 23a and an airplane 23b are displayed as moving objects 23 in the surround view 20. The moving object 23 means an object that can move relatively to the driving path 22. The other vehicle 23a and the airplane 23b are captured by the camera 12 of the vehicle 10 and exist as real entities. The other vehicle 23a is traveling ahead of the vehicle 10 in the lane L1 in which the vehicle 10 is traveling. Therefore, in the surround view 20, the other vehicle 23a is located ahead of the host vehicle object 21a on the lane L1. The airplane 23b is flying in the sky.

[0038] In a first example, the control unit 13b displays the virtual object 21 at a display position corresponding to the display position of the moving object 23 in the surround view 20. For example, as shown in FIG. 6, the control unit 13b replaces each moving object 23 with virtual objects 21b and 21c representing dinosaurs in the surround view 20. Specifically, the control unit 13b displays the virtual object 21b representing a dinosaur running on the ground at a display position that approximately matches the display position of the other vehicle 23a, and hides the other vehicle 23a. The control unit 13b also displays the virtual object 21c representing a dinosaur flying in the sky at a display position that approximately matches the display position of the airplane 23b, and hides the airplane 23b.

[0039] Note that the virtual object 21 displayed at the display position corresponding to the display position of the moving object 23 may be a virtual object 21 representing an object other than a dinosaur. Furthermore, the control unit 13b may display the virtual object 21 at the display position corresponding to the display position of the moving object 23, without hiding the moving object 23, in the surround view 20. In that case, for example, the control unit 13b may display the virtual object 21 in the surround view 20 so that the moving object 23 is covered by the virtual object 21.

[0040] 6, control unit 13b displays virtual object 21b, which represents a dinosaur running on the ground, at a display position corresponding to the display position of other vehicle 23a in surround view 20. The display position of virtual object 21b in surround view 20 moves relative to road 22. That is, the display position of virtual object 21b in surround view 20 moves even when vehicle 10 is stopped.

[0041] 6, the control unit 13b displays a virtual object 21c, which represents a dinosaur flying in the sky, at a display position corresponding to the display position of the airplane 23b in the surround view 20. The display position of the virtual object 21c in the surround view 20 moves relative to the driving path 22. That is, the display position of the virtual object 21c in the surround view 20 moves even when the vehicle 10 is stopped.

[0042] Here, the user can perform a selection operation to select the virtual object 21 as an operation on the surround view 20. For example, the user can perform the selection operation by tapping the display device 11 on which the surround view 20 is displayed. In this case, the acquisition unit 13a can acquire information indicating an operation on the surround view 20 by the user from the display device 11 that accepts the operation.

[0043] In the following, an example will be described in which the operation on the surround view 20 is an operation performed on the display device 11 (tapping in the above example). However, the operation on the surround view 20 may be an operation using a device other than the display device 11 (for example, an operation of moving a pointer displayed on the display device 11 using a controller), or may be an operation using a voice uttered by the user.

[0044] The example of FIG. 7 is an example of the surround view 20 when a selection operation (e.g., a touch operation) is performed by the user to select a virtual object 21b representing a dinosaur running on the ground. In the first example, the control unit 13b performs a first performance in response to the selection operation by the user. The first performance is, for example, a performance that notifies the user that points have been awarded. In the example of FIG. 7, the control unit 13b displays a message M1 in the surround view 20 near the virtual object 21b in the first performance. The message M1 indicates, for example, that points corresponding to the selection operation of selecting the virtual object 21b have been awarded to the user.

[0045] When the user performs a selection operation to select the virtual object 21c, the control unit 13b, for example, in the first rendering, displays a message near the virtual object 21c in the surround view 20. In this case, the message indicates, for example, that the points corresponding to the selection operation to select the virtual object 21c have been granted to the user.

[0046] As described above, in the first example of the superimposition control executed in the display control, the user can play a game in which points can be obtained by selecting virtual objects 21b and 21c representing dinosaurs moving relatively to the running path 22. In this way, according to the first example, it is possible to create added value by utilizing the surround view 20.

[0047] In particular, in the first example, as described above, the control unit 13b displays the virtual object 21 at a display position corresponding to the display position of the moving object 23 in the surround view 20. Since the display position of such a virtual object 21 moves relatively to the running path 22, it becomes necessary to select the virtual object 21 that moves relatively to the running path 22, which tends to increase entertainment value.

[0048] 8 to 12 are diagrams showing an example of the surround view 20 in a second example of the superimposition control. In the second example, the user can play a game in which the display position of the host vehicle object 21a in the surround view 20 is changed to cause the host vehicle object 21a to come into contact with another virtual object 21 or to avoid such contact.

[0049] In the second example, the control unit 13b displays the virtual object 21 at a display position corresponding to the display position of the road 22 in the surround view 20. Then, the user can perform an operation to change the display position of the host vehicle object 21a in the surround view 20. In the second example, the control unit 13b performs a second performance to change the display position of the host vehicle object 21a in the surround view 20 in response to the above operation. This allows the user to play a game in which the host vehicle object 21a comes into contact with another virtual object 21, or the user avoids such contact. Various examples related to the second example will be described in order below.

[0050] 8 and 9, the user can play a game in which the object is to bring the host vehicle object 21a into contact with the virtual objects 21d and 21e. In this example, the control unit 13b displays the virtual objects 21d and 21e on the road 22 in the surround view 20, as shown in FIG.

[0051] Specifically, the control unit 13b displays a plurality of virtual objects 21d representing coins in the lane L1 in which the vehicle 10 travels. The plurality of virtual objects 21d are arranged ahead of the host vehicle object 21a at intervals in the extension direction of the lane L1. The control unit 13b also displays a virtual object 21e representing an item box in the opposing lane L2, which is the lane L2. The virtual object 21e is arranged ahead of the host vehicle object 21a. Note that the objects represented by the virtual objects 21d and 21e are not limited to the example in FIG. 8.

[0052] The display positions of the multiple virtual objects 21d representing coins do not move relative to the running path 22. That is, the display positions of the virtual objects 21d in the surround view 20 move in a direction approaching the host vehicle object 21a as the vehicle 10 moves forward, but do not change when the vehicle 10 is stopped. Similarly, the display position of the virtual object 21e representing the item box does not move relative to the running path 22. That is, the display position of the virtual object 21e in the surround view 20 moves in a direction approaching the host vehicle object 21a as the vehicle 10 moves forward, but does not change when the vehicle 10 is stopped.

[0053] As described above, in the second example, the user can perform an operation to change the display position of the host vehicle object 21a in the surround view 20. Then, the control unit 13b performs a second performance to change the display position of the host vehicle object 21a in the surround view 20 in response to the above operation. For example, the user can change the display position of the host vehicle object 21a by swiping the host vehicle object 21a in the surround view 20. For example, when an operation to swipe the host vehicle object 21a to the right is performed, the control unit 13b performs a performance to change the display position of the host vehicle object 21a to the right as shown by an arrow in FIG. 8, as the second performance. Thereby, as shown by a two-dot chain line in FIG. 8, the user can move the host vehicle object 21a from lane L1 to lane L2.

[0054] The example of FIG. 9 is an example of the surround view 20 in a case where the host vehicle object 21a comes into contact with a virtual object 21d representing a coin. In the second example, the control unit 13b performs a third performance according to the positional relationship between the host vehicle object 21a and a virtual object 21 other than the host vehicle object 21a in the surround view 20. The third performance is, for example, a performance that notifies the user of the content set according to the virtual object 21 that comes into contact with the host vehicle object 21a. In the example of FIG. 9, the control unit 13b displays a message M2 in the surround view 20 near the host vehicle object 21a in the third performance. The message M2 indicates, for example, that a coin has been acquired.

[0055] In addition, when the host vehicle object 21a moves to lane L2 and comes into contact with the virtual object 21e representing an item box, the control unit 13b, for example, in the third performance, displays a message indicating that an item has been acquired near the host vehicle object 21a in the surround view 20.

[0056] As described above, in the examples of Figures 8 and 9, the user can change the display position of the host vehicle object 21a in the surround view 20, and bring the host vehicle object 21a into contact with the virtual object 21d representing a coin or the virtual object 21e representing an item box, thereby playing a game to acquire coins or items.

[0057] 10 and 11, the user can play a game in which the objective is to avoid contact between the host vehicle object 21a and a virtual object 21f. In this example, the control unit 13b displays the virtual object 21f on the road 22 in the surround view 20, as shown in FIG.

[0058] Specifically, the control unit 13b displays a virtual object 21f indicating another vehicle in the lane L1 in which the vehicle 10 is traveling. The virtual object 21f is disposed ahead of the host vehicle object 21a. Note that the object indicated by the virtual object 21f is not limited to the example in FIG.

[0059] The display position of the virtual object 21f indicating the other vehicle moves relative to the driving path 22, and moves in a direction approaching the vehicle 10. In other words, the display position of the virtual object 21f in the surround view 20 moves in a direction approaching the host vehicle object 21a even when the vehicle 10 is stopped. However, the display position of the virtual object 21f indicating the other vehicle does not have to move relative to the driving path 22. In this case, the display position of the virtual object 21f in the surround view 20 moves in a direction approaching the host vehicle object 21a as the vehicle 10 moves forward, but does not change when the vehicle 10 is stopped.

[0060] As described above, in the second example, the user can perform an operation to change the display position of the host vehicle object 21a in the surround view 20. Then, the control unit 13b performs a second performance to change the display position of the host vehicle object 21a in the surround view 20 in response to the above operation. For example, when an operation to swipe the host vehicle object 21a to the right is performed, the control unit 13b performs a performance to change the display position of the host vehicle object 21a to the right as shown by an arrow in FIG. 10 as the second performance. Thereby, as shown by a two-dot chain line in FIG. 10, the user can move the host vehicle object 21a from lane L1 to lane L2.

[0061] The example of Fig. 10 is an example of the surround view 20 when the host vehicle object 21a comes into contact with a virtual object 21f representing another vehicle. As described above, in the second example, the control unit 13b performs the third performance according to the positional relationship between the host vehicle object 21a and a virtual object 21 other than the host vehicle object 21a in the surround view 20. In the example of Fig. 10, the control unit 13b displays a message M3 in the vicinity of the host vehicle object 21a in the surround view 20 in the third performance. The message M3 indicates, for example, that the host vehicle has collided with another vehicle or that the game is over.

[0062] Here, as described above, the user can move the host vehicle object 21a from lane L1 to lane L2. This can avoid contact between the host vehicle object 21a and the virtual object 21f representing the other vehicle. In this case, for example, game over is avoided, and the user can continue the game avoiding contact between the host vehicle object 21a and the virtual object 21f representing the other vehicle.

[0063] As described above, in the examples of Figures 10 and 11, the user can play a game in which the user changes the display position of the host vehicle object 21a in the surround view 20 to avoid contact between the host vehicle object 21a and the virtual object 21f representing another vehicle.

[0064] Here, from the viewpoint of enhancing entertainment value through the superimposition control, it is preferable that the control unit 13b superimposes the virtual object 21 on the surround view 20 based on environmental information. The environmental information is information on the current environment around the vehicle 10. For example, the acquisition unit 13a can acquire the environmental information by performing image processing on the image acquired by the camera 12.

[0065] The example of Fig. 12 is an example of the surround view 20 when the vehicle 10 is traveling in a forest area. In this case, the acquisition unit 13a acquires, as environmental information, information indicating that the surroundings of the vehicle 10 are a forest area. In this case, as shown in Fig. 12, the control unit 13b displays a virtual object 21g indicating a fallen tree on the travel path 22 in the surround view 20. The virtual object 21g indicating a fallen tree is a target for avoiding contact with the host vehicle object 21a, similar to the virtual object 21f indicating the other vehicle in the examples of Figs. 10 and 11.

[0066] Specifically, the virtual object 21g is placed ahead of the host vehicle object 21a. The virtual object 21g lies across the lane L1 to lane L2 on the travel path 22. The display position of the virtual object 21g indicating a fallen tree does not move relative to the travel path 22. In other words, the display position of the virtual object 21g in the surround view 20 moves in a direction approaching the host vehicle object 21a as the vehicle 10 moves forward, but does not change when the vehicle 10 is stopped.

[0067] As described above, in the second example, the user can perform an operation to change the display position of the host vehicle object 21a in the surround view 20. Then, the control unit 13b performs a second performance to change the display position of the host vehicle object 21a in the surround view 20 in response to the above operation. For example, when an operation to swipe the host vehicle object 21a upward is performed, the control unit 13b performs a performance to change the display position of the host vehicle object 21a upward as shown by an arrow in FIG. 12 as the second performance. This allows the user to make the host vehicle object 21a jump as shown by a two-dot chain line in FIG. 12. Then, a collision between the host vehicle object 21a and the virtual object 21g representing a fallen tree can be avoided.

[0068] In addition, when the host vehicle object 21a comes into contact with a virtual object 21g representing a fallen tree, the control unit 13b, for example, in the third performance, displays a message indicating that the host vehicle has collided with a fallen tree or that the game is over near the host vehicle object 21a in the surround view 20.

[0069] As described above, the first example shown in Fig. 5 to Fig. 7 and the second example shown in Fig. 8 to Fig. 12 have been described as examples of the superimposition control performed by the processing device 13. However, the processing performed by the processing device 13 is not limited to the above processing examples, and may be, for example, a processing in which the above processing examples are appropriately modified.

[0070] For example, in the first example shown in Fig. 5 to Fig. 7, the control unit 13b may display the virtual object 21 at a display position other than the display position corresponding to the display position of the moving object 23 in the surround view 20. Specifically, in the first example, the control unit 13b may display the virtual object 21 at a display position of the running path 22 in the surround view 20 where the moving object 23 is not located. In this case, the user may be able to earn points by, for example, selecting the virtual object 21 that does not move relatively to the running path 22.

[0071] 8 to 12, the control unit 13b may display the virtual object 21 at a display position other than the display position corresponding to the display position of the running path 22 in the surround view 20. Specifically, the control unit 13b may display the virtual object 21 outside the running path 22 in the surround view 20 in the second example. In this case, the user may, for example, move the host vehicle object 21a outside the running path 22 to bring it into contact with the virtual object 21 (for example, a virtual object 21d indicating a coin) located outside the running path 22.

[0072] 5 to 7, the control unit 13b may superimpose the virtual object 21 on the surround view 20 based on the environmental information. In this case, the user may be able to earn points by selecting the virtual object 21 (e.g., virtual object 21g indicating a fallen tree) superimposed based on the environmental information.

[0073] In the above, an example has been described in which information indicating that the surroundings of the vehicle 10 are forest areas is used as the environmental information. However, the environmental information is not limited to the above example. For example, information indicating the weather or outside temperature around the vehicle 10 may be used as the environmental information.

[0074] In addition, for example, in the above description, the effects (specifically, the first effect, the second effect, and the third effect) performed by the control unit 13b are display effects. However, the effects performed by the control unit 13b may be sound effects instead of display effects, or may be both sound effects and display effects.

[0075] <Effects of the treatment device> The effects of the processing device 13 according to the embodiment of the present invention will be described.

[0076] The processing device 13 includes a control unit 13b that executes display control to display a surround view 20, which is a composite image of the vehicle 10 as if viewed from the outside. In the display control, the control unit 13b executes superimposition control to superimpose a virtual object 21 on the surround view 20 and perform a performance according to an operation on the surround view 20 by a user. This makes it possible to provide entertainment (for example, the above-mentioned game experience) to the user through superimposition control using the surround view 20. Therefore, the surround view 20 not only assists the driver in driving when parking, etc., but also entertains the user. In this way, according to the present embodiment, it is possible to create added value by utilizing the surround view 20.

[0077] Preferably, in the processing device 13, the control unit 13b, in the superimposition control, displays the virtual object 21 at a display position corresponding to the display position of the moving object 23 in the surround view 20. This appropriately realizes entertainment for the user by a performance according to an operation on the surround view 20 in which the virtual object 21 is displayed at a display position corresponding to the display position of the moving object 23. For example, as in the first example shown in Figs. 5 to 7 described above, the user can experience a game in which points can be obtained by selecting the virtual object 21 displayed in the surround view 20.

[0078] Preferably, in the processing device 13, the control unit 13b, in the superimposition control, displays the virtual object 21 at a display position corresponding to the display position of the road 22 in the surround view 20. This appropriately realizes entertainment for the user through a performance according to an operation on the surround view 20 in which the virtual object 21 is displayed at a display position corresponding to the display position of the road 22. For example, as in the second example shown in Figs. 8 to 12 described above, the display position of the host vehicle object 21a in the surround view 20 is changed, and the user can experience a game in which the host vehicle object 21a is brought into contact with another virtual object 21 or the object of avoiding such contact is made.

[0079] Preferably, in the processing device 13, the control unit 13b performs the superimposition control to superimpose the virtual object 21 onto the surround view 20 based on the environmental information. This makes it possible to enhance the entertainment value of the superimposition control in accordance with the current surrounding environment of the vehicle 10.

[0080] Preferably, in the processing device 13, the operation includes a selection operation for selecting a virtual object 21, and the rendering includes a first rendering performed in response to the selection operation. This appropriately realizes providing entertainment to the user through superimposition control. For example, as in the first example shown in Figs. 5 to 7 described above, it appropriately realizes allowing the user to experience a game in which points can be obtained by selecting a virtual object 21 displayed in the surround view 20.

[0081] Preferably, in the processing device 13, the virtual object 21 includes a host vehicle object 21a indicating the vehicle 10, and the effect includes a second effect in which the display position of the host vehicle object 21a in the surround view 20 is changed in response to an operation. This appropriately realizes providing entertainment to the user through superimposition control. For example, as in the second example shown in Figs. 8 to 12 described above, it is appropriately realized to allow the user to experience a game in which the display position of the host vehicle object 21a in the surround view 20 is changed to cause the host vehicle object 21a to come into contact with another virtual object 21, or to avoid such contact.

[0082] Preferably, in the processing device 13, the rendering includes a third rendering that is performed according to the positional relationship between the host vehicle object 21a in the surround view 20 and a virtual object 21 other than the host vehicle object 21a. This more appropriately realizes providing entertainment to the user through superimposition control. For example, as in the second example shown in Figs. 8 to 12 described above, it is more appropriately realized to allow the user to experience a game in which the display position of the host vehicle object 21a in the surround view 20 is changed to cause the host vehicle object 21a to come into contact with another virtual object 21 or to avoid such contact.

[0083] Preferably, in the processing device 13, the control unit 13b executes the superimposition control in a situation other than when the vehicle 10 is parked. Thereby, when the vehicle 10 is parked, the surround view 20 is used for driving assistance related to parking, while the superimposition control can provide entertainment to the user in a situation other than when the vehicle 10 is parked where driving assistance related to parking is not required.

[0084] The above describes a preferred embodiment of the present invention with reference to the attached drawings. However, it goes without saying that the present invention is not limited to the above-described embodiment, and various modified or altered examples within the scope of the claims also fall within the technical scope of the present invention.

[0085] For example, the processes described herein using flowcharts do not necessarily have to be performed in the order shown in the flowcharts, some process steps may be performed in parallel, additional process steps may be employed, and some process steps may be omitted.

[0086] Also, for example, the series of processes by the processing device 13 described above may be realized using any of software, hardware, and a combination of software and hardware. The programs constituting the software are stored in advance in, for example, a storage medium provided inside or outside the information processing device. [Explanation of symbols]

[0087] 10 Vehicles 11 Display device 12 Camera 13 Processing equipment 13a Acquisition part 13b Control section 13c storage section 20 Surround View 21 Virtual Objects 21a Vehicle object 21b Virtual Objects 21c Virtual Objects 21d Virtual Objects 21e Virtual Objects 21f Virtual Objects 21g Virtual Objects 22 Running track 23 Mobile 23a Other vehicles 23b airplane L1 Lane L2 Lane M1 Messages M2 Messages M3 Messages

Claims

1. A processing device (13) for a vehicle (10), comprising: a control unit (13b) that executes display control to display a surround view (20) that is a composite image of the vehicle (10) as if viewed from the outside; The control unit (13b) executes superimposition control in the display control to superimpose a virtual object (21) on the surround view (20) and perform a presentation in response to a user's operation on the surround view (20). Processing unit.

2. The control unit (13b) displays the virtual object (21) at a display position corresponding to a display position of a moving object (23) in the surround view (20) in the superimposition control. The processing device of claim 1 .

3. The control unit (13b) displays the virtual object (21) at a display position corresponding to a display position of a road (22) in the surround view (20) in the superimposition control. The processing device of claim 1 .

4. The control unit (13b) superimposes the virtual object (21) on the surround view (20) based on environmental information in the superimposition control. The processing device of claim 1 .

5. The operation includes a selection operation for selecting the virtual object (21), The performance includes a first performance performed in response to the selection operation. The processing device of claim 1 .

6. The virtual object (21) includes a host vehicle object (21 a) representing the vehicle (10); The presentation includes a second presentation in which a display position of the host vehicle object (21 a) in the surround view (20) is changed in response to the operation. The processing device of claim 1 .

7. The performance includes a third performance that is performed according to a positional relationship between the host vehicle object (21 a) and the virtual object (21) other than the host vehicle object (21 a) in the surround view (20). The processing device of claim 6.

8. The control unit (13b) executes the superimposition control in a situation other than when the vehicle (10) is parked. The processing device according to any one of claims 1 to 7.

9. A method of treating a vehicle (10), comprising the steps of: A control unit (13b) of the processing device (13) executes display control to display a surround view (20) which is a composite image as if the vehicle (10) were viewed from the outside, The control unit (13b) executes superimposition control in the display control to superimpose a virtual object (21) on the surround view (20) and perform a presentation in response to a user's operation on the surround view (20). Processing methods.

Images