Information processing method, information processing system, and information processing device

Abstract

An information processing system 100 displays a captured image V0 on a display 20 if a target vehicle 40 is traveling in an imaging-enabled range X of each roadside camera 10, displays a virtual image V1 on the display 20 if the target vehicle 40 is traveling in a range other than the imaging-enabled range X, determines whether or not a first imaging direction D01 of a first roadside camera 10a and a second imaging direction D02 of a second roadside camera 10b located on the front side of the first roadside camera 10a are different, and when the first imaging direction D01 and the second imaging direction D02 are different, changes a display direction D1 of the virtual image V1 such that a second display direction D12 when the target vehicle 40 is located at a second vehicle position P02 on the front side of a first vehicle position P01 is closer to the second imaging direction D02 than a first display direction D11 when the target vehicle 40 is located at a first vehicle position P01.

Description

Information processing method, information processing system, and information processing device

[0001] The present invention relates to an information processing method, an information processing system, and an information processing device.

[0002] Patent Document 1 describes a mobile object monitoring system that uses monitoring cameras installed at multiple locations to capture images of a vehicle to be monitored. The mobile object monitoring system described in Patent Document 1 appropriately selects from the multiple monitoring cameras a monitoring camera that can capture an image of the vehicle to be monitored, so that the system can follow and monitor the vehicle.

[0003] Japanese Patent Application Laid-Open No. 2006-163764

[0004] However, in the mobile object monitoring system of Patent Document 1, when the monitoring camera performing the shooting is switched as the target vehicle moves, the user viewing the captured image may not immediately understand from which direction the image was captured, and may lose sight of the target vehicle.

[0005] The problem that the present invention aims to solve is to provide an information processing method, an information processing system, and an information processing device that, when switching between roadside cameras (infrastructure cameras) that capture images of a target vehicle, allows a user viewing the captured image after switching to easily identify the target vehicle.

[0006] The present invention solves the above problem by displaying on a display a virtual image of the target vehicle and the road in a predetermined display direction relative to the road when the target vehicle is traveling outside the imageable range, determining whether the first image capturing direction of the first roadside camera relative to the road when the target vehicle leaves the first image capturing range of the first roadside camera is different from the second image capturing direction of the second roadside camera relative to the road when the target vehicle enters the second image capturing range of the second roadside camera located forward of the direction of travel of the target vehicle of the first roadside camera, and if the first image capturing direction and the second image capturing direction are different, changing the predetermined display direction of the virtual image to match the position of the target vehicle so that the second display direction when the target vehicle is at a second vehicle position near the position of the second roadside camera forward of the first vehicle position is closer to the second image capturing direction than the first display direction when the target vehicle is at the first vehicle position near the position of the first roadside camera.

[0007] According to the present invention, when switching between roadside cameras capturing an image of a target vehicle, a user viewing the captured image after the switching can easily identify the target vehicle.

[0008] FIG. 1 is a block diagram showing the configuration of an information processing system according to a first embodiment. FIG. 2 is a diagram showing an example of the positional relationship between a first roadside camera, a second roadside camera, and a target vehicle on a map. FIG. 3 is an example of a captured image captured by a first roadside camera of the information processing system shown in FIG. 1. FIG. 4 is an example of a virtual image displayed on a display by a virtual image display control unit of the information processing system shown in FIG. 1. FIG. 5 is an example of a virtual image displayed on a display by a virtual image display control unit of the information processing system shown in FIG. 1. FIG. 6 is an example of a captured image captured by a second roadside camera of the information processing system shown in FIG. 1. FIG. 7 is a flowchart showing the procedure of an information processing method executed by the information processing system shown in FIG. 1. FIG. 8 is a block diagram showing the configuration of an information processing device according to a second embodiment.

[0009] Embodiments of the present invention will be described below with reference to the drawings. First Embodiment An information processing system 100 according to a first embodiment will be described with reference to FIGS. 1 to 7. The information processing system 100 shown in FIG. 1 includes multiple roadside cameras 10, a display 20, and a server 30. The information processing system 100 uses a controller 3 of the server 30 to display an image including a target vehicle 40 (see FIG. 2) traveling on a road 5 on a display 20 external to the target vehicle 40. The information processing system 100 can communicate with the multiple roadside cameras 10 and the display 20 via a communication network 4. As indicated by the dashed lines, the server 30 may also be capable of communicating with the target vehicle 40 via the communication network 4. While the number of roadside cameras 10 is not particularly limited as long as it is two or more, this embodiment will be described assuming that the multiple roadside cameras 10 are composed of two cameras (a first roadside camera 10a and a second roadside camera 10b) for ease of explanation. In addition, the controller 3 of the server 30 may transmit data of the captured image V0 and the virtual image V1 to the target vehicle 40, and display the captured image V0 and the virtual image V1 on an in-vehicle display provided in the target vehicle 40.

[0010] The roadside cameras 10 (first roadside camera 10a and second roadside camera 10b) are infrastructure cameras arranged along the road 5 to capture images of the road 5. In this embodiment, the imaging direction D0 (first imaging direction D01 and second imaging direction D02) of the roadside cameras 10 is fixed, but this is not limiting. The roadside cameras 10 may be rotatably mounted, and the imaging direction D0 may be changed as appropriate. In the example shown in FIG. 2 , the second roadside camera 10b is located forward of the first roadside camera 10a. The first imaging direction D01 of the first roadside camera 10a is a direction toward the front of the travel path of the target vehicle 40. On the other hand, the second imaging direction D02 of the second roadside camera 10b is a direction toward the rear of the travel path of the target vehicle 40. In other words, the imaging direction angle difference θ1, which is the relative angle of the second imaging direction D02 with respect to the first imaging direction D01, is 180°. Therefore, the second imageable range X2 of the second roadside camera 10b is located forward of the first imageable range X1 of the first roadside camera 10a. The imageable range X of the roadside camera 10 is determined by the imaging direction D0 and imageable range of the roadside camera 10. The area between the first imageable range X1 and the second imageable range X2 is a blind spot B that the roadside camera 10 cannot capture. The "forward side" refers to the direction downstream of the target vehicle 40's traveling path in the direction of travel. The "rear side" refers to the direction upstream of the target vehicle 40's traveling path in the direction of travel. The first imageable direction D01 is the imageable direction of the first roadside camera 10a when the target vehicle 40 leaves the first imageable range X1 of the first roadside camera 10a. The second imaging direction D02 is the imaging direction of the second roadside camera 10b when the target vehicle 40 enters the second imaging range X2 of the second roadside camera 10b. The image V0 captured by the roadside camera 10b may be a video (image) or a still image that is continuously switched at a predetermined interval.

[0011] 1 is a display for a user to remotely monitor a target vehicle 40 that is a monitoring target. The user can select a target vehicle 40 to be monitored from among the vehicles displayed on the display 20 and transmit the selection result to the server 30 via a communication unit (not shown) of the display 20. A vehicle that can communicate with the server 30 may be set in advance as the target vehicle 40. The user may communicate with the driver of the target vehicle 40 while checking the driving status of the target vehicle 40 via the display 20, and may assist the driver in driving the target vehicle 40. The user may remotely control the driving of the target vehicle 40 while checking the driving status of the target vehicle 40 via the display 20.

[0012] The server 30 also has a map database 1, an infrastructure database 2, and a controller 3. The map database 1 stores high-precision three-dimensional map information including location information of various facilities and specific points. The infrastructure database 2 also stores roadside camera information for each of the multiple roadside cameras 10 (a first roadside camera 10a and a second roadside camera 10b). The roadside camera information includes the imaging direction D0 and imaging range of each roadside camera 10. The roadside camera information may also include the installation position of each roadside camera 10. The controller 3 also controls the image to be displayed on the display 20 based on the map information in the map database 1, the roadside camera information in the infrastructure database 2, and the images captured by the roadside cameras 10.

[0013] The target vehicle 40 has an on-board sensor 41 and a vehicle position detection unit 42. The on-board sensor 41 is a camera or radar mounted on the target vehicle 40. The on-board sensor 41 detects moving objects and obstacles around the target vehicle 40. The moving objects include, for example, other vehicles, bicycles, or pedestrians. The vehicle position detection unit 42 is composed of a GPS unit, a gyro sensor, and a vehicle speed sensor. The vehicle position detection unit 42 detects the current vehicle position of the target vehicle 40.

[0014] The controller 3 of the server 30 has a communication unit 31, a captured image acquisition unit 32, a roadside camera information acquisition unit 33, a vehicle position acquisition unit 34, a vehicle position determination unit 35, and an image display control unit 36. The controller 3 executes the functions of the communication unit 31, the captured image acquisition unit 32, the roadside camera information acquisition unit 33, the vehicle position acquisition unit 34, the vehicle position determination unit 35, and the image display control unit 36 ​​by using the CPU to execute a program stored in the ROM.

[0015] The communication unit 31 exchanges information with the first roadside camera 10a, the second roadside camera 10b, and the display 20 via the communication network 4. Specifically, the communication unit 31 receives data of the captured image V0 (see FIGS. 3 and 6) from the first roadside camera 10a and the second roadside camera 10b. The communication unit 31 also transmits image data to the display 20 based on a control command from the image display control unit 36.

[0016] Furthermore, the communication unit 31 may exchange information with the target vehicle 40 via the communication network 4. Specifically, the communication unit 31 receives the detection results of the on-board sensor 41 from the target vehicle 40. The communication unit 31 also receives vehicle information indicating the driving status of the target vehicle 40 from the target vehicle 40. The vehicle information includes the vehicle position of the target vehicle 40. The vehicle information may also include at least one of the vehicle speed, acceleration, direction, turn signal status, and stop lamp status of the target vehicle 40.

[0017] The captured image acquisition unit 32 acquires a first captured image V01 (see Figure 3) captured by the first roadside camera 10a and a second captured image V02 (see Figure 6) captured by the second roadside camera 10b based on the image data received by the communication unit 31.

[0018] In addition, the roadside camera information acquisition unit 33 acquires roadside camera information of the first roadside camera 10 a and the second roadside camera 10 b from the infrastructure database 2 .

[0019] Furthermore, the vehicle position acquisition unit 34 acquires the vehicle position of the target vehicle 40 based on the captured image V0 acquired by the captured image acquisition unit 32, with reference to the map information in the map database 1 and the roadside camera information in the infrastructure database 2. Specifically, as shown in Fig. 3, when the target vehicle 40 is included in the first captured image V01 of the first roadside camera 10a, the vehicle position acquisition unit 34 identifies the vehicle position of the target vehicle 40 within the first imageable range X1 with reference to the map information and roadside camera information. Furthermore, as shown in Fig. 6, when the target vehicle 40 is included in the second captured image V02 of the second roadside camera 10b, the vehicle position acquisition unit 34 identifies the vehicle position of the target vehicle 40 within the second imageable range X2 with reference to the map information and roadside camera information. Furthermore, when the target vehicle 40 is traveling in the blind spot area B and the controller 3 cannot acquire an image, the vehicle position acquisition unit 34 estimates the vehicle position of the target vehicle 40 based on the vehicle speed and direction when the target vehicle 40 exits the first imageable range X1. That is, the vehicle position acquisition unit 34 acquires vehicle information of the target vehicle 40 from the captured image V0, and acquires the vehicle position of the target vehicle 40 based on the vehicle information.

[0020] In addition, the vehicle position acquisition unit 34 may acquire the vehicle position of the target vehicle 40 based on the vehicle information received by the communication unit 31 from the target vehicle 40 .

[0021] Furthermore, the vehicle position determination unit 35 determines whether the target vehicle 40 is traveling within the image capture range X (first image capture range X1 or second image capture range X2) of the roadside camera 10, based on the roadside camera information acquired from the roadside camera information acquisition unit 33 and the vehicle position acquired from the vehicle position acquisition unit 34. That is, the vehicle position determination unit 35 determines whether the target vehicle 40 is traveling within the image capture range X of the roadside camera 10, based on the image capture range X of the roadside camera 10 and the vehicle position relative to the road 5.

[0022] Furthermore, the image display control unit 36 ​​controls the image data that the communication unit 31 transmits to the display 20 based on the determination result of the vehicle position determination unit 35. That is, the image display control unit 36 ​​controls the image to be displayed on the display 20. The image display control unit 36 ​​has a captured image display control unit 36a and a virtual image display control unit 36b.

[0023] When the target vehicle 40 is traveling within the imageable range X, the captured image display control unit 36a generates a control command to display a captured image V0 including the target vehicle 40 on the display 20. Specifically, when the target vehicle 40 is traveling within a first imageable range X1, the captured image display control unit 36a displays a first captured image V01 on the display 20. Furthermore, when the target vehicle 40 is traveling within a second imageable range X2, the captured image display control unit 36a displays a second captured image V02 on the display 20. In other words, the captured image display control unit 36a selects an image including the target vehicle 40 from the multiple captured images V0 captured by each of the multiple roadside cameras 10 and displays the image on the display 20.

[0024] When the target vehicle 40 is traveling outside the image capture range X, the virtual image display control unit 36b displays a virtual image V1 of the target vehicle 40 displayed in a predetermined display direction D1 on the display 20 (see FIGS. 4 and 5 ). The virtual image V1 is an image in which an icon or CG of the target vehicle 40 is superimposed on an omnidirectional image or a 3DCG (omnidirectional spatial image) generated based on map information including the road 5 on which the target vehicle 40 is traveling.

[0025] The virtual image display control unit 36b determines whether the first imaging direction D01 of the first roadside camera 10a differs from the second imaging direction D02 of the second roadside camera 10b when the target vehicle 40 is in the vicinity of the first roadside camera 10a. When the first imaging direction D01 and the second imaging direction D02 differ, the virtual image display control unit 36b updates (changes) the display direction D1 of the virtual image V1 one or more times so that the second display direction D12 when the target vehicle 40 is in the second vehicle position P02 forward of the first vehicle position P01 is closer to the second imaging direction D02 than the first display direction D11 when the target vehicle 40 is in the first vehicle position P01. The first vehicle position P01 is located near the position of the first roadside camera 10a, and the second vehicle position P02 is located near the position of the second roadside camera 10b. That is, the virtual image display control unit 36b updates the virtual image V1 so that the display direction D1 of the virtual image V1 approaches the second imaging direction D02 as the vehicle position of the target vehicle 40 traveling in the blind spot area B approaches the second imaging range X2.

[0026] Specifically, first, the virtual image display control unit 36b generates a first virtual image V11 including the target vehicle 40 that has exited the first image capture range X1, as shown in Fig. 4. As shown in Fig. 2, the first virtual image V11 is an image generated on the assumption that the image is captured from a first virtual image capture position P11 toward a first display direction D11. Note that a first relative angle θ21 of the first display direction D11 with respect to the first image capture direction D01 is set to a value greater than 0° and smaller than the image capture direction angle difference θ1.

[0027] The virtual image display control unit 36b also switches the first virtual image V11 shown in FIG. 4 to the second virtual image V12 shown in FIG. 5 at a predetermined time point. As shown in FIG. 2, the second virtual image V12 is an image generated assuming that the image is captured from the second virtual imaging position P12 toward the second display direction D12. The second relative angle θ22 of the second display direction D12 with respect to the first imaging direction D01 is set to a value greater than 0° and smaller than the imaging direction angular difference θ1. The second display direction D12 is closer to the second imaging direction D02 than the first display direction D11. That is, the second relative angle θ22 is greater than the first relative angle θ21 and closer to the imaging direction angular difference θ1. That is, the virtual image display control unit 36b updates the first display direction D11 of the first virtual image V11 displayed on the display 20 to the second display direction D12 of the second virtual image V12 at a predetermined time point.

[0028] That is, the image display control unit 36 ​​displays the images on the display 20 in the order of the first captured image V01 shown in Fig. 3, the first virtual image V11 shown in Fig. 4, the second virtual image V12 shown in Fig. 5, and the second captured image V02 shown in Fig. 6. As a result, to a user looking at the display 20, the target vehicle 40, which initially appears to be moving away to the back of the screen, appears to gradually approach the front of the screen as the target vehicle 40 moves.

[0029] 2, the virtual image display control unit 36b switches from the first virtual image V11 shown in FIG. 4 to the second virtual image V12 shown in FIG. 5 when the target vehicle 40 is traveling between the first vehicle position P01 and the second vehicle position P02 or when the target vehicle 40 reaches the second vehicle position P02. More specifically, the virtual image display control unit 36b calculates the blind spot traveling time from when the target vehicle 40 leaves the first image capture range X1 to when it enters the second image capture range X2. The virtual image display control unit 36b then calculates a display angle change speed by dividing the imaging direction angle difference θ1 between the first imaging direction D01 and the second imaging direction D02 by the blind spot traveling time, and changes the display direction D1 at a timing according to the display angle change speed. Furthermore, the virtual image display control unit 36b may continuously change the display direction D1 of the virtual image V1 at predetermined intervals (for example, every second) in accordance with the display angle change speed.

[0030] The virtual image display control unit 36b also acquires the exit time when the target vehicle 40 will exit the first imageable range X1, and calculates the entry time when the target vehicle 40 will enter the second imageable range X2 based on the vehicle information (such as the speed and acceleration of the target vehicle 40) and the exit time. When the target vehicle 40 is traveling between the first imageable range X1 and the second imageable range X2, information T indicating the entry time is displayed on the virtual image V1. Note that in the examples shown in FIGS. 3 and 4 , the information T is text information indicating the time remaining until the target vehicle 40 enters the second imageable range X2, but is not limited to this. The information T may also be information indicating the time when the target vehicle 40 will enter the second imageable range X2.

[0031] Furthermore, on the first virtual image V11 and the second virtual image V12, the image (CG) of the target vehicle 40 moves on the road 5 according to the vehicle position estimated by the vehicle position acquisition unit 34. That is, the first virtual image V11 and the second virtual image V12 include information indicating the vehicle position and orientation of the target vehicle 40. Furthermore, the virtual image display control unit 36b displays the target vehicle 40 on the first virtual image V11 and the second virtual image V12 by surrounding it with a frame line L. The frame line L may be displayed in a conspicuous color such as red. Furthermore, the virtual image display control unit 36b may highlight the target vehicle 40. That is, the virtual image display control unit 36b displays the target vehicle 40 in a manner that allows it to be distinguished from other vehicles. Furthermore, the virtual image display control unit 36b displays the driving route R of the target vehicle 40 superimposed on the first virtual image V11 and the second virtual image V12. The virtual image display control unit 36b also displays the first image capture range X1 on the first virtual image V11 and the second image capture range X2 on the second virtual image V12.

[0032] In addition, the virtual image display control unit 36b may display information indicating at least one of the vehicle position, orientation, turn signal status, and stop lamp status of the target vehicle 40 on the virtual image V1 based on vehicle information obtained from the target vehicle 40.

[0033] Furthermore, the virtual image display control unit 36b may determine whether or not a moving object exists around the target vehicle 40, based on the detection result of the on-board sensor 41. If a moving object exists, the virtual image display control unit 36b may display vehicle surroundings information indicating either or both of the position and the shape of the moving object on the virtual image V1.

[0034] Next, the steps of the information display method executed by the information processing system 100 will be described using the flowchart in Fig. 7. First, in step S1, the captured image acquisition unit 32 acquires a captured image V0 captured by each of the multiple roadside cameras 10. Next, in step S2, the roadside camera information acquisition unit 33 acquires roadside camera information including the imaging direction D0 and imaging range X of each of the multiple roadside cameras 10. Next, in step S3, the vehicle position acquisition unit 34 acquires the vehicle position of the target vehicle 40 based on the captured image V0 or the vehicle information acquired from the target vehicle 40.

[0035] Then, in step S4, the vehicle position determination unit 35 determines whether the target vehicle 40 is traveling within the image capture range X of the roadside camera 10 based on the image capture range X and the vehicle position.

[0036] If the vehicle position determination unit 35 determines in step S4 that the target vehicle 40 is traveling within the image capture range X of the roadside camera 10, then in step S5 the captured image display control unit 36a displays the captured image V0 including the target vehicle 40 on the display 20. Thereafter, the process proceeds to step S9.

[0037] Also, in S4, if the vehicle position determination unit 35 determines that the target vehicle 40 is traveling outside the imaging range X of the roadside camera 10, in step S6, the virtual image display control unit 36b displays on the display 20 a virtual image V1 of the target vehicle 40 displayed in a predetermined display direction D1.

[0038] Then, in step S7, the virtual image display control unit 36b determines whether the first imaging direction D01 of the first roadside camera 10a and the second imaging direction D02 of the second roadside camera 10b are different.

[0039] If the virtual image display control unit 36b determines in step S7 that "the first imaging direction D01 and the second imaging direction D02 are different," then in step S8, the virtual image display control unit 36b updates (changes) the display direction D1 of the virtual image V1 so that the second display direction D12 when the target vehicle 40 is at the second vehicle position P02 is closer to the second imaging direction D02 than the first display direction D11 when the target vehicle 40 is at the first vehicle position P01. On the other hand, if the virtual image display control unit 36b determines in step S7 that "the first imaging direction D01 and the second imaging direction D02 are the same," the virtual image display control unit 36b does not update (change) the display direction D1 of the virtual image V1. That is, the information processing system 100 skips the processing of step S8.

[0040] Next, in step S9, the information processing system 100 determines whether the target vehicle 40 has arrived at the destination. If the target vehicle 40 has arrived at the destination, that is, if remote monitoring of the target vehicle 40 has ended, the processing ends. On the other hand, if the target vehicle 40 has not arrived at the destination, the processing returns to step S1.

[0041] As described above, in the information processing system 100 of this embodiment, when the target vehicle 40 is traveling outside the imageable range X of the roadside camera 10, after the target vehicle 40 leaves the first imageable range X1 of the first roadside camera 10a, the information processing system 100 displays on the display 20 a virtual image V1 of the target vehicle 40 and the road 5 displayed in a predetermined display direction D1 relative to the road 5, as viewed from a predetermined position (first virtual image capturing position P11 or second virtual image capturing position P12) between the first roadside camera 10a and the second roadside camera 10b located ahead of the first roadside camera 10a in the direction of travel of the target vehicle 40. The information processing system 100 then determines whether a first imaging direction D01 of the first roadside camera 10a with respect to the road 5 when the target vehicle 40 leaves the first imageable range X1 of the first roadside camera 10a differs from a second imaging direction D02 with respect to the road 5 when the target vehicle 40 enters the second imageable range X2 of the second roadside camera 10b. When the first imaging direction D01 and the second imaging direction D02 differ, the information processing system 100 changes the display direction D1 of the virtual image V1 to match the position of the target vehicle 40 so that the second display direction D12 when the target vehicle 40 is at a second vehicle position P02 forward of the first vehicle position P01 and near the position of the second roadside camera 10b is closer to the second imaging direction D02 than the first display direction D11 when the target vehicle 40 is at a first vehicle position P01 near the position of the first roadside camera 10a. That is, when the first captured image V01 displayed on the display 20 is switched to the virtual image V1, the display direction D1 of the virtual image V1 gradually or continuously approaches the second capturing direction D02 from the first capturing direction D01. This prevents a sudden change in the display direction of the image displayed on the display 20 when the roadside camera 10 capturing the target vehicle 40 is switched from the first roadside camera 10a to the second roadside camera 10b. Therefore, the user can easily identify the target vehicle 40 without losing sight of it in the captured image V0 (second captured image V02) captured by the roadside camera 10 (second roadside camera 10b) after the switch.In addition, the information processing system 100 displays the virtual image V1 on the display 20 when the target vehicle 40 is not traveling within the imaging range X, so that the user can check the traveling status of the target vehicle 40 even when the roadside camera 10 cannot image the target vehicle 40.

[0042] Furthermore, the information processing system 100 acquires the exit time when the target vehicle 40 exits the first imageable range X1, calculates the entry time when the target vehicle 40 enters the second imageable range X2 based on the vehicle information and the exit time, and, when the target vehicle 40 is traveling between the first imageable range X1 and the second imageable range X2, displays the virtual image V1 including information T indicating the entry time on the display 20. This allows the user to grasp the timing (time) when the virtual image V1 switches to the second captured image V02, even while the virtual image V1 is being displayed on the display 20.

[0043] Furthermore, the information processing system 100 calculates the blind spot traveling time from when the target vehicle 40 exits the first image capture range X1 to when it enters the second image capture range X2, calculates the imaging direction angle difference θ1 of the second imaging direction D02 with respect to the first imaging direction D01, calculates the display angle change speed by dividing the imaging direction angle difference θ1 by the blind spot traveling time, and changes the display direction at a timing based on the display angle change speed. As a result, the display direction D1 of the virtual image V1 changes to approach the second imaging direction D2 in accordance with the movement of the target vehicle 40, so that the user can easily identify the target vehicle 40 even after the virtual image V1 is switched to the second captured image V02.

[0044] The virtual image V1 is an image in which an icon or CG of the target vehicle 40 is superimposed on an omnidirectional image or a 3DCG (omnidirectional spatial image) generated based on map information including the road 5 on which the target vehicle 40 is traveling. This allows the user to easily visually grasp the traveling situation of the target vehicle 40 even when the target vehicle 40 is traveling in the blind spot area B.

[0045] The virtual image V1 also includes information indicating at least one of the vehicle position, orientation, blinker status, and stop lamp status of the target vehicle 40. This allows the user to easily visually grasp the driving status of the target vehicle 40 even when the target vehicle 40 is driving in the blind spot area B.

[0046] Furthermore, the information processing system 100 may acquire the detection results of the on-board sensors 41 of the target vehicle 40, determine whether or not there is a moving object around the target vehicle 40 based on the detection results, and, if there is a moving object, display vehicle surroundings information indicating either or both of the position and shape of the moving object on the virtual image V1. This allows the user to visually confirm information about the driving environment, including the moving objects around the target vehicle 40, even when the target vehicle 40 is driving in the blind spot area B.

[0047] Furthermore, the information processing system 100 displays the target vehicle 40 included in either or both of the captured image V0 and the virtual image V1 by surrounding it with a frame line L or highlighting it, thereby displaying the target vehicle 40 in a manner that makes it distinguishable from other vehicles. This allows the user to easily identify the target vehicle 40.

[0048] Furthermore, the information processing system 100 displays the travel route R of the target vehicle 40 superimposed on either or both of the captured image V0 and the virtual image V1. This allows the user to visually grasp the travel route R of the target vehicle 40.

[0049] Furthermore, the information processing system 100 displays the captured image V0 and the virtual image V1 on the display 20 outside the target vehicle 40. This allows the user to remotely monitor the target vehicle 40.

[0050] Second Embodiment An information processing device 200 according to a second embodiment will be described with reference to Fig. 8. Note that the same reference numerals as those in Fig. 1 indicate the same or similar configurations, and therefore detailed description thereof will be omitted below. The procedure of the information processing method executed by the information processing device 200 is the same as the procedure shown in the flowchart of Fig. 7.

[0051] 8 , the information processing device 200 is mounted on a target vehicle 40. The display 20 is an in-vehicle display provided in the target vehicle 40. That is, the controller 3 of the information processing device 200 displays a captured image V0 and a virtual image V1 on the display 20 provided in the target vehicle 40. The target vehicle 40 also has a map database 1, an infrastructure database 2, an on-board sensor 41, and a vehicle position detection unit 42.

[0052] As described above, the information processing device 200 according to this embodiment displays the captured image V0 and the virtual image V1 on the display 20 provided in the target vehicle 40. This allows the driver of the target vehicle 40 to check the driving status of his / her own vehicle (target vehicle 40) on the display 20 inside the vehicle.

[0053] DESCRIPTION OF SYMBOLS 100... Information processing system 200... Information processing device 3... Controller 10... Roadside camera 20... Display 30... Server 32... Captured image acquisition unit 33... Roadside camera information acquisition unit 34... Vehicle position acquisition unit 35... Vehicle position determination unit 36a... Captured image display control unit 36b... Virtual image display control unit 40... Target vehicle D0... Capture direction D1... Display direction R... Travel route L... Frame line T... Information indicating entry time V0... Captured image V1... Virtual image X... Captureable range θ1... Capture direction angle difference

Claims

1. An information processing method that uses a controller to display an image on a screen that includes a target vehicle traveling on a road, The aforementioned controller, The images captured by each of the multiple roadside cameras installed along the aforementioned road are acquired. Acquire roadside camera information including the imaging direction and imaging range of each of the multiple roadside cameras, Based on the vehicle information of the aforementioned target vehicle, the vehicle position of the aforementioned target vehicle is obtained. Based on the imaging range of the roadside camera and the vehicle position relative to the road, it is determined whether the target vehicle is traveling within the imaging range. If the target vehicle is traveling within the image capture range, the captured image including the target vehicle is displayed on the display. If the aforementioned vehicle is traveling outside the imaging range, After the target vehicle exits the first imaging range of the first roadside camera, a virtual image of the target vehicle and the road is displayed on the display at a predetermined display direction relative to the road, as viewed from a predetermined position between the first roadside camera and a second roadside camera located in front of the first roadside camera in the direction of travel of the target vehicle. It is determined whether the first imaging direction of the first roadside camera relative to the road when the target vehicle exits the first imaging range of the first roadside camera and the second imaging direction of the second roadside camera relative to the road when the target vehicle enters the second imaging range of the second roadside camera are different. An information processing method that, when the first imaging direction and the second imaging direction are different, changes the predetermined display direction of the virtual image to match the position of the target vehicle, such that the second display direction when the target vehicle is at a second vehicle position near the position of the second roadside camera, which is in front of the first vehicle position, is closer to the second imaging direction than the first display direction when the target vehicle is at a first vehicle position near the position of the first roadside camera.

2. The aforementioned controller, The exit time at which the target vehicle leaves the first imageable range is obtained. Based on the vehicle information and the exit time, the entry time at which the target vehicle enters the second imaging range is calculated. The information processing method according to claim 1, wherein when the target vehicle is traveling between the first imageable range and the second imageable range, the virtual image including information indicating the entry time is displayed on the display.

3. The aforementioned controller, The time taken for the target vehicle to travel in the blind spot from the time it leaves the first imaging range until it enters the second imaging range is calculated. The angular difference between the second imaging direction and the first imaging direction is calculated. The display angle change speed is calculated by dividing the aforementioned imaging direction angle difference by the blind spot travel time. The information processing method according to claim 1 or 2, wherein the display direction is changed at a timing based on the display angle change speed.

4. The information processing method according to claim 1 or 2, wherein the virtual image includes information indicating at least one of the vehicle position, orientation, turn signal status, and brake lamp status of the target vehicle.

5. The aforementioned controller, The detection results from the on-board sensors of the aforementioned vehicle are obtained, Based on the detection results, it is determined whether or not there are moving objects around the target vehicle. The information processing method according to claim 1 or 2, wherein if the moving object exists, vehicle surrounding information indicating either or both of the position and shape of the moving object is displayed on the virtual image.

6. The information processing method according to claim 1 or 2, wherein the controller displays the target vehicle, which is included in either or both of the captured image and the virtual image, by surrounding it with a frame or by highlighting it, thereby displaying the target vehicle in a manner that allows it to be distinguished from other vehicles.

7. The aforementioned controller, The information processing method according to claim 1 or 2, wherein the travel route of the target vehicle is superimposed and displayed on either the captured image or the virtual image or both.

8. The information processing method according to claim 1 or 2, wherein the controller displays the captured image and the virtual image on the display outside the target vehicle.

9. The information processing method according to claim 1 or 2, wherein the controller displays the captured image and the virtual image on the display provided on the target vehicle.

10. A display that shows images including target vehicles traveling on the road, Multiple roadside cameras installed along the aforementioned road, The system includes the aforementioned display and a server capable of communicating with each of the multiple roadside cameras, The aforementioned server, An image acquisition unit that acquires images captured by each of the multiple roadside cameras, A roadside camera information acquisition unit acquires roadside camera information including the imaging direction and imaging range of each of the multiple roadside cameras, A vehicle position acquisition unit that acquires the vehicle position of the target vehicle based on the vehicle information of the target vehicle, A vehicle position determination unit determines whether the target vehicle is traveling within the image-capable range based on the image-capable range of the roadside camera and the vehicle position relative to the road, When the target vehicle is traveling within the image capture range, the image capture display control unit displays the captured image including the target vehicle on the display, If the target vehicle is traveling outside the image capture range, the system includes a virtual image display control unit that, after the target vehicle has left the first image capture range of the first roadside camera, displays a virtual image of the target vehicle and the road on the display, viewed from a predetermined position between the first roadside camera and a second roadside camera located in front of the first roadside camera in the direction of travel of the target vehicle, in a predetermined display direction relative to the road. The virtual image display control unit, It is determined whether the first imaging direction of the first roadside camera relative to the road when the target vehicle exits the first imaging range of the first roadside camera and the second imaging direction of the second roadside camera relative to the road when the target vehicle enters the second imaging range of the second roadside camera are different. An information processing system that, when the first imaging direction and the second imaging direction are different, changes the predetermined display direction of the virtual image to match the position of the target vehicle, such that the second display direction when the target vehicle is at a second vehicle position near the position of the second roadside camera, which is in front of the first vehicle position, is closer to the second imaging direction than the first display direction when the target vehicle is at a first vehicle position near the position of the first roadside camera.

11. An information processing device that displays an image on a display that includes a target vehicle traveling on a road, An image acquisition unit that acquires images captured by each of the multiple roadside cameras installed along the aforementioned road, A roadside camera information acquisition unit acquires roadside camera information including the imaging direction and imaging range of each of the multiple roadside cameras, A vehicle position acquisition unit that acquires the vehicle position of the target vehicle based on the vehicle information of the target vehicle, A vehicle position determination unit determines whether the target vehicle is traveling within the image-capable range based on the image-capable range of the roadside camera and the vehicle position relative to the road, When the target vehicle is traveling within the image capture range, the image capture display control unit displays the captured image including the target vehicle on the display, If the target vehicle is traveling outside the image capture range, the system includes a virtual image display control unit that, after the target vehicle has left the first image capture range of the first roadside camera, displays a virtual image of the target vehicle and the road on the display, viewed from a predetermined position between the first roadside camera and a second roadside camera located in front of the first roadside camera in the direction of travel of the target vehicle, in a predetermined display direction relative to the road. The virtual image display control unit, It is determined whether the first imaging direction of the first roadside camera relative to the road when the target vehicle exits the first imaging range of the first roadside camera and the second imaging direction of the second roadside camera relative to the road when the target vehicle enters the second imaging range of the second roadside camera are different. An information processing device that, when the first imaging direction and the second imaging direction are different, changes the predetermined display direction of the virtual image to match the position of the target vehicle, such that the second display direction when the target vehicle is at a second vehicle position near the position of the second roadside camera, which is in front of the first vehicle position, is closer to the second imaging direction than the first display direction when the target vehicle is at a first vehicle position near the position of the first roadside camera.