Information processing method, information processing system, and information processing device.

Abstract

An information processing system 100 displays a captured image V0 on a display 20 if a target vehicle 40 is traveling in an imaging-enabled range X of each roadside camera 10, displays a virtual image V1 on the display 20 if the target vehicle 40 is traveling in a range other than the imaging-enabled range X, determines whether or not a first imaging direction D01 of a first roadside camera 10a and a second imaging direction D02 of a second roadside camera 10b located on the front side of the first roadside camera 10a are different, and when the first imaging direction D01 and the second imaging direction D02 are different, changes a display direction D1 of the virtual image V1 such that a second display direction D12 when the target vehicle 40 is located at a second vehicle position P02 on the front side of a first vehicle position P01 is closer to the second imaging direction D02 than a first display direction D11 when the target vehicle 40 is located at a first vehicle position P01.

Description

[Technical Field]

[0001] The present invention relates to an information processing method, an information processing system, and an information processing apparatus. [Background Art]

[0002] Patent Literature 1 describes a moving object monitoring system that images a vehicle to be monitored by monitoring cameras installed at a plurality of locations. The moving object monitoring system described in Patent Literature 1 appropriately selects, from among a plurality of monitoring cameras, a monitoring camera capable of capturing an image of the vehicle to be monitored, and performs imaging so that the vehicle to be monitored can be tracked and monitored. [Prior Art Documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Unexamined Patent Publication No. 2006-163764 [Summary of the Invention] [Problem to be Solved by the Invention]

[0004] However, in the moving object monitoring system of Patent Document 1, when the monitoring camera that performs imaging is switched in accordance with the movement of the target vehicle, a user viewing the captured image cannot immediately grasp from which direction the image is captured, and may lose sight of the target vehicle.

[0005] A problem to be solved by the present invention is to provide an information processing method, an information processing system, and an information processing apparatus that allow a user viewing a captured image after switching to easily identify a target vehicle when switching a roadside camera (infrastructure camera) that images the target vehicle. [Means for Solving the Problem]

[0006] The present invention solves the above problem by, when the target vehicle is traveling outside the imaging range, displaying a virtual image of the target vehicle and the road on the display in a predetermined display direction relative to the road, determining whether the first imaging direction of the first roadside camera relative to the road when the target vehicle exits the first imaging range of the first roadside camera and the second imaging direction of the second roadside camera relative to the road when the target vehicle enters the second imaging range of the second roadside camera located in front of the target vehicle's direction of travel of the first roadside camera, and if the first imaging direction and the second imaging direction are different, changing the predetermined display direction of the virtual image to match the position of the target vehicle so that the second display direction when the target vehicle is at the second vehicle position, in front of the first vehicle position and near the position of the second roadside camera, is closer to the second imaging direction than the first display direction when the target vehicle is at the first vehicle position near the position of the first roadside camera. [Effects of the Invention]

[0007] According to the present invention, when switching roadside cameras that capture images of a target vehicle, the user viewing the image after the switch can easily identify the target vehicle. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing the configuration of the information processing system according to the first embodiment. [Figure 2] This diagram shows an example of the relative positions of the first roadside camera, the second roadside camera, and the target vehicle on a map. [Figure 3] Figure 1 shows an example of an image captured by the first roadside camera of the information processing system. [Figure 4] Figure 1 shows an example of a virtual image displayed on a screen by the virtual image display control unit of the information processing system shown. [Figure 5] Figure 1 shows an example of a virtual image displayed on a screen by the virtual image display control unit of the information processing system shown. [Figure 6]Figure 1 shows an example of an image captured by the second roadside camera of the information processing system. [Figure 7] Figure 1 is a flowchart showing the steps of the information processing method performed by the information processing system. [Figure 8] This is a block diagram showing the configuration of the information processing device according to the second embodiment. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described below with reference to the drawings. 《First Embodiment》 The information processing system 100 according to the first embodiment will be described with reference to Figures 1 to 7. The information processing system 100 shown in Figure 1 includes a plurality of roadside cameras 10, a display 20, and a server 30. The information processing system 100 uses the controller 3 of the server 30 to display an image including a target vehicle 40 (see Figure 2) traveling on the road 5 on the display 20 located outside the target vehicle 40. The information processing system 100 can communicate with the plurality of roadside cameras 10 and the display 20 via the communication network 4. Also, as shown by the dashed line, the server 30 may also be able to communicate with the target vehicle 40 via the communication network 4. The number of plurality of roadside cameras 10 is not particularly limited as long as there are two or more, but in this embodiment, for the sake of explanation, the plurality of roadside cameras 10 will be described as being composed of two cameras (a first roadside camera 10a and a second roadside camera 10b). The controller 3 of the server 30 may transmit the captured image V0 and virtual image V1 data to the target vehicle 40 and display the captured image V0 and virtual image V1 on an in-vehicle display provided in the target vehicle 40.

[0010] The roadside cameras 10 (the first roadside camera 10a and the second roadside camera 10b) are infrastructure cameras positioned along the road 5 to capture images of the road 5. In this embodiment, the imaging direction D0 of the roadside camera 10 (the first imaging direction D01 and the second imaging direction D02) is fixed, but is not limited to this, and the roadside camera 10 may be mounted so as to be rotatable, and the imaging direction D0 may be changed as appropriate. In the example shown in Figure 2, the second roadside camera 10b is located in front of the first roadside camera 10a. The first imaging direction D01 of the first roadside camera 10a is the direction towards the front of the target vehicle 40's travel path. On the other hand, the second imaging direction D02 of the second roadside camera 10b is the direction towards the rear of the target vehicle 40's travel path. That is, the imaging direction angle difference θ1, which is the relative angle of the second imaging direction D02 with respect to the first imaging direction D01, is 180°. Therefore, the second imaging range X2 of the second roadside camera 10b is located in front of the first imaging range X1 of the first roadside camera 10a. The imaging range X of the roadside camera 10 is determined by the imaging direction D0 and imaging range of the roadside camera 10. The area between the first imaging range X1 and the second imaging range X2 is a blind spot area B that the roadside camera 10 cannot image. Furthermore, "front side" refers to the direction downstream of the vehicle 40 in the direction of travel along the vehicle's travel path. Also, "rear side" refers to the direction upstream of the vehicle 40 in the direction of travel along the vehicle's travel path. Furthermore, the first imaging direction D01 is the imaging direction of the first roadside camera 10a when the target vehicle 40 exits the first imaging range X1 of the first roadside camera 10a. Furthermore, the second imaging direction D02 is the imaging direction of the second roadside camera 10b when the target vehicle 40 enters the second imaging range X2 of the second roadside camera 10b. Furthermore, the images V0 captured by the roadside camera 10 may be video (video footage), or they may be still images that switch continuously at predetermined intervals.

[0011] Furthermore, the display 20 shown in Figure 1 is a display for the user to remotely monitor the target vehicle 40 that is being monitored. The user can select the target vehicle 40 to be monitored from the vehicles displayed on the display 20 and send the selection result to the server 30 via the communication unit (not shown) of the display 20. In addition, vehicles that can communicate with the server 30 may be set in advance as target vehicles 40. The user may also communicate with the driver of the target vehicle 40 while checking the driving status of the target vehicle 40 via the display 20 and assist in the driving of the target vehicle 40. In addition, the user may remotely control the driving of the target vehicle 40 while checking the driving status of the target vehicle 40 via the display 20.

[0012] The server 30 also includes a map database 1, an infrastructure database 2, and a controller 3. The map database 1 stores three-dimensional high-precision map information, including location information for various facilities and specific points. The infrastructure database 2 stores roadside camera information for each of the multiple roadside cameras 10 (a first roadside camera 10a and a second roadside camera 10b). The roadside camera information includes the imaging direction D0 and imaging range of each roadside camera 10. The roadside camera information may also include the installation location of each roadside camera 10. The controller 3 controls the image displayed on the display 20 based on the map information from the map database 1, the roadside camera information from the infrastructure database 2, and the images captured by the roadside cameras 10.

[0013] The target vehicle 40 has an on-board sensor 41 and a vehicle position detection unit 42. The on-board sensor 41 is a camera or radar mounted on the target vehicle 40. The on-board sensor 41 detects moving objects and obstacles around the target vehicle 40. Moving objects include, for example, other vehicles, bicycles, or pedestrians. The vehicle position detection unit 42 consists of a GPS unit, a gyro sensor, and a vehicle speed sensor. The vehicle position detection unit 42 detects the current vehicle position of the target vehicle 40.

[0014] A controller 3 of a server 30 includes a communication unit 31, a captured image acquisition unit 32, a roadside camera information acquisition unit 33, a vehicle position acquisition unit 34, a vehicle position determination unit 35, and an image display control unit 36. The controller 3 implements the respective functions of the communication unit 31, the captured image acquisition unit 32, the roadside camera information acquisition unit 33, the vehicle position acquisition unit 34, the vehicle position determination unit 35, and the image display control unit 36 by causing a CPU to execute a program stored in a ROM.

[0015] The communication unit 31 exchanges information with a first roadside camera 10a, a second roadside camera 10b, and a display 20 via a communication network 4. Specifically, the communication unit 31 receives data of a captured image V0 (see FIGS. 3 and 6) from the first roadside camera 10a and the second roadside camera 10b. The communication unit 31 also transmits image data to the display 20 based on a control command from the image display control unit 36.

[0016] Further, the communication unit 31 may exchange information with a target vehicle 40 via the communication network 4. Specifically, the communication unit 31 receives a detection result from an in-vehicle sensor 41 from the target vehicle 40. The communication unit 31 also receives vehicle information indicating the traveling status of the target vehicle 40 from the target vehicle 40. The vehicle information includes the vehicle position of the target vehicle 40. The vehicle information may also include at least any one of the vehicle speed, acceleration, direction, turn signal state, and stop lamp state of the target vehicle 40.

[0017] The captured image acquisition unit 32 acquires a first captured image V01 (see FIG. 3) captured by the first roadside camera 10a and a second captured image V02 (see FIG. 6) captured by the second roadside camera 10b, based on the image data received by the communication unit 31.

[0018] Further, the roadside camera information acquisition unit 33 acquires roadside camera information of the first roadside camera 10a and the second roadside camera 10b from an infrastructure database 2.

[0019] Furthermore, the vehicle position acquisition unit 34 refers to the map information in the map database 1 and the roadside camera information in the infrastructure database 2 to acquire the vehicle position of the target vehicle 40 based on the captured image V0 acquired by the image acquisition unit 32. Specifically, as shown in Figure 3, if the target vehicle 40 is included in the first captured image V01 of the first roadside camera 10a, the vehicle position acquisition unit 34 refers to the map information and the roadside camera information to identify the vehicle position of the target vehicle 40 within the first imageable range X1. Also, as shown in Figure 6, if the target vehicle 40 is included in the second captured image V02 of the second roadside camera 10b, the vehicle position acquisition unit 34 refers to the map information and the roadside camera information to identify the vehicle position of the target vehicle 40 within the second imageable range X2. Furthermore, if the target vehicle 40 is traveling in the blind spot area B and the controller 3 cannot acquire the captured image, the vehicle position acquisition unit 34 estimates the vehicle position of the target vehicle 40 based on the vehicle speed and direction when the target vehicle 40 exits the first imageable range X1. That is, the vehicle position acquisition unit 34 acquires vehicle information of the target vehicle 40 from the captured image V0 and acquires the vehicle position of the target vehicle 40 based on the vehicle information.

[0020] Furthermore, the vehicle position acquisition unit 34 may acquire the vehicle position of the target vehicle 40 based on the vehicle information received from the target vehicle 40 by the communication unit 31.

[0021] Furthermore, the vehicle position determination unit 35 determines whether the target vehicle 40 is traveling within the imaging range X of the roadside camera 10 (first imaging range X1 or second imaging range X2) based on the roadside camera information acquired from the roadside camera information acquisition unit 33 and the vehicle position acquired from the vehicle position acquisition unit 34. In other words, the vehicle position determination unit 35 determines whether the target vehicle 40 is traveling within the imaging range X of the roadside camera 10 based on the imaging range X of the roadside camera 10 relative to the road 5 and the vehicle position.

[0022] Furthermore, the image display control unit 36 ​​controls the image data that the communication unit 31 transmits to the display 20 based on the determination result of the vehicle position determination unit 35. In other words, the image display control unit 36 ​​controls the image to be displayed on the display 20. The image display control unit 36 ​​includes an captured image display control unit 36a and a virtual image display control unit 36b.

[0023] The image capture display control unit 36a generates a control command to display the captured image V0, which includes the target vehicle 40, on the display 20 when the target vehicle 40 is traveling within the image capture range X. Specifically, the image capture display control unit 36a displays the first captured image V01 on the display 20 when the target vehicle 40 is traveling within the first image capture range X1. Furthermore, the image capture display control unit 36a displays the second captured image V02 on the display 20 when the target vehicle 40 is traveling within the second image capture range X2. In other words, the image capture display control unit 36a selects an image containing the target vehicle 40 from the multiple captured images V0 captured by each of the multiple roadside cameras 10 and displays it on the display 20.

[0024] If the target vehicle 40 is traveling outside the image capture range X, the virtual image display control unit 36b displays a virtual image V1 of the target vehicle 40 on the display 20 in a predetermined display direction D1 (see Figures 4 and 5). The virtual image V1 is an image created by compositing an icon or CG of the target vehicle 40 onto an all-around image or 3DCG (all-around spatial image) generated based on map information including the road 5 on which the target vehicle 40 is traveling.

[0025] The virtual image display control unit 36b controls the first imaging direction D01 of the first roadside camera 10a and , theThe system determines whether the first imaging direction D01 is different from the second imaging direction D02 of the second roadside camera 10b. Then, as shown in Figure 2, the virtual image display control unit 36b updates (changes) the display direction D1 of the virtual image V1 once or more, so that the second display direction D12 when the target vehicle 40 is at the second vehicle position P02, which is in front of the first vehicle position P01, is closer to the second imaging direction D02 than the first display direction D11 when the target vehicle 40 is at the first vehicle position P01. The first vehicle position P01 is a position near the position of the first roadside camera 10a, and the second vehicle position P02 is a position near the position of the second roadside camera 10b. In other words, the virtual image display control unit 36b updates the virtual image V1 so that the display direction D1 of the virtual image V1 approaches the second imaging direction D02 as the vehicle position of the target vehicle 40 traveling in the blind spot area B approaches the second imaging range X2.

[0026] Specifically, first, the virtual image display control unit 36b generates a first virtual image V11 that includes the target vehicle 40 that has exited the first imageable range X1, as shown in Figure 4. As shown in Figure 2, the first virtual image V11 is an image generated assuming that it was captured from the first virtual imaging position P11 toward the first display direction D11. The first relative angle θ21 of the first display direction D11 with respect to the first imaging direction D01 is set to be greater than 0° and less than the imaging direction angle difference θ1.

[0027] Furthermore, the virtual image display control unit 36b switches the first virtual image V11 shown in Figure 4 to the second virtual image V12 shown in Figure 5 at a predetermined time. As shown in Figure 2, the second virtual image V12 is an image generated assuming that it was captured from the second virtual imaging position P12 toward the second display direction D12. The second relative angle θ22 of the second display direction D12 with respect to the first imaging direction D01 is set to be greater than 0° and less than the imaging direction angle difference θ1. Also, the second display direction D12 is closer to the second imaging direction D02 than to the first display direction D11. That is, the second relative angle θ22 is greater than the first relative angle θ21 and close to the imaging direction angle difference θ1. In other words, the virtual image display control unit 36b updates the first display direction D11 of the first virtual image V11 displayed on the display 20 to the second display direction D12 of the second virtual image V12 at a predetermined time.

[0028] In other words, the image display control unit 36 ​​displays the images on the display 20 in the following order: the first captured image V01 shown in Figure 3, the first virtual image V11 shown in Figure 4, the second virtual image V12 shown in Figure 5, and the second captured image V02 shown in Figure 6. As a result, to the user viewing the display 20, the target vehicle 40, which initially appears to be moving away from the screen, gradually appears to be moving closer to the screen as the target vehicle 40 moves.

[0029] In the example shown in Figure 2, the virtual image display control unit 36b switches the first virtual image V11 shown in Figure 4 to the second virtual image V12 shown in Figure 5 when the target vehicle 40 is traveling between the first vehicle position P01 and the second vehicle position P02, or when the target vehicle 40 reaches the second vehicle position P02. More specifically, the virtual image display control unit 36b calculates the blind spot travel time from when the target vehicle 40 exits the first imageable range X1 until it enters the second imageable range X2. The virtual image display control unit 36b then calculates the display angle change speed by dividing the imaging direction angle difference θ1 between the first imaging direction D01 and the second imaging direction D02 by the blind spot travel time, and changes the display direction D1 at a timing corresponding to the display angle change speed. Furthermore, the virtual image display control unit 36b may continuously change the display direction D1 of the virtual image V1 at predetermined intervals (for example, every second) in accordance with the display angle change speed.

[0030] Furthermore, the virtual image display control unit 36b acquires the exit time when the target vehicle 40 leaves the first imageable range X1, and calculates the entry time when the target vehicle 40 enters the second imageable range X2 based on the vehicle information (vehicle speed, acceleration, etc. of the target vehicle 40) and the exit time. When the target vehicle 40 is traveling between the first imageable range X1 and the second imageable range X2, information T indicating the entry time is displayed on the virtual image V1. In the examples shown in Figures 3 and 4, information T is text information indicating the remaining time until the target vehicle 40 enters the second imageable range X2, but it is not limited to this, and information T may also be information indicating the time when the target vehicle 40 enters the second imageable range X2.

[0031] Furthermore, on the first virtual image V11 and the second virtual image V12, the image (CG) of the target vehicle 40 moves along the road 5 according to the vehicle position estimated by the vehicle position acquisition unit 34. That is, the first virtual image V11 and the second virtual image V12 include information indicating the vehicle position and orientation of the target vehicle 40. The virtual image display control unit 36b also displays the target vehicle 40 on the first virtual image V11 and the second virtual image V12 surrounded by a frame line L. The frame line L may be displayed in a conspicuous color such as red. The virtual image display control unit 36b may also highlight the target vehicle 40. That is, the virtual image display control unit 36b displays the target vehicle 40 in a manner that makes it distinguishable from other vehicles. The virtual image display control unit 36b also superimposes and displays the travel path R of the target vehicle 40 on the first virtual image V11 and the second virtual image V12. Furthermore, the virtual image display control unit 36b displays the first image-capable range X1 on the first virtual image V11. Also, the virtual image display control unit 36b displays the second image-capable range X2 on the second virtual image V12.

[0032] Furthermore, the virtual image display control unit 36b may display information on the virtual image V1 indicating at least one of the vehicle position, orientation, turn signal status, and brake lamp status of the target vehicle 40, based on vehicle information acquired from the target vehicle 40.

[0033] Furthermore, the virtual image display control unit 36b may determine whether or not there are moving objects around the target vehicle 40 based on the detection results of the on-board sensor 41. If there are moving objects, the virtual image display control unit 36b may display vehicle surrounding information in the virtual image V1 that shows either or both of the position and shape of the moving objects.

[0034] Next, the procedure for the information display method performed by the information processing system 100 will be explained using the flowchart in Figure 7. First, in step S1, the image acquisition unit 32 acquires the captured image V0 captured by each of the multiple roadside cameras 10. Next, in step S2, the roadside camera information acquisition unit 33 acquires roadside camera information, including the imaging direction D0 and imaging range X of each of the multiple roadside cameras 10. Next, in step S3, the vehicle position acquisition unit 34 acquires the vehicle position of the target vehicle 40 based on the vehicle information acquired from the captured image V0 or the target vehicle 40.

[0035] Then, in step S4, the vehicle position determination unit 35 determines whether the target vehicle 40 is traveling within the imaging range X of the roadside camera 10, based on the imaging range X and the vehicle position.

[0036] In step S4, if the vehicle position determination unit 35 determines that "the target vehicle 40 is traveling within the imaging range X of the roadside camera 10", then in step S5, the image display control unit 36a displays the image V0 including the target vehicle 40 on the display 20. The process then proceeds to step S9.

[0037] Furthermore, in step S4, if the vehicle position determination unit 35 determines that "the target vehicle 40 is traveling outside the imaging range X of the roadside camera 10", then in step S6, the virtual image display control unit 36b displays a virtual image V1 of the target vehicle 40, displayed in a predetermined display direction D1, on the display 20.

[0038] Then, in step S7, the virtual image display control unit 36b determines whether the first imaging direction D01 of the first roadside camera 10a and the second imaging direction D02 of the second roadside camera 10b are different.

[0039] In step S7, if the virtual image display control unit 36b determines that "the first imaging direction D01 and the second imaging direction D02 are different", then in step S8, the virtual image display control unit 36b updates (changes) the display direction D1 of the virtual image V1 so that the second display direction D12 when the target vehicle 40 is at the second vehicle position P02 is closer to the second imaging direction D02 than the first display direction D11 when the target vehicle 40 is at the first vehicle position P01. On the other hand, if in step S7 the virtual image display control unit 36b determines that "the first imaging direction D01 and the second imaging direction D02 are the same", the virtual image display control unit 36b does not update (change) the display direction D1 of the virtual image V1. In other words, the information processing system 100 skips the processing in step S8.

[0040] Next, in step S9, the information processing system 100 determines whether the target vehicle 40 has arrived at its destination. If the target vehicle 40 has arrived at its destination, that is, if remote monitoring of the target vehicle 40 has ended, the process ends. On the other hand, if the target vehicle 40 has not arrived at its destination, the process returns to step S1.

[0041] Based on the above, if the target vehicle 40 is traveling outside the imaging range X of the roadside camera 10, the information processing system 100 according to this embodiment displays a virtual image V1 of the target vehicle 40 and the road 5 on the display 20, viewed from a predetermined position (first virtual imaging position P11 or second virtual imaging position P12) between the first roadside camera 10a and the second roadside camera 10b located in front of the target vehicle 40 in the direction of travel of the first roadside camera 10a, with a predetermined display direction D1 relative to the road 5. The information processing system 100 then determines whether the first imaging direction D01 of the first roadside camera 10a with respect to the road 5 when the target vehicle 40 exits the first imaging range X1 of the first roadside camera 10a is different from the second imaging direction D02 with respect to the road 5 when the target vehicle 40 enters the second imaging range X2 of the second roadside camera 10b. If the first imaging direction D01 and the second imaging direction D02 are different, the information processing system 100 changes the display direction D1 of the virtual image V1 to match the position of the target vehicle 40, such that the second display direction D12 when the target vehicle 40 is at the second vehicle position P02, near the position of the second roadside camera 10b, which is in front of the first vehicle position P01, is closer to the second imaging direction D02 than the first display direction D11 when the target vehicle 40 is at the first vehicle position P01 near the position of the first roadside camera 10a. In other words, when the first captured image V01 displayed on the display 20 is switched to the virtual image V1, the display direction D1 of the virtual image V1 gradually or continuously approaches the second capturing direction D02 from the first capturing direction D01. This prevents a sudden change in the orientation of the image displayed on the display 20 when the roadside camera 10 capturing the target vehicle 40 switches from the first roadside camera 10a to the second roadside camera 10b. As a result, the user can easily identify the target vehicle 40 without losing sight of it in the captured image V0 (second captured image V02) captured by the roadside camera 10 after the switch (second roadside camera 10b). Furthermore, the information processing system 100 displays a virtual image V1 on the display 20 when the target vehicle 40 is not traveling within the image capture range X, so the user can check the driving status of the target vehicle 40 even when the roadside camera 10 cannot capture the target vehicle 40.

[0042] Furthermore, the information processing system 100 acquires the exit time when the target vehicle 40 leaves the first imageable range X1, calculates the entry time when the target vehicle 40 enters the second imageable range X2 based on the vehicle information and exit time, and displays a virtual image V1 containing information T indicating the entry time on the display 20 when the target vehicle 40 is traveling between the first imageable range X1 and the second imageable range X2. This allows the user to understand the timing (time) when the virtual image V1 switches to the second imaged image V02, even while the virtual image V1 is displayed on the display 20.

[0043] Furthermore, the information processing system 100 calculates the blind spot travel time from when the target vehicle 40 exits the first imageable range X1 until it enters the second imageable range X2, calculates the difference in imaging direction angle θ1 between the second imaging direction D02 and the first imaging direction D01, calculates the display angle change speed by dividing the imaging direction angle difference θ1 by the blind spot travel time, and changes the display direction at a timing based on the display angle change speed. As a result, the display direction D1 of the virtual image V1 changes to approach the second imaging direction D2 in accordance with the movement of the target vehicle 40, so that the user can easily identify the target vehicle 40 even after the virtual image V1 switches to the second image V02.

[0044] Furthermore, the virtual image V1 is an image created by compositing an icon or CG of the target vehicle 40 onto an all-around image or 3DCG (all-around spatial image) generated based on map information including the road 5 on which the target vehicle 40 is traveling. This allows the user to easily visually understand the driving status of the target vehicle 40, even when the target vehicle 40 is traveling in the blind spot area B.

[0045] Furthermore, the virtual image V1 includes information indicating at least one of the vehicle position, orientation, turn signal status, and brake light status of the target vehicle 40. This allows the user to easily visually understand the driving status of the target vehicle 40, even when the target vehicle 40 is driving in the blind spot area B.

[0046] Furthermore, the information processing system 100 may acquire the detection results from the on-board sensors 41 of the target vehicle 40, determine whether or not there are moving objects around the target vehicle 40 based on the detection results, and if there are moving objects, display vehicle surrounding information showing either or both of the position and shape of the moving objects in the virtual image V1. This allows the user to visually confirm information about the driving environment, including moving objects around the target vehicle 40, even when the target vehicle 40 is driving in the blind spot area B.

[0047] Furthermore, the information processing system 100 displays the target vehicle 40, which is included in either or both of the captured image V0 and the virtual image V1, by enclosing it in a frame L or by highlighting it, thereby displaying the target vehicle 40 in a manner that makes it distinguishable from other vehicles. This allows the user to easily identify the target vehicle 40.

[0048] Furthermore, the information processing system 100 overlays and displays the travel route R of the target vehicle 40 on either the captured image V0 or the virtual image V1, or on both. This allows the user to visually understand the travel route R of the target vehicle 40.

[0049] Furthermore, the information processing system 100 displays the captured image V0 and virtual image V1 on the external display 20 of the target vehicle 40. This allows the user to remotely monitor the target vehicle 40.

[0050] 《Second Embodiment》 The information processing device 200 according to the second embodiment will be described with reference to Figure 8. Note that the same symbols as those shown in Figure 1 represent the same or similar configurations, and therefore, detailed explanations will be omitted in the following description. Furthermore, the procedure for the information processing method executed by the information processing device 200 is the same as the procedure shown in the flowchart of Figure 7.

[0051] As shown in Figure 8, the information processing device 200 is mounted on the target vehicle 40. The display 20 is an in-vehicle display provided in the target vehicle 40. Specifically, the controller 3 of the information processing device 200 displays the captured image V0 and virtual image V1 on the display 20 provided in the target vehicle 40. The target vehicle 40 also has a map database 1, an infrastructure database 2, an on-board sensor 41, and a vehicle position detection unit 42.

[0052] Based on the above, the information processing device 200 according to this embodiment displays the captured image V0 and the virtual image V1 on the display 20 provided in the target vehicle 40. As a result, the driver of the target vehicle 40 can check the driving status of their vehicle (target vehicle 40) on the in-vehicle display 20. [Explanation of Symbols]

[0053] 100… Information Processing Systems 200… Information Processing Equipment 3…Controller 10…Roadside camera 20…Display 30… Server 32...Image acquisition unit 33...Roadside camera information acquisition unit 34... Vehicle position acquisition unit 35... Vehicle position determination unit 36a...Image display control unit 36b…Virtual Image Display Control Unit 40…Target vehicles D0...Imaging direction D1…Display direction R...Route L... border T... Information indicating entry time V0... Captured image V1…Virtual image X...Imageable area θ1…Angle difference in imaging direction

Claims

1. An information processing method that uses a controller to display an image on a display that includes a target vehicle traveling on a road, The aforementioned controller, The images captured by each of the multiple roadside cameras installed along the aforementioned road are acquired. Acquire roadside camera information including the imaging direction and imaging range of each of the multiple roadside cameras, Based on the vehicle information of the target vehicle, the vehicle position of the target vehicle is obtained. Based on the imaging range of the roadside camera and the vehicle position relative to the road, it is determined whether the target vehicle is traveling within the imaging range. If the target vehicle is traveling within the image capture range, the captured image including the target vehicle is displayed on the display. If the aforementioned vehicle is traveling outside the imaging range, After the target vehicle exits the first imaging range of the first roadside camera, a virtual image of the target vehicle and the road is displayed on the display at a predetermined display direction relative to the road, as viewed from a predetermined position between the first roadside camera and a second roadside camera located in front of the first roadside camera in the direction of travel of the target vehicle. It is determined whether the first imaging direction of the first roadside camera relative to the road when the target vehicle exits the first imaging range of the first roadside camera and the second imaging direction of the second roadside camera relative to the road when the target vehicle enters the second imaging range of the second roadside camera are different. An information processing method that, when the first imaging direction and the second imaging direction are different, changes the predetermined display direction of the virtual image to match the position of the target vehicle, such that the second display direction when the target vehicle is at a second vehicle position near the position of the second roadside camera, which is in front of the first vehicle position, is closer to the second imaging direction than the first display direction when the target vehicle is at a first vehicle position near the position of the first roadside camera.

2. The aforementioned controller, The exit time at which the target vehicle leaves the first imageable range is obtained. Based on the vehicle information and the exit time, the entry time at which the target vehicle enters the second image-capable range is calculated. The information processing method according to claim 1, wherein when the target vehicle is traveling between the first image-capable range and the second image-capable range, the virtual image including information indicating the entry time is displayed on the display.

3. An information processing method for displaying an image on a display that includes a target vehicle traveling on a road, using a controller, The aforementioned controller, The images captured by each of the multiple roadside cameras installed along the aforementioned road are acquired. Acquire roadside camera information including the imaging direction and imaging range of each of the multiple roadside cameras, Based on the vehicle information of the target vehicle, the vehicle position of the target vehicle is obtained. Based on the imaging range of the roadside camera and the vehicle position relative to the road, it is determined whether the target vehicle is traveling within the imaging range. If the target vehicle is traveling within the image capture range, the captured image including the target vehicle is displayed on the display. If the aforementioned vehicle is traveling outside the imaging range, After the target vehicle exits the first imaging range of the first roadside camera, a virtual image of the target vehicle and the road is displayed on the display at a predetermined display direction relative to the road, as viewed from a predetermined position between the first roadside camera and a second roadside camera located in front of the first roadside camera in the direction of travel of the target vehicle. It is determined whether the first imaging direction of the first roadside camera relative to the road when the target vehicle exits the first imaging range of the first roadside camera and the second imaging direction of the second roadside camera relative to the road when the target vehicle enters the second imaging range of the second roadside camera are different. When the first imaging direction and the second imaging direction are different, the blind spot travel time from when the target vehicle exits the first imaging range until it enters the second imaging range is calculated. The angular difference between the second imaging direction and the first imaging direction is calculated. The display angle change speed is calculated by dividing the aforementioned imaging direction angle difference by the blind spot travel time. An information processing method that changes the predetermined display direction of the virtual image in accordance with the display angle change speed, such that the second display direction when the target vehicle is at a second vehicle position near the position of the second roadside camera, which is in front of the first vehicle position, is closer to the second imaging direction than the first display direction when the target vehicle is at a first vehicle position near the position of the first roadside camera.

4. The information processing method according to claim 1 or 2, wherein the virtual image includes information indicating at least one of the vehicle position, orientation, turn signal status, and brake lamp status of the target vehicle.

5. The aforementioned controller, The detection results from the on-board sensors of the aforementioned vehicle are obtained, Based on the detection results, it is determined whether or not there are moving objects around the target vehicle. The information processing method according to claim 1 or 2, wherein if the moving object exists, vehicle surrounding information indicating either or both of the position and shape of the moving object is displayed on the virtual image.

6. The information processing method according to claim 1 or 2, wherein the controller displays the target vehicle, which is included in either or both of the captured image and the virtual image, by surrounding it with a frame or by highlighting it, thereby displaying the target vehicle in a manner that allows it to be distinguished from other vehicles.

7. The aforementioned controller, The information processing method according to claim 1 or 2, wherein the travel route of the target vehicle is superimposed and displayed on either the captured image or the virtual image or both.

8. The information processing method according to claim 1 or 2, wherein the controller displays the captured image and the virtual image on the display outside the target vehicle.

9. The information processing method according to claim 1 or 2, wherein the controller displays the captured image and the virtual image on the display provided on the target vehicle.

10. A display that shows images including target vehicles traveling on the road, Multiple roadside cameras installed along the aforementioned road, The system includes the aforementioned display and a server capable of communicating with each of the multiple roadside cameras, The aforementioned server, An image acquisition unit that acquires images captured by each of the multiple roadside cameras, A roadside camera information acquisition unit acquires roadside camera information including the imaging direction and imaging range of each of the multiple roadside cameras, A vehicle position acquisition unit that acquires the vehicle position of the target vehicle based on the vehicle information of the target vehicle, A vehicle position determination unit determines whether the target vehicle is traveling within the image-capable range based on the image-capable range of the roadside camera and the vehicle position relative to the road, If the target vehicle is traveling within the image capture range, the image capture display control unit displays the captured image including the target vehicle on the display, If the target vehicle is traveling outside the image capture range, the system includes a virtual image display control unit that, after the target vehicle has left the first image capture range of the first roadside camera, displays a virtual image of the target vehicle and the road on the display, viewed from a predetermined position between the first roadside camera and a second roadside camera located in front of the first roadside camera in the direction of travel of the target vehicle, in a predetermined display direction relative to the road. The virtual image display control unit, It is determined whether the first imaging direction of the first roadside camera relative to the road when the target vehicle exits the first imaging range of the first roadside camera and the second imaging direction of the second roadside camera relative to the road when the target vehicle enters the second imaging range of the second roadside camera are different. An information processing system that, when the first imaging direction and the second imaging direction are different, changes the predetermined display direction of the virtual image to match the position of the target vehicle, such that the second display direction when the target vehicle is at a second vehicle position near the position of the second roadside camera, which is in front of the first vehicle position, is closer to the second imaging direction than the first display direction when the target vehicle is at a first vehicle position near the position of the first roadside camera.

11. An information processing device that displays an image on a display that includes a target vehicle traveling on a road, An image acquisition unit that acquires images captured by each of the multiple roadside cameras installed along the aforementioned road, A roadside camera information acquisition unit acquires roadside camera information including the imaging direction and imaging range of each of the multiple roadside cameras, A vehicle position acquisition unit that acquires the vehicle position of the target vehicle based on the vehicle information of the target vehicle, A vehicle position determination unit determines whether the target vehicle is traveling within the image-capable range based on the image-capable range of the roadside camera and the vehicle position relative to the road, If the target vehicle is traveling within the image capture range, the image capture display control unit displays the captured image including the target vehicle on the display, If the target vehicle is traveling outside the image capture range, the system includes a virtual image display control unit that, after the target vehicle has left the first image capture range of the first roadside camera, displays a virtual image of the target vehicle and the road on the display, viewed from a predetermined position between the first roadside camera and a second roadside camera located in front of the first roadside camera in the direction of travel of the target vehicle, in a predetermined display direction relative to the road. The virtual image display control unit, It is determined whether the first imaging direction of the first roadside camera relative to the road when the target vehicle exits the first imaging range of the first roadside camera and the second imaging direction of the second roadside camera relative to the road when the target vehicle enters the second imaging range of the second roadside camera are different. An information processing device that, when the first imaging direction and the second imaging direction are different, changes the predetermined display direction of the virtual image to match the position of the target vehicle, such that the second display direction when the target vehicle is at a second vehicle position near the position of the second roadside camera, which is in front of the first vehicle position, is closer to the second imaging direction than the first display direction when the target vehicle is at a first vehicle position near the position of the first roadside camera.

Images