Information processing apparatus and method

Abstract

This helps to prevent misidentification of queues of vehicles waiting to enter a designated facility with queues of vehicles waiting to turn left or right at an intersection. [Solution] The information processing device detects the existence of a convoy of vehicles, exceeding a first threshold, traveling or stopped at a speed below a first speed, on a first lane of a road containing multiple lanes, based on the location information of multiple vehicles. The information processing device determines that the convoy is a first convoy caused by entry into the first facility if the convoy is within a predetermined range from a first facility located on the first lane side of the road, and regardless of the color of the traffic light at the first intersection located ahead of the first facility in the direction of travel of the convoy, there are a number of vehicles exceeding a second threshold turning towards the first facility within the predetermined range. The information processing device causes a first object related to the first convoy to be superimposed on the real space and displayed on a display device for a first vehicle traveling ahead of the first convoy on the first lane.

Description

Technical Field

[0001] This disclosure relates to driving assistance.

Background Art

[0002] An advanced driver assistance system is disclosed that detects a queue of vehicles waiting to enter a facility and, when there is a queue of vehicles waiting to enter on the progress lane, causes a display unit to display guidance information including information about a moving body having characteristic attributes in the queue of vehicles waiting to enter (for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] One aspect of this disclosure aims to provide an information processing apparatus and method for suppressing misidentification between a queue of vehicles waiting to enter a predetermined facility and a queue of vehicles waiting to turn left or right at an intersection.

Means for Solving the Problems

[0005] One aspect of this disclosure is detecting, from position information of a plurality of vehicles, the presence of a queue of vehicles in which a number of vehicles equal to or greater than a first threshold are continuously traveling or stopped at a speed less than a first speed on a first lane leading to a left-turn lane or a right-turn lane of a road including a plurality of lanes; If the aforementioned queue is within a predetermined range from the first facility located on the first lane side of the road, and regardless of the color of the traffic light at the first intersection located ahead of the first facility in the direction of travel of the queue, if there are more than a second threshold number of vehicles turning toward the first facility within the predetermined range, then it is determined that the queue is the first queue resulting from entry into the first facility. To display a first object relating to the first convoy of vehicles superimposed onto real space on a display device mounted on a first vehicle traveling ahead of the first convoy of vehicles in the first lane, A control unit that executes This is an information processing device equipped with [a specific feature / feature].

[0006] Another aspect of this disclosure is, Computers Based on the location information of multiple vehicles, the system detects the existence of a convoy of vehicles exceeding a first threshold that are continuously traveling or stopped at a speed below a first threshold on a first lane that connects to a left-turn or right-turn lane on a road containing multiple lanes, If the aforementioned queue is within a predetermined range from the first facility located on the first lane side of the road, and regardless of the color of the traffic light at the first intersection located ahead of the first facility in the direction of travel of the queue, if there are more than a second threshold number of vehicles turning toward the first facility within the predetermined range, then it is determined that the queue is the first queue resulting from entry into the first facility. To display a first object relating to the first convoy of vehicles superimposed onto real space on a display device mounted on a first vehicle traveling ahead of the first convoy of vehicles in the first lane, This is how to do it. [Effects of the Invention]

[0007] According to one aspect of this disclosure, it is possible to prevent misidentification of a queue of vehicles waiting to enter a designated facility with a queue of vehicles waiting to turn left or right at an intersection. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 shows an example of the processing of the driver assistance system according to the first embodiment. [Figure 2] Figure 2 shows an example of the hardware configuration between the server and the vehicle. [Figure 3] Figure 3 shows an example of the functional configuration of the server and vehicle. [Figure 4] Figure 4 is an example of a flowchart for the server's notification process for convoy information. [Figure 5] Figure 5 shows an example of a flowchart for updating the server's convoy information. [Figure 6] Figure 6 is an example of a flowchart for processing vehicle convoy information. [Modes for carrying out the invention]

[0009] One aspect of the present disclosure is an information processing device comprising a control unit. The control unit detects, from the position information of multiple vehicles, the existence of a convoy of vehicles, numbering more than a first threshold, traveling or stopped at a speed below a first speed, on a first lane connected to a left-turn lane or a right-turn lane of a road containing multiple lanes. The control unit determines that the convoy is a first convoy resulting from entry into a first facility if the convoy is within a predetermined range from a first facility located on the first lane side of the road, and regardless of the color of the traffic light at the first intersection located beyond the first facility in the direction of travel of the convoy, there are more than a second threshold number of vehicles turning toward the first facility within the predetermined range. The control unit causes a first object relating to the first convoy to be superimposed on real space and displayed on a display device mounted on a first vehicle traveling ahead of the first convoy on the first lane.

[0010] An information processing device is, for example, a server that can communicate with a vehicle. An information processing device is also a data communication module, navigation system, etc., installed in a vehicle. This may also include a dashcam, etc. The information processing device is not limited to these, and may also include, for example, a computer mounted on a mobile body other than a vehicle, and mobile devices such as a smartphone, tablet, PC (Personal Computer), and game console. The information processing device is not limited to being mounted on a vehicle, but may also be mounted on a mobile body other than a vehicle. Examples of mobile bodies include automobiles, motorcycles, bicycles, and trains. Furthermore, mobile bodies are not limited to vehicles that travel on the ground, but may also include, for example, mobile bodies that travel on the sea, such as ships, and mobile bodies that travel in the air, such as aircraft and drones.

[0011] The control unit includes processors such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a DSP (Digital Signal Processor). Furthermore, the control unit is not limited to a processor, but may be a circuit such as an FPGA (Field Programmable Gate Array), an integrated circuit (IC), or a CPLD (Complex Programmable Logic Device).

[0012] Display devices installed in vehicles include, for example, organic EL displays attached to the windshield and car navigation system displays. The act of superimposing objects onto real space is known as AR (Augmented Reality) display. It is also called a graphic object. An object is a graphic object that includes, for example, a message, illustration, or mark. An object may also be a video image.

[0013] The first facility is, for example, a commercial facility such as a shopping mall, an amusement park, and a supermarket, a tourist facility such as a temple, or a public facility such as a hospital and a library. In this specification, the term "facility" shall refer to the entire site including a building and a parking lot. The first object is, for example, a graphic object such as an arrow-shaped figure, a message, or an illustration indicating that the first vehicle queue is a queue waiting to enter the first facility. Or, it is a graphic object such as an arrow-shaped figure, a message, or an illustration that prompts the vehicle to change lanes from the first lane and drive alongside the first vehicle queue waiting to enter the first facility that exists ahead in the traveling direction of the first vehicle. ーマーケット等の商業施設、寺院などの観光施設、病院及び図書館などの公共施設などの施設である。なお、本明細書では、「施設」という文言は、建物と駐車場とを含む敷地全体を示すこととする。第1のオブジェクトは、例えば、第1の車列が第1の施設への入場待ちの車列であることを示す、矢印状の図形、メッセージ、イラストなどのグラフィックオブジェクトである。または、第1の車線から車線変更して、第1の車両の進行方向の先に存在する第1の施設への入場待ちの第1の車列の横を走行することを促す、矢印状の図形、メッセージ、イラストなどのグラフィックオブジェクトである。 It is a facility such as a commercial facility such as a supermarket, a tourist facility such as a temple, or a public facility such as a hospital and a library. In this specification, the term "facility" shall refer to the entire site including a building and a parking lot. The first object is, for example, a graphic object such as an arrow-shaped figure, a message, or an illustration indicating that the first vehicle queue is a queue waiting to enter the first facility. Or, it is a graphic object such as an arrow-shaped figure, a message, or an illustration that prompts the vehicle to change lanes from the first lane and drive alongside the first vehicle queue waiting to enter the first facility that exists ahead in the traveling direction of the first vehicle.

[0014] According to one aspect of the present disclosure, when there is a first vehicle queue waiting to enter the first facility ahead in the traveling direction on the first lane on which the first vehicle is traveling, a first object related to the first vehicle queue is displayed superimposed on the real space on the display device of the first vehicle. As a result, the first vehicle queue waiting to enter the first facility can be clearly shown to the driver of the first vehicle, and misidentification between the first vehicle queue waiting to enter the first facility and the vehicle queue waiting to turn left or right at an intersection can be suppressed.

[0015] As another aspect of the present disclosure, it can be specified as a method for a computer to execute the processing executed by the information processing device, a program for causing the computer to execute the method, and a non-temporary computer-readable recording medium on which the program is recorded.

[0016] Hereinafter, embodiments of the present disclosure will be described based on the drawings. The configurations of the following embodiments are examples, and the present disclosure is not limited to the configurations of the embodiments.

[0017] <First Embodiment> Figure 1 shows an example of the processing of the driver assistance system 100 according to the first embodiment. The driver assistance system 100 is a system that notifies vehicles of the queue of vehicles waiting to enter a predetermined facility. The driver assistance system 100 includes a server 1 and a plurality of vehicles 2. The vehicles 2 are vehicles equipped with communication functions. Each vehicle 2 can communicate with the server 1 via a network N1. The network N1 is, for example, a public network such as the Internet.

[0018] Each vehicle 2 transmits driving information to server 1 at predetermined intervals. This driving information includes, for example, the location, speed, direction of travel, and steering information of each vehicle 2. Based on the driving information of each vehicle 2, server 1 detects a queue of vehicles waiting to enter a first facility. The first facility is, for example, a commercial facility such as a shopping mall, supermarket, or restaurant, an amusement park, a museum, an art gallery, or a hospital. A queue of vehicles waiting to enter a first facility is an example of a "first queue of vehicles."

[0019] Server 1 detects a first vehicle 2A traveling in the same lane at a predetermined distance behind a queue of vehicles waiting to enter the first facility, based on the driving information of each vehicle 2. Server 1 transmits to the first vehicle 2A a first object indicating information about the queue of vehicles waiting to enter the first facility, and an instruction to display the first object in augmented reality (AR). The first object is, for example, an arrow-shaped graphic object indicating the direction of travel for the first vehicle 2A, such as object G1 shown in Figure 1, to prompt the first vehicle 2A to change lanes and pass the first queue. However, it is not limited to this, and for example, the first object may be a graphic object that includes a message indicating that the queue is a queue of vehicles waiting to enter the first facility.

[0020] When the first vehicle 2A receives the first object and an AR display instruction from the server 1, it superimposes the first object onto the real space and displays it on the display device. Hereinafter, the superimposition and display of image data onto the real space will be referred to as AR display. In Figure 1, within the speech bubble of vehicle 2A... An example of the display screen of the vehicle 2A's display device is shown, in which an object is displayed using AR. By displaying a first object in AR that indicates information about a queue of vehicles waiting to enter the first facility, the driver of the first vehicle 2A can recognize that the queue ahead is a queue of vehicles waiting to enter the first facility. This prevents situations in which the driver of the first vehicle 2A, even though they do not intend to enter the first facility, mistakenly joins the queue of vehicles waiting to enter the first facility, mistaking it for a queue of vehicles waiting to turn left or right at an intersection.

[0021] In the first embodiment, in addition to the first object, Server 1 also generates a factor object indicating the factors of the convoy, a vehicle pointer indicating the position of the leading vehicle in the convoy waiting to enter the first facility, and a confidence object indicating the confidence level indicating the certainty that it is a convoy waiting to enter the first facility. These are sent to the first vehicle 2A and displayed in AR on the first vehicle 2A. Furthermore, the vehicle pointer can be changed by the driver of the first vehicle 2A using a hand gesture.

[0022] Figure 2 shows an example of the hardware configuration between Server 1 and Vehicle 2. Server 1 comprises a processor 101, memory 102, auxiliary storage device 103, and communication unit 104. Memory 102 and auxiliary storage device 103 are computer-readable recording media. Auxiliary storage device 103 is, for example, an EPROM (Erasable Programmable ROM), a hard disk drive (Hard Disk Drive), or an SSD (Solid State Drive). Programs stored in auxiliary storage device 103 include, for example, O This includes an operating system (OS) and driver assistance control programs. Memory 102 includes semiconductor memory such as ROM (Read Only Memory) and RAM (Random Access Memory).

[0023] The processor 101 is, for example, a CPU, GPU, or DSP (Digital Signal Processor). There is not limited to one processor 101, and there may be multiple processors. The processor 101 is an example of a "control unit". The communication unit 104 is a communication circuit that connects to a wired network such as an optical line or a LAN line. The communication unit 104 may also be a wireless communication circuit that connects to a wireless network of a predetermined wireless communication method.

[0024] Next, vehicle 2 is equipped with an in-vehicle device 210, a display 220, an external camera 230, a speedometer 240, a position information acquisition sensor 250, a direction sensor 260, and an internal camera 270. In Figure 2, the hardware components related to the processing of the first embodiment are extracted and shown as the hardware configuration of vehicle 2, and the hardware configuration of vehicle 2 is not limited to the configuration shown in Figure 2. The in-vehicle device 210 is, for example, a data communication module, a car navigation system, or a drive recorder.

[0025] The in-vehicle device 210 comprises a processor 211, memory 212, auxiliary storage device 213, wireless communication unit 214, and interface 216 as its hardware configuration. The processor 211, memory 212, and auxiliary storage device 213 are the same as those of the processor 101, memory 102, and auxiliary storage device 103. However, the auxiliary storage device 103 stores, for example, a driver assistance control program, an image recognition program, etc.

[0026] The wireless communication unit 214 is a wireless communication circuit that supports cellular communication methods such as 5G, 4G, or 6G. The interface 216 is an interface that connects to an in-vehicle network to which other devices are connected. The in-vehicle network is, for example, an in-vehicle Ethernet network or a CAN (Controller Area Network).

[0027] The display 220, exterior camera 230, speedometer 240, position information acquisition sensor 250, direction sensor 260, and interior camera 270 are connected to the in-vehicle network via interface 216. It connects to the network. The external camera 230 is installed, for example, above the windshield, facing outwards from the vehicle 2. However, it is not limited to this, and multiple external cameras 230 may be provided, each facing a different direction on the outside of the vehicle 2. The speedometer 240 is a sensor that detects the speed of the vehicle 2. The position information acquisition sensor 250 acquires the position information of the vehicle 2 based, for example, on GPS (Global Positioning System) signals and GNSS (Global Navigation Satellite System) signals. The direction sensor 260 detects, for example, acceleration This is a sensor that detects the direction of travel of vehicle 2, such as a degree sensor. The in-vehicle camera 270 is installed, for example, above the windshield, facing inward towards vehicle 2, so that the driver is within the imaging range.

[0028] The display 220 is, for example, a display used in a car navigation system, or an organic EL display installed on the windshield of vehicle 2, with the entire windshield serving as the display screen. However, the display 220 is not limited to these. The hardware configuration of server 1 and vehicle 2 is not limited to that shown in Figure 2.

[0029] Figure 3 shows an example of the functional configuration of Server 1 and Vehicle 2. Server 1 includes a control unit 11, a map information database (DB) 12, and a road information DB 13 as functional components. The control unit 11 is a functional component whose processing is executed, for example, by the Server 1's processor 101 executing a predetermined program.

[0030] The control unit 11 acquires driving information from each vehicle 2 at predetermined intervals. The driving information includes, for example, identification information, timestamp, location information, speed, direction of travel, turn signal information, steering information, and images captured by the external camera 230 of the vehicle 2. The turn signal information includes whether the left turn signal is ON or OFF and whether the right turn signal is ON or OFF.

[0031] The control unit 11 detects, for example, a line of vehicles formed on a first lane leading to a left-turn lane or right-turn lane on a road with multiple lanes, based on the driving information of each vehicle 2, the map information DB 12, and the road information DB 13. A road with multiple lanes may be, for example, a road with two or more lanes on one side. A line of vehicles is, for example, a line of vehicles with a number equal to or greater than a first threshold that are traveling or stopped at a speed less than a first speed, with a distance between vehicles less than a predetermined distance. The first speed is, for example, a speed considered to be slow. The distance between vehicles considered to be a line of vehicles is, for example, a distance in the range of 1m to 5m. A line of vehicles may be detected, for example, based on position information included in the driving information of multiple vehicles 2, or by image recognition of images captured by an external camera 230 included in the driving information of multiple vehicles 2.

[0032] The control unit 11 determines whether the detected queue is a queue of vehicles waiting to enter the first facility. In the first embodiment, the control unit 11 determines a queue of vehicles waiting to enter the first facility if (1) it is located within a predetermined range from the first facility on the lane side where the queue of vehicles is located on the road, and (2) regardless of the color of the traffic light at the first intersection located beyond the first facility in the direction of travel of the queue, there are at least two second threshold numbers of vehicles turning toward the first facility within a predetermined range from the first facility. By using the determination conditions described in (1) and (2) above, it is possible to more accurately determine whether a queue of vehicles is waiting to enter the first facility.

[0033] The predetermined range from the first facility is, for example, several tens of meters to 1 kilometer from the first facility. The second threshold for the number of vehicles in the convoy is a smaller value than the first threshold. Vehicles turning toward the first facility may be identified, for example, from turn signal information included in the driving information of vehicle 2, or by image recognition processing of captured images of vehicle 2 or surrounding vehicles 2 included in the convoy. The first intersection ahead of the first facility in the direction of travel of the convoy, and whether or not there are traffic lights at that intersection, are identified, for example, based on map information DB 12. The color of the traffic light installed at a given point may be determined, for example, from images of vehicles 2 present around the intersection, or it may be obtained from a designated organization such as a traffic control center.

[0034] The control unit 11 identifies, for example, a first vehicle 2A that is traveling in the same lane as a queue of vehicles waiting to enter the first facility, and is less than a predetermined distance behind the last vehicle in the queue, that is, a first vehicle 2A that is approaching the queue of vehicles waiting to enter the first facility, based on the vehicle 2's driving information and the road information DB 13. Multiple first vehicles 2A may be detected.

[0035] The control unit 11 acquires a first object relating to the queue of vehicles waiting to enter the first facility, a factor object indicating the factors of the queue, a vehicle pointer indicating the position of the leading vehicle in the queue of vehicles waiting to enter the first facility, and a confidence object indicating the degree of confidence that it is a queue of vehicles waiting to enter the first facility, and transmits this queue information to the first vehicle 2A along with an AR display instruction. The AR display instruction also includes, for example, information on the placement position of each object in the virtual space corresponding to the real space. The vehicle pointer is an example of a "second object". The confidence object is an example of a "third object".

[0036] The first object may be, for example, a graphic object containing a message indicating that the queue of vehicles is waiting to enter a first facility. If, as one of the driving information for the first vehicle 2A, a destination set in the car navigation system is obtained, and the destination is not the first facility corresponding to the queue of vehicles, then the first object may be, for example, a graphic object in the shape of an arrow prompting the vehicle to change lanes and avoid the queue of vehicles waiting to enter the first facility.

[0037] A factor object indicating the cause of a convoy is an object that indicates the existence of a first facility, which is the cause of the convoy and the destination of the convoy. For example, a factor object may contain a message such as "You are heading to XX Shopping Mall." Information about the first facility may be obtained, for example, from the map information DB 12.

[0038] In the first embodiment, the leading vehicle in a queue of vehicles waiting to enter the first facility is the leading vehicle in the queue whose head is not facing the first facility. The direction of the head of vehicle 2 can be determined, for example, based on steering information included in the driving information. However, the definition of the leading vehicle in a queue of vehicles waiting to enter the first facility is not limited to this.

[0039] The confidence level indicating the certainty that a queue of vehicles is waiting to enter the first facility is calculated using a predetermined algorithm based on, for example, the history of past queues at the location where the queue is occurring, the type of the first facility, and the operating hours of the first facility and the current time of day. This confidence level may be obtained, for example, using a machine learning model.

[0040] Furthermore, when the control unit 11 receives gesture information from the first vehicle 2A, it generates updated vehicle convoy information based on the gesture information and transmits it to the first vehicle. The gesture information includes, for example, information indicating that an occupant of the first vehicle 2A has made either a gesture of pushing their hand forward or a gesture of pulling their hand to their side, and the number of times the gesture has occurred. If the gesture of pushing the hand forward has occurred, the control unit 11 generates updated information indicating that the position of the vehicle pointer in the virtual space should be changed to a position corresponding to a number of vehicles prior to the vehicle currently pointed to by the vehicle pointer in the convoy, corresponding to the number of times the gesture has occurred. However, if the vehicle pointer has reached the lead vehicle, it will not move any further.

[0041] If a gesture of pulling the hand towards the body occurs, the control unit 11 will temporarily display the vehicle pointer. Update information is generated indicating that the position in the imaginary space should be changed from the vehicle currently pointed to by the vehicle pointer in the convoy to a position corresponding to a number of vehicles behind that vehicle corresponding to the number of times the gesture has occurred. However, the vehicle pointer will not move any further if it has reached the last vehicle in the convoy. The last vehicle in the convoy is, for example, a vehicle in the convoy whose distance from the vehicle behind is greater than or equal to a predetermined distance, or a vehicle located directly in front of the first vehicle. By moving the vehicle pointer in response to the hand gesture of the occupant of the first vehicle 2A in this way, the occupant of the first vehicle 2A can be made to intuitively recognize the length of the convoy waiting to enter the first facility.

[0042] Map information DB 12 holds map information. Map information includes, for example, information about facilities, intersections, traffic lights, etc. Road information DB 13 holds road information. Road information includes, for example, road type, number of lanes on the road, and number of lanes on each side.

[0043] Next, the vehicle 2 comprises a control unit 21 and a gesture detection unit 22 as its functional configuration. The control unit 21 is a functional component whose processing is performed, for example, by the processor 211 of the in-vehicle device 210 executing a predetermined program.

[0044] The control unit 21 acquires detection values ​​from, for example, the external camera 230, speedometer 240, position information acquisition sensor 250, and direction sensor 260 at predetermined intervals, generates driving information, and transmits it to the server 1. When the control unit 21 receives the convoy information and AR display instruction from the server 1, it places each object included in the convoy information at the specified position in the virtual space, superimposes the virtual space on the real space, and displays it on the display 220 using AR. The AR display of each object included in the convoy information will hereinafter simply be referred to as "display of convoy information" or "AR display of convoy information."

[0045] While the control unit 21 is displaying the convoy information on the display 220, if it receives notification from the gesture detection unit 22 that a gesture has been detected in which an occupant pushes their hand forward or pulls their hand to the side of their body, it generates gesture information and sends it to the server 1. If it receives updated convoy information from the server 1, it updates the display of the vehicle pointer according to the updated information. The control unit 21 terminates the display of the convoy information when the termination conditions are met. The termination conditions include, for example, the elapsed time, the first vehicle 2A leaving the first facility outside a predetermined range by moving to avoid the convoy, or the first vehicle 2A joining the convoy and entering the grounds of the first facility.

[0046] The gesture detection unit 22 performs predetermined image recognition processing on the image captured by the in-vehicle camera 270 to detect gestures in which the occupant pushes their hand forward or pulls their hand towards their body. Gesture detection by the gesture detection unit 22 is started and stopped, for example, by instructions from the control unit 21. The control unit 21 may instruct the gesture detection unit 22 to start gesture detection when the display of the convoy information begins, and may instruct the gesture detection unit 22 to stop gesture detection when the display of the convoy information ends. Note that the functional configuration of the server 1 and vehicle 2 is not limited to that shown in Figure 3.

[0047] Figure 4 is an example of a flowchart for the notification process of vehicle convoy information on Server 1. The process shown in Figure 4 is repeatedly executed at predetermined intervals while Server 1 is running. The main execution entity for the process shown in Figure 4 is the processor 101 of Server 1, but for convenience, the functional components will be described as the main components.

[0048] In OP101, the control unit 11 determines whether a queue of vehicles exists on a road with multiple lanes, based on driving information from each vehicle 2, map information DB 12, and road information DB 13. The driving information from the vehicles 2 used for the determination is the most recent. If a queue of vehicles is present on the road (OP101:YES), the process proceeds to OP102. If no queue of vehicles is present on a road with multiple lanes (OP101:NO), the process shown in Figure 4 ends.

[0049] In OP102, the control unit 11 determines, based on the map information DB 12 and the road information DB 13, whether the queue detected in OP101 is a queue of vehicles waiting to enter the first facility. The determination method for OP102 is as described above. If the queue detected in OP101 is a queue of vehicles waiting to enter the first facility (OP102: YES), the process proceeds to OP103. If the queue detected in OP101 is not a queue of vehicles waiting to enter the first facility (OP102: NO), the process shown in Figure 4 ends.

[0050] In OP103, the control unit 11 detects a first vehicle approaching from behind on the same lane as the queue of vehicles waiting to enter the first facility, based on the driving information of each vehicle 2 and the road information DB 13. In OP104, the control unit 11 generates queue information. Specifically, the control unit 11 obtains a first object relating to the queue of vehicles waiting to enter the first facility, a factor object indicating the factors of the queue, a vehicle pointer indicating the position of the leading vehicle in the queue of vehicles waiting to enter the first facility, and a confidence object indicating the confidence level indicating the certainty that it is a queue of vehicles waiting to enter the first facility.

[0051] In OP105, the control unit 11 transmits convoy information and AR display instructions to the first vehicle 2A. If multiple first vehicles 2A are detected in OP103, convoy information and AR display instructions are transmitted to each of the first vehicles 2A.

[0052] In OP106, the control unit 11 determines whether or not a queue of vehicles waiting to enter the first facility is being maintained. For example, if the number of vehicles included in the queue of vehicles waiting to enter the first facility falls below a first threshold or a third threshold smaller than the first threshold, the control unit 11 determines that a queue of vehicles waiting to enter the first facility is not being maintained. If a queue of vehicles waiting to enter the first facility is being maintained (OP106: YES), the process proceeds to OP103, and a new first vehicle is detected. If a queue of vehicles waiting to enter the first facility is not being maintained (OP106: NO), the process shown in Figure 4 ends.

[0053] Figure 5 is an example of a flowchart for updating the convoy information of Server 1. The convoy information update process is the process of changing the vehicle pointed to by the vehicle pointer in the convoy waiting to enter the first facility in response to a hand gesture from the occupant of the first vehicle 2A. The process shown in Figure 5 is started, for example, when convoy information is sent to the first vehicle 2A (OP105) in the convoy information notification process shown in Figure 4.

[0054] In OP201, the control unit 11 determines whether or not it has received gesture information from the first vehicle 2A. If gesture information has been received from the first vehicle 2A (OP201:YES), the process proceeds to OP202. If gesture information has not been received from the first vehicle 2A (OP201:NO), the process proceeds to OP204.

[0055] In OP202, the control unit 11 generates updated convoy information. Specifically, the updated convoy information includes the new position of the vehicle pointer in the virtual space due to a change in the vehicle in the convoy that the vehicle pointer points to, corresponding to the hand gesture of the occupant of the first vehicle 2A included in the gesture information. In OP203, the control unit 11 transmits the updated convoy information to the first vehicle 2A.

[0056] In OP204, the control unit 11 determines whether or not it has received a termination notification from the first vehicle 2A. The termination notification is a notification indicating that the display of the convoy information has ended in the first vehicle 2A. Yes. If a termination notification is received from the first vehicle 2A (OP204:YES), the process shown in Figure 5 ends. If no termination notification is received from the first vehicle 2A (OP204:NO), the process proceeds to OP201.

[0057] Figure 6 is an example of a flowchart for the vehicle 2's process of receiving convoy information. The process shown in Figure 6 is repeatedly executed while vehicle 2 is in operation. The main entity executing the process shown in Figure 4 is the processor 211 of the in-vehicle device 210, but for convenience, the functional components will be explained in detail.

[0058] In OP301, the control unit 21 determines whether or not it has received convoy information from server 1. If convoy information is received from server 1 (OP301: YES), the process proceeds to OP302. In OP302, the control unit 21 displays the convoy information on display 220 using AR. If convoy information is not received (OP301: NO), the process shown in Figure 6 ends. If convoy information is received, vehicle 2 is the first vehicle 2A. Hereafter, the execution entity in Figure 6 will be referred to as the first vehicle 2A.

[0059] In OP303, the control unit 21 determines whether it has received notification from the gesture detection unit 22 that a gesture of pushing or pulling the hand of the occupant of the first vehicle 2A has been detected. If it has received notification of gesture detection from the gesture detection unit 22 (OP303:YES), the process proceeds to OP304. If it has not received notification of gesture detection from the gesture detection unit 22 (OP303:NO), the process proceeds to OP307.

[0060] In OP304, the control unit 21 transmits gesture information to the server 1. In OP305, the control unit 21 receives updated convoy information from the server 1. In OP306, the control unit 21 updates the AR display of the convoy information according to the updated convoy information.

[0061] In OP307, the control unit 21 determines whether the display of the convoy information has ended, depending on whether the conditions for ending the display of the convoy information have been met. If the display of the convoy information has ended (OP307: YES), the process proceeds to OP308. If the display of the convoy information has not ended (OP307: NO), the process proceeds to OP303.

[0062] In OP308, the control unit 21 sends a termination notification to the server 1. After that, the process shown in Figure 6 is completed. Note that the processes shown in Figures 4 to 6 are examples and can be modified as appropriate depending on the embodiment.

[0063] <Effects of the First Embodiment> In the first embodiment, a first object relating to a queue of vehicles waiting to enter a first facility is displayed in augmented reality (AR) on the display device of a first vehicle 2A traveling behind the queue in the same lane. This prevents the driver of the first vehicle 2A from mistaking the queue of vehicles waiting to enter the first facility for a queue of vehicles waiting to turn left or right at an intersection.

[0064] Furthermore, in the first embodiment, in addition to the first object, a factor object indicating the factors of the queue, a vehicle pointer indicating the position of the lead vehicle, and a confidence level indicating that the queue is a queue of vehicles waiting to enter the first facility are also displayed. As a result, the driver of the first vehicle 2A can obtain more information about the queue of vehicles waiting to enter the first facility, and is less likely to mistake it for a queue of vehicles waiting to turn left or right at an intersection.

[0065] <Other Embodiments> The embodiments described above are merely examples, and this disclosure may be modified as appropriate without departing from its essence.

[0066] In the first embodiment, server 1 generates and updates convoy information. Alternatively, an in-vehicle device 210 may generate and update convoy information as described in the first embodiment. However, since the in-vehicle device 210 cannot acquire driving information of other vehicles 2, convoy detection and the like are performed based on images captured by the external camera 230 of the vehicle 2. In this case, the in-vehicle device 210 is an example of an "information processing device".

[0067] In the first embodiment, a vehicle pointer on the display screen moves between vehicles in the convoy in response to a hand gesture by the occupant of the first vehicle 2A. The convoy information may further include a vehicle information object that presents information about the vehicle 2 pointed to by the vehicle pointer. In this case, the position of the vehicle pointer in the virtual space is updated in response to the hand gesture by the occupant of the first vehicle 2A, and the vehicle information object is also updated to present information about the vehicle 2 newly pointed to by the vehicle pointer. In this case, the vehicle information object is also an example of a "second object".

[0068] The processes and methods described in this disclosure can be freely combined and implemented, provided that no technical inconsistencies arise.

[0069] Furthermore, a process described as being performed by a single device may be divided and executed by multiple devices. Conversely, a process described as being performed by different devices may be executed by a single device. In a computer system, the hardware configuration (server configuration) by which each function is implemented can be flexibly changed.

[0070] The present disclosure can also be realized by supplying a computer program implementing the functions described in the embodiments above to a computer, and having one or more processors in the computer read and execute the program. Such a computer program may be provided to the computer by a non-temporary computer-readable storage medium that can be connected to the computer's system bus, or it may be provided to the computer via a network. Non-temporary computer-readable storage mediums include, for example, any type of disk such as magnetic disks (floppy disks, hard disk drives (HDDs), etc.), optical disks (CD-ROMs, DVDs, Blu-ray discs, etc.), read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic cards, flash memory, optical cards, and any type of medium suitable for storing electronic instructions. [Explanation of symbols]

[0071] 1. Server 2. Vehicles 11. Control Unit 12. Map Information Database 13...Road information DB 21. Control Unit 22. Gesture detection unit 101, 211... Processors 102,212...memory 103, 213...Auxiliary storage device 104 Communications Department 210...In-vehicle equipment

Claims

1. Based on the location information of multiple vehicles, the system detects the existence of a convoy of vehicles exceeding a first threshold that are continuously traveling or stopped at a speed below a first threshold on a first lane that connects to a left-turn or right-turn lane on a road containing multiple lanes, If the convoy is located within a predetermined range from the first facility located on the first lane side of the road, and regardless of the color of the traffic light at the first intersection located ahead of the first facility in the direction of travel of the convoy, if there are more than a second threshold number of vehicles turning toward the first facility within the predetermined range, then it is determined that the convoy is the first convoy resulting from entry into the first facility. To display a first object relating to the first convoy of vehicles superimposed onto real space on a display device mounted on a first vehicle traveling ahead of the first convoy of vehicles in the first lane, A control unit that executes An information processing device equipped with the following features.

2. The control unit, Based on the location information of the plurality of vehicles, a second vehicle located at a predetermined position within the first convoy is identified. The first vehicle is to be displayed on the display device by superimposing a second object, which presents information about the second vehicle in the first convoy, onto a predetermined position in real space corresponding to the second vehicle. The information processing apparatus according to claim 1.

3. The control unit, Information indicating the gestures of the occupants of the first vehicle is obtained, When the information indicating the gesture indicates that the occupant has performed a gesture of pushing out their hand, the vehicle located a predetermined number of vehicles ahead of the second vehicle in the first convoy is designated as a new second vehicle, and the second object is superimposed on the first vehicle at a predetermined position corresponding to the new second vehicle in real space and displayed on the display device. When the information indicating the gesture indicates that the occupant has performed a gesture of pulling their hand, the vehicle located a predetermined number of vehicles behind the second vehicle in the first convoy is designated as a new second vehicle, and the second object is superimposed on the first vehicle at a predetermined position corresponding to the new second vehicle in real space and displayed on the display device. The information processing apparatus according to claim 2.

4. The control unit, Obtain a third object that represents a numerical value indicating the certainty that the aforementioned convoy is the first convoy, The first vehicle is superimposed with the third object in real space and displayed on the display device. The information processing apparatus according to claim 1.

5. Computers Based on the location information of multiple vehicles, the system detects the existence of a convoy of vehicles exceeding a first threshold that are continuously traveling or stopped at a speed below a first threshold on a first lane that connects to a left-turn or right-turn lane on a road containing multiple lanes, The aforementioned convoy is located within a predetermined range from the first facility located on the first lane side of the road, and regardless of the color of the traffic light at the first intersection located ahead of the first facility in the direction of travel of the convoy, it turns towards the first facility within the predetermined range. If the number of vehicles exceeds a second threshold, it is determined that the convoy is the first convoy resulting from entry into the first facility. To display a first object relating to the first convoy of vehicles superimposed onto real space on a display device mounted on a first vehicle traveling ahead of the first convoy of vehicles in the first lane, How to do it.

Images