Output device, output method, and computer program

Abstract

To facilitate smooth communication with other mobile objects and realize smooth automatic driving.SOLUTION: An output device 10 includes a first acquisition part 110, a second acquisition part 120, and a first output part 150. The first acquisition 110 acquires first behavior information indicating the first behavior of a passenger of a first moving body. The second acquisition part 120 acquires second behavior information indicating the second behavior of a second moving body around the first moving body within a predetermined period from the occurrence of the first behavior. The first output part 150 outputs control information for controlling the travel of the first moving body based on the first behavior information and the second behavior information.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an output device, an output method, and a computer program.

Background Art

[0002] In autonomous driving, the next operation of the host vehicle is determined while estimating the next movements of surrounding vehicles, pedestrians, etc. using various sensing results.

[0003] Patent Document 1 describes that automatic steering control by a system is executed with reference to the driving operation of a driver after detection of a pedestrian or the like, thereby increasing the degree of driver involvement in the driving of the vehicle.

[0004] Further, Patent Document 2 describes that when a specific message is received from another moving body, a response message including intention information indicating the intention to execute a specific behavior is transmitted to that moving body.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, with the technology of Patent Document 1, communication of intentions with pedestrians or the like could not be achieved.

[0007] Further, with the technology of Patent Document 2, since information transmission and reception intervene, the smoothness of communication of intentions with other moving bodies may be impaired. Also, when no message or the like is transmitted from other moving bodies, communication of intentions could not be achieved.

[0008] As one of the problems to be solved by the present invention, smooth communication with other moving bodies and realization of smooth automatic driving can be cited as an example.

Means for Solving the Problem

[0009] The first invention is a first acquisition unit that acquires first behavior information indicating a first behavior of a passenger of a first moving body, a second acquisition unit that acquires second behavior information indicating a second behavior of a second moving body around the first moving body within a predetermined period from the occurrence of the first behavior, and a first output unit that outputs control information for controlling the travel of the first moving body based on the first behavior information and the second behavior information. It is an output device provided with.

[0010] The second invention is a first acquisition step of acquiring first behavior information indicating a first behavior of a passenger of a first moving body, a second acquisition step of acquiring second behavior information indicating a second behavior of a second moving body around the first moving body within a predetermined period from the occurrence of the first behavior, and a first output step of outputting control information for controlling the travel of the first moving body based on the first behavior information and the second behavior information. It is an output method including.

[0011] The third invention is a computer program for realizing an output device, the computer is a first acquisition means for acquiring first behavior information indicating a first behavior of a passenger of a first moving body, a second acquisition means for acquiring second behavior information indicating a second behavior of a second moving body around the first moving body within a predetermined period from the occurrence of the first behavior, and a first output means for outputting control information for controlling the travel of the first moving body based on the first behavior information and the second behavior information. It is a computer program that functions as

Brief Description of Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Modes for Carrying Out the Invention

[0013] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same reference numerals are assigned to the same components, and the description will be omitted as appropriate.

[0014] In the following description, the first acquisition unit 110, the second acquisition unit 120, the first detection unit 112, the second detection unit 122, the third acquisition unit 130, the determination unit 140, the first output unit 150, the second output unit 152, the first input unit 160, the second input unit 162, and the storage unit 170 of the output device 10 do not represent a configuration in hardware units, but rather blocks in functional units. The first acquisition unit 110, the second acquisition unit 120, the first detection unit 112, the second detection unit 122, the third acquisition unit 130, the determination unit 140, the second output unit 152, the first output unit 150, the second input unit 162, the first input unit 160, and the storage unit 170 of the output device 10 are realized by any combination of hardware and software centered around the CPU, memory, program loaded into the memory, storage media such as a hard disk storing the program, and network connection interface of any computer. And there are various variations in the realization method and device.

[0015] FIG. 1 is a block diagram illustrating the functional configuration of the output device 10 according to the embodiment. The output device 10 according to the present embodiment includes a first acquisition unit 110, a second acquisition unit 120, and a first output unit 150. The first acquisition unit 110 acquires first behavior information indicating the first behavior of the passengers of the first moving body. The second acquisition unit 120 acquires second behavior information indicating the second behavior of the second moving body around the first moving body within a predetermined period T1 from the occurrence of the first behavior. The first output unit 150 outputs control information for controlling the running of the first moving body based on the first behavior information and the second behavior information. Details will be described below.

[0016] In a moving body that travels using partial or fully automatic driving, the surrounding situation is judged by the control device of the automatic driving, and the operation of the moving body is controlled based on the judgment result. However, when it is difficult to predict the next behavior of other moving bodies existing around, it may not be possible to drive smoothly with automatic driving.

[0017] On the one hand, a person boarding an autonomously operated moving body can communicate with, for example, the passengers of surrounding vehicles or pedestrians. According to this embodiment, the surrounding situation and the next behavior of other moving bodies are estimated from the behavior of the passengers in the host vehicle and the behavior of the passengers of surrounding vehicles and pedestrians, and smooth driving by autonomous driving is maintained.

[0018] In this embodiment, the first moving body is a vehicle such as an automobile, a motorized bicycle, or a light vehicle. The output device 10 is mounted on the first moving body. The first moving body is at least partially autonomously driven by control information output from the first output unit 150. There is at least one passenger in the first moving body.

[0019] The first behavior is the behavior of the passenger of the first moving body and is a gesture or vocalization directed at the second moving body. Specifically, as the first behavior, there are behaviors such as holding out the palm upward, turning the palm toward the second moving body, nodding, communicating intentions by voice, operating the headlights, hazard lights, or brake lights of the first moving body to flash one or more times, operating the horn of the first moving body to sound one or more times, operating to decelerate the first moving body, and the like. The first behavior information is an ID or the like indicating what kind of behavior the first behavior is. The first behavior may or may not be a behavior directed at the second moving body. It should be noted that the first behavior is preferably a behavior that can be recognized directly or indirectly from the second moving body.

[0020] The second moving body is a moving body located around the first moving body. The periphery of the first moving body is, for example, a range within which the behavior can be recognized and detected from the first moving body. The periphery of the first moving body is not particularly limited, but is, for example, a range within 50 m from the first moving body.

[0021] The second moving body is, for example, a pedestrian or a bicycle. In that case, examples of the second behavior include the behavior of a pedestrian or a cyclist making a greeting gesture, stopping, raising one hand, etc. The second behavior information is, for example, an ID indicating which behavior the second behavior is. The second behavior may or may not be a behavior directed at the first moving body. Note that the second behavior is preferably a behavior that can be recognized directly or indirectly from the first moving body.

[0022] The methods for generating and acquiring the first behavior information and the second behavior information will be described in detail in the embodiments described later.

[0023] For example, when a pedestrian on one side of a crosswalk is the second moving body, the passenger of the first moving body raises the palm upward at face height (the first behavior) and urges the pedestrian to cross the crosswalk. Then, the pedestrian makes a greeting gesture (the second behavior) and shows the intention to accept the offer of the passenger of the first moving body. The first output unit 150 outputs control information for stopping the first moving body based on the first behavior information and the second behavior information indicating these behaviors respectively.

[0024] Also, for example, when a pedestrian on one side of a crosswalk is the second moving body, the passenger of the first moving body raises the palm upward at face height (the first behavior) and urges the pedestrian to cross the crosswalk. Then, the pedestrian shakes the head (the second behavior) and shows the intention to reject the offer of the passenger of the first moving body. The first output unit 150 outputs control information for causing the first moving body to travel based on the first behavior information and the second behavior information indicating these behaviors respectively. Note that the second behavior at this time may be a behavior in which the pedestrian vertically holds the hand and shakes it horizontally, a behavior in which the pedestrian moves the palm upward horizontally in the traveling direction of the vehicle, or a behavior in which the pedestrian looks at the passenger side but does not respond.

[0025] The second moving body may be an oncoming vehicle of the first moving body. In that case, examples of the second behavior include the behavior of the second moving body flashing its headlights, hazard lights, or brake lights one or more times, the behavior of sounding the horn one or more times, the behavior of decelerating, and the behavior of approaching the edge of the road. Also, the second behavior may be the behavior of the passengers in the oncoming vehicle that is the second moving body. Examples of the behavior of the passengers in the oncoming vehicle include the behavior of making a gesture of acknowledgement and the behavior of raising one hand. The second behavior information is an ID or the like indicating what behavior the second behavior is.

[0026] For example, when the oncoming vehicle is the second moving body on a narrow road, the passenger of the first moving body turns their palm towards the second moving body so that the second moving body waits for them to pass (the first behavior). Then, the second moving body moves to the edge of the road and gives way (the second behavior). Based on the first behavior information and the second behavior information indicating these behaviors respectively, control information is output so that the first moving body travels past the second moving body.

[0027] Examples of the scene where the first output unit 150 outputs a control signal based on the first behavior information and the second behavior information include a scene where the first moving body or the second moving body is about to turn right or merge onto the road.

[0028] The second behavior is performed within a predetermined period T1 from the occurrence of the first behavior. The predetermined period T1 is predetermined based on the time assumed to be required for the second moving body to react upon receiving the first behavior. The predetermined period T1 is, for example, within 5 seconds or within 3 seconds.

[0029] FIG. 2 is a flowchart illustrating an output method according to an embodiment. This method includes a first acquisition step (S10), a second acquisition step (S20), and a first output step (S50). In the first acquisition step (S10), first behavior information indicating the first behavior of the passenger of the first moving body is acquired. In the second acquisition step (S20), second behavior information indicating the second behavior of the second moving body around the first moving body within a predetermined period T1 from the occurrence of the first behavior is acquired. In the first output step (S50), control information for controlling the travel of the first moving body is output based on the first behavior information and the second behavior information.

[0030] The output method according to this embodiment is realized by, for example, the output device 10 as described above.

[0031] As described above, according to this embodiment, the first output unit 150 outputs control information for controlling the travel of the first moving body based on the first behavior information and the second behavior information. Therefore, smooth communication of intentions with other moving bodies is achieved, and smooth automatic driving is realized.

[0032] (Example 1) FIG. 3 is a block diagram illustrating the functional configuration of the output device 10 according to Example 1. The output device 10 according to this example has the configuration of the output device 10 according to the embodiment. The output device 10 according to this example further includes a first input unit 160, a storage unit 170, a first detection unit 112, a second detection unit 122, and a determination unit 140. The functional configuration of the output device 10 will be described in detail later.

[0033] FIG. 4 is a diagram illustrating the hardware configuration of the output device 10. In this figure, the output device 10 is implemented using an integrated circuit 40. The integrated circuit 40 is, for example, a SoC (System On Chip).

[0034] Integrated circuit 40 has a bus 402, a processor 404, a memory 406, a storage device 408, an input / output interface 410, and a network interface 412. The bus 402 is a data transmission path for the processor 404, the memory 406, the storage device 408, the input / output interface 410, and the network interface 412 to transmit and receive data to and from each other. However, the method of connecting the processor 404 and the like to each other is not limited to bus connection. The processor 404 is an arithmetic processing unit realized using a microprocessor or the like. The memory 406 is a memory realized using a RAM (Random Access Memory) or the like. The storage device 408 is a storage device realized using a ROM (Read Only Memory), a flash memory, or the like.

[0035] The input / output interface 410 is an interface for connecting the integrated circuit 40 to peripheral devices. In this figure, a first camera 420, a second camera 421, a monitor 422, an input panel 423, a microphone 424, and a speaker 425 are connected to the input / output interface 410.

[0036] The network interface 412 is an interface for connecting the integrated circuit 40 to a communication network. This communication network is, for example, a CAN (Controller Area Network) communication network. Note that the method by which the network interface 412 connects to the communication network may be a wireless connection or a wired connection.

[0037] The storage device 408 stores program modules for realizing the functions of a first input unit 160, a first detection unit 112, a second detection unit 122, a first acquisition unit 110, a second acquisition unit 120, a determination unit 140, and a first output unit 150, respectively. The processor 404 reads out and executes this program module in the memory 406 to realize the functions of the first input unit 160, the first detection unit 112, the second detection unit 122, the first acquisition unit 110, the second acquisition unit 120, the determination unit 140, and the first output unit 150.

[0038] The hardware configuration of the integrated circuit 40 is not limited to the configuration shown in this figure. For example, the program module may be stored in the memory 406. In this case, the integrated circuit 40 may not include the storage device 408.

[0039] Referring to FIG. 3, the functional configuration of the output device 10 will be described in detail below. In the output device 10 according to this embodiment, the first acquisition unit 110 can receive the input of the passenger to the first input unit 160. And an example of the first behavior is the input to the first input unit 160.

[0040] In the example of FIG. 4, the input panel 423 and the microphone 424 function as the first input unit 160. Also, the first input unit 160 may be a mechanical handle, switch, pedal, etc. And the first behavior includes, for example, an input operation on the touch panel or an input by voice. The input of the passenger includes, for example, an operation of turning on the headlights, hazard lights, or brake lights of the first moving body one or more times, an operation of sounding the horn of the first moving body one or more times, and an operation of decelerating the first moving body. At least one of the passengers of the first moving body is boarding in a state where they can access the first input unit 160 as a driver. For example, the passenger as a driver is boarding at an operation position provided with a touch panel or a microphone, etc., which is the first input unit 160. Also, the passenger as a driver is holding an operation terminal provided with a touch panel or a microphone, etc., which is the first input unit 160. When the passenger performs any one of a plurality of input operations defined as candidates for the first behavior on the first input unit 160, the first acquisition unit 110 acquires information indicating the operation content as the first behavior information. Also, the first input unit 160 may be able to receive an input from the passenger indicating that the first behavior has been performed. For example, a screen display or voice output such as "Did you prompt a pedestrian to cross?" may be performed, and the passenger may be prompted for a touch input or a voice input.

[0041] Since the first behavior is an input to the first input unit 160, the first acquisition unit 110 can directly acquire the first behavior information, and the first behavior can be recognized with high accuracy.

[0042] As described above, the output device 10 according to the present embodiment includes a first detection unit 112. The first detection unit 112 detects the behavior of the first moving body. The first acquisition unit 110 acquires the first behavior information from the first detection unit 112. The first detection unit 112 acquires an image including at least a part of the passengers of the first moving body, detects the first behavior using the image, and generates the first behavior information. The first detection unit 112 acquires the image captured by the first camera 420. The first camera 420 is provided so as to capture at least a part of the passengers as the driver of the first moving body. Further, instead of the first camera 420, a ToF (Time of Flight) camera or a lidar (LIDAR: Laser Imaging Detection and Ranging, Laser Illuminated Detection and Ranging or LiDAR: Light Detection and Ranging) may be used to acquire an image.

[0043] The first detection unit 112 detects the first behavior of the passengers by, for example, extracting and analyzing feature points in the acquired image. Information indicating the pattern of feature points for the first behavior is stored in the storage unit 170 in advance for each behavior content, and the first detection unit 112 can read it out and use it for detection. When the first detection unit 112 detects the first behavior, it generates information indicating the content of the behavior as the first behavior information. When the first behavior information is generated by the first detection unit 112, the first acquisition unit 110 acquires the first behavior information. Note that the first detection unit 112 may detect the first behavior based on a plurality of images such as a video.

[0044] Specifically, characteristic quantity information indicating the characteristic quantities of a human body is stored in advance in the storage unit 170. The first detection unit 112 specifies the position of the human body part in the image by using the characteristic quantity information from the image. The posture of the occupant is estimated from the specified position and the human body model information, and the behavior is estimated based on the temporal change of the posture.

[0045] Furthermore, the first detection unit 112 may utilize deep learning. In this case, video data of behaviors and information indicating what kind of behavior the video data represents are stored in advance in the storage unit 170. The first detection unit 112 estimates the behavior of the occupant based on the video data of the occupant and the video data stored in advance.

[0046] Note that the output device 10 may not include either the first input unit 160 or the first detection unit 112. The first acquisition unit 110 only needs to be able to acquire the first behavior information from at least one of the first input unit 160 and the first detection unit 112.

[0047] As described above, the output device 10 according to the present embodiment includes the second detection unit 122. The second detection unit 122 detects the behavior of the second moving body. The second acquisition unit 120 acquires second behavior information from the second detection unit 122. The second detection unit 122 acquires an image including at least a part of the second moving body, detects the second behavior by using the image, and generates second behavior information.

[0048] The second detection unit 122 acquires the image captured by the second camera 421. The second camera 421 is provided to image the surroundings of the first moving body. The second detection unit 122 performs, for example, extraction and analysis of feature points in the image acquired from the second camera 421 to extract the second moving body. For example, a pedestrian existing at the end of a crosswalk or an oncoming vehicle located diagonally in front of the first moving body is extracted as the second moving body.

[0049] In the acquired image, the second detection unit 122 performs, for example, extraction and analysis of feature points to detect the second behavior of the second moving body. Information indicating the pattern of feature points for the second behavior is stored in advance in the storage unit 170 for each content of the behavior, and the second detection unit 122 can read it out and use it for detection. When the second detection unit 122 detects the second behavior, it generates information indicating the content of the behavior as first behavior information. When the second behavior information is generated by the second detection unit 122, the second acquisition unit 120 acquires the second behavior information.

[0050] Note that the second detection unit 122 may perform extraction of the second moving body and detection of the second behavior based on a plurality of images such as a video. Also, similar to the first detection unit 112, the second detection unit 122 may detect the second behavior by using a ToF camera or a lidar, or may use deep learning.

[0051] Here, the second acquisition unit 120 acquires second behavior information indicating the second behavior within a predetermined period T1 from the occurrence of the first behavior. The second detection unit 122 may detect only the behavior within the predetermined period T1 after the first detection unit 112 detects the first behavior as the second behavior, for example.

[0052] Also, when the second detection unit 122 detects the second behavior regardless of timing, if the second behavior is performed after the predetermined period T1 has elapsed since the first behavior was detected, the second acquisition unit 120 may not acquire the second behavior information indicating the second behavior. In this case, the second acquisition unit 120 acquires together the second behavior information and information indicating the time when the second behavior indicated by the second behavior information was performed. Further, the second acquisition unit 120 acquires information indicating the occurrence time of the first behavior. Note that when the occurrence time of the first behavior is substantially the same as the time when the first acquisition unit 110 acquires the first behavior information, and the occurrence time of the second behavior is substantially the same as the time when the second acquisition unit 120 attempts to acquire the second behavior information, the second acquisition unit 120 may compare the time when the first acquisition unit 110 acquires the first behavior information with the time when the second acquisition unit 120 attempts to acquire the second behavior information, and determine whether the second behavior was performed within the predetermined period T1 after the first behavior was detected.

[0053] Further, the first acquisition unit 110 and the second acquisition unit 120 may acquire the first behavior information and the second behavior information regardless of timing. This case will be described in detail later.

[0054] As described above, the output device 10 includes a determination unit 140. The determination unit 140 determines whether the first output unit 150 should output control information based on the first behavior information and the second behavior information. When the determination unit 140 determines that output should be made, the first output unit 150 outputs control information based on the first behavior information and the second behavior information.

[0055] Depending on the determination result of the determination unit 140, the output device 10 switches between a standby state and a non-standby state. That is, when the determination unit 140 determines that the first output unit 150 should output control information based on the first behavior information and the second behavior information, the output device 10 enters the standby state, and when the determination unit 140 determines that there is no need for the first output unit 150 to output control information based on the first behavior information and the second behavior information, the output device 10 enters the non-standby state. The first output unit 150 outputs control information based on the first behavior information and the second behavior information in the standby state. On the other hand, the first output unit 150 does not output control information based on the first behavior information and the second behavior information in the non-standby state.

[0056] For example, in the non-standby state, the first detection unit 112 and the second detection unit 122 may not perform behavior detection. Also, in the non-standby state, the first acquisition unit 110 and the second acquisition unit 120 may not acquire the first behavior information and the second behavior information, respectively. Further, the first output unit 150 may not generate control information based on the first behavior information and the second behavior information. Note that the first output unit 150 may generate and output other control information not based on the first behavior information and the second behavior information in the non-standby state.

[0057] By providing a determination unit 140 that determines whether the first output unit 150 should output control information based on the first behavior information and the second behavior information, the control information is generated based on the first behavior information and the second behavior information only when necessary, and the processing load on the output device 10 is reduced. For example, when detection by the first detection unit 112 and the second detection unit 122 is not performed in the non-standby state, the first detection unit 112 and the second detection unit 122 do not need to continue image processing.

[0058] In this embodiment, when the speed of the first moving body becomes equal to or lower than a predetermined speed V, the determination unit 140 determines that the first output unit 150 should output control information based on the first behavior information and the second behavior information. The speed V is not particularly limited. For example, it is 2 km / h or more and 20 km / h or less. The speed reduction of the first moving body may be due to deceleration by automatic driving or deceleration by the driver's instruction or operation. For example, when the control device for automatic driving cannot determine the next behavior of the second moving body based only on the behavior of the second moving body alone, it may be decelerated. In addition, when the passenger of the first moving body discovers the second moving body and considers that communication is necessary, or when the surrounding environment such as a narrow road requires attention, it may be decelerated. Information indicating the predetermined speed V is held in the storage unit 170, and the determination unit 140 can read it out and use it for determination.

[0059] Note that when the acceleration of the first moving body becomes equal to or lower than a predetermined acceleration A, the determination unit 140 may determine that the first output unit 150 should output control information based on the first behavior information and the second behavior information. By defining the negative acceleration as the acceleration A, the determination unit 140 can put the output device 10 into the standby state when the first moving body decelerates.

[0060] When the determination unit 140 determines that the first output unit 150 should output control information based on the first behavior information and the second behavior information, and when the output device 10 enters the standby state, it may be presented to the passengers of the first moving body by displaying on a display such as the monitor 422 or by voice from the speaker 425 that it is in the standby state. The passengers can perform the first behavior upon receiving the presentation.

[0061] Also, when the output device 10 enters the standby state, the first output unit 150 may output control information to slow down or stop the first moving body. Further, when the output device 10 enters the standby state and the presence of the second moving body is detected, the first output unit 150 may output control information to slow down or stop the first moving body. Note that the second moving body can be detected by the second detection unit 122 in the same manner as described above.

[0062] Also, the output device 10 may enter the standby state when the determination unit 140 determines that the first output unit 150 should output control information based on the first behavior information and the second behavior information, and the second detection unit 122 detects the presence of the second moving body. In this case, the second detection unit 122 may always perform the process of detecting the second moving body during the startup of the output device 10, or may start after the determination unit 140 determines that the first output unit 150 should output control information based on the first behavior information and the second behavior information.

[0063] Also, when the first moving body is equipped with an automatic driving device, when the first moving body acquires auxiliary request information from the automatic driving device, the determination unit 140 may determine that the first output unit 150 should output control information based on the first behavior information and the second behavior information, and the output device 10 may enter the standby state. The automatic driving device outputs auxiliary request information when it is difficult to determine the operation that the first moving body should perform under automatic driving control.

[0064] The determination unit 140 makes a determination every predetermined time T3. The time T3 is not particularly limited, but for example, it is 1 second or more and 1 minute or less. When the speed of the first moving body exceeds a predetermined speed V, the determination unit 140 determines that it is not necessary for the first output unit 150 to output control information based on the first behavior information and the second behavior information, and sets the output device 10 to a non-standby state.

[0065] The first output unit 150 outputs control information based on the first behavior information and the second behavior information. In the standby state, when the first acquisition unit 110 acquires the first behavior information and the second acquisition unit 120 acquires the second behavior information, the first output unit 150 generates and outputs control information based on the combination of the acquired first behavior information and the second behavior information. The relationship between the combination of the first behavior information and the second behavior information and the control information to be generated is predetermined based on the assumed communication pattern between the first moving body and the second moving body. And reference information indicating the relationship between the combination of the first behavior information and the second behavior information and the control information to be generated is held in the storage unit 170 in advance, and the first output unit 150 can read the reference information from the storage unit 170 and use it for generating the control information.

[0066] When the first acquisition unit 110 and the second acquisition unit 120 acquire the first behavior information and the second behavior information regardless of timing, the first output unit 150 may determine whether the second behavior is performed within a predetermined period T1 after detecting the first behavior. The first output unit 150 can make the determination using at least one of the time when the first behavior information and the second behavior information are respectively generated and the time when the first behavior information and the second behavior information are respectively acquired by the first acquisition unit 110 and the second acquisition unit 120. The first output unit 150 generates and outputs control information based on the first behavior information and the second behavior information only when the second behavior information indicating the second behavior is acquired by the second acquisition unit 120 within the predetermined period T1 from the occurrence of the first behavior.

[0067] The first moving body is controlled by the control information output from the first output unit 150. The control information is, for example, information for stopping the first moving body, information for starting the first moving body, and the like.

[0068] FIG. 5 is a flowchart illustrating the processing performed by the output device 10 according to the first embodiment. Hereinafter, an example in which the first output unit 150 determines whether or not the second behavior is performed within a predetermined period T1 after detecting the first behavior will be described. However, the processing performed by the output device 10 is not limited to this example. In the output device 10, the standby state and the non-standby state are switched by the determination unit 140. First, in step S100, the first output unit 150 checks whether the output device 10 is in the standby state. The first output unit 150 can, for example, acquire information indicating the state at that time from the determination unit 140. When the output device 10 is not in the standby state, that is, when it is in the non-standby state, the first output unit 150 performs step S100 again after a predetermined time T4 has elapsed. On the other hand, when the output device 10 is in the standby state, the process proceeds to step S200. The time T4 is not particularly limited, but is, for example, 1 second or more and 1 minute or less.

[0069] In step S200, the first output unit 150 checks whether the first acquisition unit 110 has acquired the first behavior information. If the first behavior information is not acquired by the first acquisition unit 110 during the predetermined waiting time T5, the process returns to step S100. On the other hand, when the first behavior information is acquired by the first acquisition unit 110 during the predetermined waiting time T5, the process proceeds to step S300. The waiting time T5 is not particularly limited, but is, for example, 1 second or more and 1 minute or less.

[0070] In step S300, the first output unit 150 checks whether the second acquisition unit 120 has acquired the second behavior information. If the second behavior information is not acquired by the second acquisition unit 120 within a predetermined period T1 after the first behavior information is acquired, the process returns to step S100. On the other hand, if the second behavior information is acquired by the second acquisition unit 120 within the predetermined period T1 after the first behavior information is acquired, the process proceeds to step S500. In this example, it is assumed that the occurrence time of the first behavior is approximately the same as the time when the first acquisition unit 110 acquires the first behavior information, and the occurrence time of the second behavior is approximately the same as the time when the second acquisition unit 120 acquires the second behavior information.

[0071] In step S500, the first output unit 150 generates control information using the acquired first behavior information, second behavior information, and reference information, and outputs the generated control information. Then, the process returns to step S100 again.

[0072] As described above, according to this embodiment, similar to the embodiment, the first output unit 150 outputs control information for controlling the running of the first moving body based on the first behavior information and the second behavior information. Therefore, smooth communication of intentions with other moving bodies is achieved, and smooth automatic driving is realized.

[0073] In addition, according to this embodiment, when it is determined that the determination unit 140 should output, the first output unit 150 outputs control information. Therefore, control information is generated based on the first behavior information and the second behavior information only when necessary, and the processing load of the output device 10 is reduced.

[0074] (Embodiment 2) FIG. 6 is a flowchart illustrating the processing performed by the output device 10 according to Embodiment 2. The output device 10 according to this embodiment is the same as the output device 10 according to Embodiment 1 except for the points described below. In the output device 10 according to this embodiment, the first acquisition unit 110 further acquires third behavior information indicating a third behavior of the passenger within a predetermined period T2 from the occurrence of the second behavior. The first output unit 150 outputs control information based on the third behavior information as well. This will be described in detail below.

[0075] The third behavior is the behavior of the passenger of the first moving body who has received the second behavior by the second moving body. The passenger of the first moving body who has received the second behavior may, for example, show a will of gratitude to the second moving body, or take an action towards the next movement of the first moving body in response to the second behavior. The third behavior is, for example, the behavior of the passenger such as raising one hand or making a bow, the behavior related to the line of sight of the passenger such as checking the safety around or staring at the pedestrian, or the behavior of operating the headlight, hazard lamp, or brake lamp of the first moving body to blink one or more times, or the behavior of operating the horn of the first moving body to sound one or more times.

[0076] In the output device 10 according to the present embodiment, the first output unit 150 outputs control information based on the third behavior information in addition to the first behavior information and the second behavior information. Therefore, the result of communication between the passenger of the first moving body and the second moving body can be grasped more accurately, and the first moving body can be controlled.

[0077] In the output device 10 according to the present embodiment, the first acquisition unit 110 acquires the third behavior in the same manner as the acquisition of the first behavior. The third behavior information is an ID or the like indicating what kind of behavior the third behavior is. The third behavior may or may not be a behavior directed towards the second moving body. It should be noted that the third behavior is preferably a behavior that can be recognized directly or indirectly from the second moving body.

[0078] Also, the third behavior may be an input to the first input unit 160, similar to the first behavior. For example, when the passenger performs any one of a plurality of input operations defined as candidates for the third behavior on the first input unit 160, the first acquisition unit 110 acquires information indicating the operation content as the third behavior information.

[0079] Furthermore, the third behavior may be the same as the first behavior and be detected by the first detection unit 112. The first detection unit 112 can detect the third behavior in the same way as it detects the first behavior. Information indicating the pattern of feature points for the third behavior is stored in advance in the storage unit 170 for each behavior content, and the first detection unit 112 can read it out and use it for detection. When the first detection unit 112 detects the third behavior, the first detection unit 112 generates information indicating the content of the behavior as the third behavior information. When the third behavior information is generated by the first detection unit 112, the first acquisition unit 110 acquires the third behavior information.

[0080] Here, the first acquisition unit 110 acquires the third behavior information indicating the third behavior within a predetermined period T2 from the occurrence of the second behavior. Note that the predetermined period T1 between the occurrence of the first behavior and the occurrence of the second behavior and the predetermined period T2 between the occurrence of the second behavior and the occurrence of the third behavior may be the same as or different from each other. The predetermined period T2 is determined in advance based on the time assumed to be required for the passengers of the first moving body to react upon receiving the second behavior. The predetermined period T2 is, for example, within 5 seconds or within 3 seconds.

[0081] The first detection unit 112 may detect only the behavior within the predetermined period T2 after the second detection unit 122 detects the second behavior as the third behavior, for example.

[0082] Also, when the first detection unit 112 detects the first behavior regardless of timing, if the third behavior is performed after a predetermined period T2 has elapsed since the second acquisition unit 120 detects the second behavior, the first acquisition unit 110 may not acquire the third behavior information indicating the third behavior. In this case, the second acquisition unit 120 acquires together the third behavior information and the information indicating the time when the third behavior indicated by the third behavior information is performed. Further, the second acquisition unit 120 acquires the information indicating the occurrence time of the third behavior. Note that when the occurrence time of the second behavior and the time when the second acquisition unit 120 acquires the second behavior information are substantially the same, and the occurrence time of the third behavior and the time when the first acquisition unit 110 attempts to acquire the third behavior information are substantially the same, the first acquisition unit 110 may compare the time when the second acquisition unit 120 acquires the second behavior information with the time when the first acquisition unit 110 attempts to acquire the third behavior information, and determine whether the third behavior is performed within the predetermined period T2 after the second behavior is detected.

[0083] Also, the second acquisition unit 120 and the first acquisition unit 110 may acquire the second behavior information and the third behavior information regardless of timing. In this case, the first output unit 150 may determine whether the third behavior is performed within the predetermined period T2 after the second behavior is detected. The first output unit 150 can perform the determination using at least one of the time when the second behavior information and the third behavior information are respectively generated, and the time when the second behavior information and the third behavior information are respectively acquired by the second acquisition unit 120 and the first acquisition unit 110. The first output unit 150 generates and outputs control information based on the first behavior information, the second behavior information, and the third behavior information only when the third behavior information indicating the third behavior is acquired by the first acquisition unit 110 within the predetermined period T2 from the occurrence of the second behavior.

[0084] Referring to FIG. 6, the output method according to this embodiment will be described below. In this embodiment, steps S100 to S300 are performed in the same manner as in the first embodiment. In this embodiment, in step S300, when the second acquisition unit 120 acquires the second behavior information within the predetermined period T1 after the first behavior information is acquired, the process proceeds to step S400.

[0085] In step S400, the first output unit 150 checks whether the first acquisition unit 110 has acquired the third behavior information. If the third behavior information is not acquired by the first acquisition unit 110 within a predetermined period T2 after the second behavior information is acquired, the process returns to step S100. On the other hand, if the third behavior information is acquired by the first acquisition unit 110 within a predetermined period T2 after the second behavior information is acquired, the process proceeds to step S500. In this example, it is assumed that the occurrence time of the second behavior is substantially the same as the time when the second acquisition unit 120 acquires the second behavior information, and the occurrence time of the third behavior is substantially the same as the time when the first acquisition unit 110 acquires the third behavior information.

[0086] In step S500, the first output unit 150 generates control information using the acquired first behavior information, second behavior information, and third behavior information, and the reference information, and outputs the generated control information. Then, the process returns to step S100 again.

[0087] In this embodiment, the reference information is information indicating the relationship between the combination of the first behavior information, the second behavior information, and the third behavior information, and the control information to be generated.

[0088] As described above, according to this embodiment, similar to the embodiment, the first output unit 150 outputs control information for controlling the travel of the first moving body based on the first behavior information and the second behavior information. Therefore, smooth communication with other moving bodies is achieved, and smooth automatic driving is realized.

[0089] In addition, according to this embodiment, the first output unit 150 outputs control information based on the third behavior information as well. Therefore, the result of communication between the passenger of the first moving body and the second moving body is grasped more accurately, and the first moving body is controlled.

[0090] (Embodiment 3) The output device 10 according to Embodiment 3 is the same as the output device 10 according to at least one of Embodiment 1 or Embodiment 2, except that when the determination unit 140 determines that the first moving body is located within a predetermined range from a crosswalk or an intersection, the first output unit 150 should output control information based on the first behavior information and the second behavior information. This will be described in detail below.

[0091] In the vicinity of a crosswalk or an intersection, there is a high possibility that communication between the passengers of the first moving body and the second moving body is required. Therefore, when the first moving body is located within a predetermined range from a crosswalk or an intersection, by setting the output device 10 to the standby state by the determination unit 140, automatic driving can be smoothly performed in the vicinity of the crosswalk or the intersection.

[0092] The determination unit 140 according to this embodiment acquires the position information of the first moving body and at least the map information of the area where the first moving body is located. The first moving body is equipped with, for example, a GPS (Global Positioning System) receiver, and the determination unit 140 acquires the position information indicating the position of the first moving body from the GPS device mounted on the first moving body. Also, the map information is stored in advance in the storage unit 170, and the determination unit 140 can read and use it. Further, the determination unit 140 may acquire the map information from an external server via a communication network. The map information includes information indicating the position of the crosswalk and information indicating the position of the intersection.

[0093] The determination unit 140 uses the acquired position information and map information to determine whether the first moving body is located within a predetermined range from a crosswalk or an intersection. The predetermined range is determined assuming a range where communication between the passengers of the first moving body and the second moving body is required. The predetermined range means, for example, that the distance between the first moving body and the crosswalk or the intersection is within L [m]. L is, for example, 2 or more and 100 or less. Note that the determination unit 140 may determine that the first moving body is located within a predetermined range from a crosswalk or an intersection only when a crosswalk or an intersection is located in front of the traveling direction of the first moving body.

[0094] When the first moving body is located within a predetermined range from a crosswalk or an intersection, the determination unit 140 determines that the first output unit 150 should output control information based on the first behavior information and the second behavior information. That is, the output device 10 is set to the standby state. On the other hand, when the first moving body is not located within a predetermined range from a crosswalk or an intersection, the determination unit 140 determines that there is no need for the first output unit 150 to output control information based on the first behavior information and the second behavior information. That is, the output device 10 is set to the non-standby state.

[0095] Also, in the present embodiment, reference information may be prepared for each fact that serves as the basis for the determination unit 140 to determine that the first output unit 150 should output control information based on the first behavior information and the second behavior information. In this case, when the determination unit 140 determines that the first output unit 150 should output control information based on the first behavior information and the second behavior information, the determination unit 140 generates information indicating the fact (for example, "being located within a predetermined range from a crosswalk", "the speed of the first moving body becoming equal to or lower than a predetermined speed") that serves as the basis for the determination.

[0096] In step S500, the first output unit 150 acquires information indicating the fact that serves as the basis for the determination from the determination unit 140, and reads out the reference information corresponding to the fact indicated by the information from the storage unit 170. Then, the first output unit 150 generates and outputs control information using the read reference information.

[0097] The content of the communication between the passenger of the first moving body and the second moving body may vary depending on the scene where the communication takes place. Therefore, by the first output unit 150 outputting control information using the reference information prepared for each fact that serves as the basis, smooth automatic driving is performed in each scene.

[0098] As described above, according to the present embodiment, similar to the embodiment, the first output unit 150 outputs control information for controlling the running of the first moving body based on the first behavior information and the second behavior information. Therefore, smooth communication with other moving bodies is achieved, and smooth automatic driving is realized.

[0099] In addition, according to the present embodiment, when the determination unit 140 determines that the first moving body is located within a predetermined range from a crosswalk or an intersection, the first output unit 150 determines that control information should be output based on the first behavior information and the second behavior information. Therefore, smooth automatic driving is performed near crosswalks and intersections.

[0100] (Embodiment 4) In the output device 10 according to Embodiment 4, the determination unit 140 is the same as the output device 10 according to at least any one of Embodiments 1 to 3, except that when the width of the road on which the first moving body travels satisfies a predetermined condition, the first output unit 150 determines that control information should be output based on the first behavior information and the second behavior information. This will be described in detail below.

[0101] In this embodiment, the first moving body is a vehicle, and the second moving body is an oncoming vehicle of the first moving body. When two vehicles pass each other on a narrow road, the two oncoming vehicles may communicate with each other and yield the road. Therefore, when the width of the road on which the first moving body travels satisfies a predetermined condition, by setting the output device 10 to the standby state by the determination unit 140, smooth automatic driving is performed even on a narrow road.

[0102] The determination unit 140 can make a determination using, for example, the road width included in the map information. Further, the determination unit 140 may make a determination using the result of actually measuring the surroundings of the first moving body.

[0103] An example in which the determination unit 140 makes a determination using the road width indicated in the map information will be described. The determination unit 140 acquires map information and the position information of the first moving body in the same manner as in the third embodiment. In this example, the map information includes information indicating the width of each road. Then, the determination unit 140 determines whether the width of the road on which the first moving body travels satisfies a predetermined condition using the acquired position information and map information. This condition indicates that the width is narrow enough to require caution when two moving bodies pass by each other. Specifically, for example, when the width of the road on which the first moving body travels is W [m] or less, the determination unit 140 determines that the road width satisfies a predetermined condition. W is, for example, 7 or more and 10 or less.

[0104] The case where the determination unit 140 makes a determination using the actually measured result of the surroundings of the first moving body will be described below. In this example, the first moving body is provided with a sensor that measures the distance between the lateral side of the first moving body and a structure facing the lateral side. Then, the determination unit 140 acquires the distance between the lateral side of the first moving body and the structure from the sensor. The structure is, for example, a building, a wall, or a guardrail provided on the side of the road. The determination unit 140 determines that the width of the road on which the first moving body travels satisfies a predetermined condition when the acquired distance is equal to or less than a predetermined reference distance.

[0105] When the width of the road on which the first moving body travels satisfies a predetermined condition, the determination unit 140 determines that the first output unit 150 should output control information based on the first behavior information and the second behavior information. That is, the output device 10 is set to the standby state. On the other hand, when the width of the road on which the first moving body travels does not satisfy a predetermined condition, the determination unit 140 determines that there is no need for the first output unit 150 to output control information based on the first behavior information and the second behavior information. That is, the output device 10 is set to the non-standby state.

[0106] Note that, also in this embodiment, similar to Embodiment 3, for each fact serving as a basis for the determination unit 140 to determine that the first output unit 150 should output control information based on the first behavior information and the second behavior information, reference information may be prepared. Then, the first output unit 150 may generate and output control information using the reference information corresponding to the fact serving as the basis for the determination.

[0107] As described above, according to this embodiment, similar to the embodiment, the first output unit 150 outputs control information for controlling the travel of the first moving body based on the first behavior information and the second behavior information. Therefore, smooth communication of intentions with other moving bodies is achieved, and smooth automatic driving is realized.

[0108] In addition, according to this embodiment, the determination unit 140 determines that the first output unit 150 should output control information based on the first behavior information and the second behavior information when the width of the road on which the first moving body travels satisfies a predetermined condition. Therefore, automatic driving is smoothly performed even on a narrow road.

[0109] (Embodiment 5) FIG. 7 is a block diagram illustrating the functional configuration of the output device 10 according to Embodiment 5. The output device 10 according to this embodiment is the same as the output device 10 according to at least any one of Embodiments 1 to 4, except for the points described below.

[0110] The output device 10 according to this embodiment further includes a third acquisition unit 130. The third acquisition unit 130 acquires information indicating an area that requires attention, where the first output unit 150 should output control information based on the first behavior information and the second behavior information. Then, the determination unit 140 determines that the first output unit 150 should output control information based on the first behavior information and the second behavior information when the first moving body is located within the area that requires attention. This will be described in detail below.

[0111] Places where communication may be necessary between the passengers of the first moving body and the second moving body, such as intersections, crosswalks, narrow roads, on-ramps to highways or store entrances, etc., can be predicted to a certain extent in advance. By designating such places as areas that require attention, the area where autonomous driving can be smoothly performed can be expanded.

[0112] The area that requires attention is an area where control information should be output based on the first behavior information and the second behavior information. The area that requires attention includes, for example, the vicinity of intersections and crosswalks, narrow roads, and the vicinity of merging points on highways or store entrances. As the area that requires attention, a plurality of areas where communication between the passengers of the first moving body and the second moving body is expected can be predicted and determined in advance. The information indicating the area that requires attention is, for example, a list showing the range of each area in terms of latitude and longitude. Also, the information indicating the area that requires attention may be information indicating an area on a map linked to map information.

[0113] The third acquisition unit 130 acquires information indicating the area that requires attention from an external server via a communication network, for example. Also, the third acquisition unit 130 may read and acquire the information indicating the area that requires attention pre-held in the storage unit 170.

[0114] The determination unit 140 acquires the position information of the first moving body in the same manner as in the third embodiment. Then, using the position information and the information indicating the area that requires attention acquired by the third acquisition unit 130, it is determined whether the first moving body is located within the area that requires attention. When the first moving body is located within the area that requires attention, it is determined that the first output unit 150 should output control information based on the first behavior information and the second behavior information. That is, the output device 10 is set to the standby state. On the other hand, when the first moving body is not located within the area that requires attention, it is determined that there is no need for the first output unit 150 to output control information based on the first behavior information and the second behavior information. That is, the output device 10 is set to the non-standby state.

[0115] Note that, also in this embodiment, similar to Embodiment 3, for each fact serving as a basis for the determination unit 140 to determine that the first output unit 150 should output control information based on the first behavior information and the second behavior information, reference information may be prepared. Then, the first output unit 150 may generate and output control information using the reference information corresponding to the fact serving as the basis for the determination.

[0116] As described above, according to this embodiment, similar to the embodiment, the first output unit 150 outputs control information for controlling the travel of the first moving body based on the first behavior information and the second behavior information. Therefore, smooth communication of intentions with other moving bodies is achieved, and smooth automatic driving is realized.

[0117] In addition, according to this embodiment, the determination unit 140 determines that the first output unit 150 should output control information based on the first behavior information and the second behavior information when the first moving body is located within the area that requires attention. Therefore, the area where automatic driving can be smoothly performed can be expanded.

[0118] (Embodiment 6) FIG. 8 is a block diagram illustrating the functional configuration of the output device 10 according to Embodiment 6. The output device 10 according to this embodiment is the same as the output device 10 according to at least any one of Embodiments 1 to 5, except for the points described below.

[0119] The output device 10 according to this embodiment further includes a second output unit 152 and a third output unit 153 for notifying the passengers. The output device 10 according to this embodiment further includes a second input unit 162 capable of receiving an input from the passengers. In the example of FIG. 4, the monitor 422 and the speaker 425 function as the second output unit 152. Also, the monitor 422 and the speaker 425 function as the third output unit 153. The input panel 423 and the microphone 424 function as the second input unit 162. The output of the second output unit 152 and the output of the third output unit 153 can be voice, screen display, AR (Augmented Reality) display, etc.

[0120] In this embodiment, the second output unit 152 outputs notification information for notifying the occupant of the content of the control information generated by the first output unit 150. The third output unit 153 outputs information for prompting the behavior of the occupant of the first moving body. Further, the second input unit 162 can receive an input for changing the content of the control information. When there is a problem with the next operation indicated by the control information generated by the first output unit 150, the occupant of the first moving body can input to the second input unit 162 to cancel the operation. Thereby, a smooth negotiation is performed between the first moving body and the second moving body. This will be described in detail below.

[0121] In this embodiment, the first moving body is equipped with an automatic driving device. For example, when the first moving body approaches a crosswalk and it is detected that there is a pedestrian (the second moving body) near the crosswalk, based on automatic driving, the first moving body stops in front of the stop line near the crosswalk to wait for the pedestrian to cross.

[0122] Here, when the pedestrian does not start crossing the crosswalk, auxiliary request information is output from the automatic driving device. Then, the determination unit 140 sets the output device 10 to the standby state.

[0123] In this embodiment, when the output device 10 enters the standby state, the third output unit 153 outputs information for prompting communication with the second moving body to the occupant of the first moving body. The information for prompting communication is also referred to as "request information" below. The request information is information for prompting the behavior of the occupant of the first moving body. As the output of the request information, for example, a voice guidance or a screen display such as "A pedestrian has been detected near the sidewalk. Could you please encourage the pedestrian to cross?" is performed. Further, the third output unit 153 may output a combination of information indicating the state of the first moving body, such as "The vehicle is stopped to wait for the pedestrian to cross." Also, the output of the third output unit 153 may be an AR display that emphasizes the second moving body.

[0124] According to the request information, the passenger of the first moving body attempts to communicate with the second moving body. Then, in the output device 10, the first behavior and the second behavior are detected. Here, the first acquisition unit 110 may acquire, as the first behavior information, information indicating the behavior of the passenger that occurred within a predetermined period from the output of the request information.

[0125] Next, the first output unit 150 generates control information for controlling the running of the first moving body based on the first behavior information and the second behavior information.

[0126] FIG. 9 is a flowchart illustrating the processing performed by the output device 10 according to the sixth embodiment. The processing performed by the output device 10 is the same as the processing performed by the output device 10 according to at least any one of the first to fifth embodiments, except for the points described below.

[0127] In this embodiment, in step S100, when the output device 10 is in the standby state, in step S150, the third output unit 153 presents the request information to the passenger of the first moving body. Then, communication is performed between the passenger of the first moving body and the second moving body, and steps S200 and S300 are performed in the same manner as in the first embodiment. However, in step S200, the first acquisition unit 110 may acquire, as the first behavior information, information indicating the behavior of the passenger that occurred within a predetermined period from the output of the request information.

[0128] When the second behavior information is acquired in step S300 (Y in step S300), in step S501, the first output unit 150 generates control information in the same manner as in step S500 of the first embodiment. Then, in step S502, the second output unit 152 outputs notification information for notifying the passenger of the content of the control information generated by the first output unit 150.

[0129] Next, in step S503, the passenger inputs to the second input unit 162 whether or not to cancel the next operation indicated in the output notification information. When an input to cancel is made to the second input unit 162 (Y in step S503), the first output unit 150 stops outputting control information (step S505) and returns to step S100 again. In this case, the second output unit 152 may further notify the passenger to request an input of an alternative operation. The passenger who has received the notification makes an input indicating an alternative operation to the second input unit 162, and the first output unit 150 outputs control information based on the input content to the second input unit 162.

[0130] On the other hand, when an input not to cancel is made to the second input unit 162 (N in step S503), the first output unit 150 outputs the generated control information (step S504). Then, the process returns to step S100 again.

[0131] Note that in step S503, an example has been described above in which the passenger inputs to the second input unit 162 whether or not to cancel the next operation indicated in the output notification information. However, the passenger may input to the second input unit 162 whether or not to approve the next operation indicated in the output notification information.

[0132] As described above, according to the present embodiment, similar to the embodiment, the first output unit 150 outputs control information for controlling the travel of the first moving body based on the first behavior information and the second behavior information. Therefore, smooth communication with other moving bodies is achieved, and smooth automatic driving is realized.

[0133] In addition, according to the present embodiment, the second output unit 152 outputs notification information for notifying the passenger of the content of the control information generated by the first output unit 150. Therefore, the result of communication between the passenger of the first moving body and the second moving body is grasped more accurately, and the first moving body is controlled.

[0134] The embodiments and examples have been described above with reference to the drawings. These are merely examples of the present invention, and various other configurations may be adopted. For example, in the sequence diagrams and flowcharts used in the above description, a plurality of steps (processes) are described in order. However, the execution order of the steps executed in each example is not limited to the order described. In each example, the order of the illustrated steps can be changed within a range that does not interfere with the content. Further, the above-described embodiments and examples can be combined within a range where their contents do not conflict with each other.

[0135] Examples of reference forms are appended below. 1-1. A first acquisition unit that acquires first behavior information indicating a first behavior of a passenger of a first moving body, a second acquisition unit that acquires second behavior information indicating a second behavior of a second moving body around the first moving body within a predetermined period from the occurrence of the first behavior, a first output unit that outputs control information for controlling the travel of the first moving body based on the first behavior information and the second behavior information. An output device comprising the above. 1-2. In the output device according to 1-1., the first acquisition unit further acquires third behavior information indicating a third behavior of the passenger within a predetermined period from the occurrence of the second behavior, and the first output unit outputs the control information further based on the third behavior information. An output device. 1-3. In the output device according to 1-1. or 1-2., the output device further comprises a determination unit that determines whether the first output unit should output the control information based on the first behavior information and the second behavior information, and when the determination unit determines that the control information should be output, the first output unit outputs the control information. An output device. 1-4. In the output device according to 1-3., the determination unit determines that the first output unit should output the control information based on the first behavior information and the second behavior information when the first moving body is located within a predetermined range from a crosswalk or an intersection. An output device. In the output device according to 1-5, 1-3, or 1-4, the determination unit determines that the first output unit should output the control information based on the first behavior information and the second behavior information when the speed of the first moving body becomes equal to or lower than a predetermined speed. An output device. In the output device according to any one of 1-6 to 1-3, the first output unit further includes a third acquisition unit that acquires information indicating an area that requires attention, which should output the control information based on the first behavior information and the second behavior information. the determination unit determines that the first output unit should output the control information based on the first behavior information and the second behavior information when the first moving body is located within the area that requires attention. An output device. In the output device according to any one of 1-7 to 1-3, the determination unit determines that the first output unit should output the control information based on the first behavior information and the second behavior information when the width of the road on which the first moving body travels satisfies a predetermined condition. An output device. In the output device according to any one of 1-8 to 1-1, the first behavior is a gesture or voice by a passenger of the first moving body with respect to the second moving body. An output device. In the output device according to any one of 1-9 to 1-1, the first moving body is a vehicle, and the second moving body is an oncoming vehicle of the first moving body. the second behavior is a behavior in which the second moving body blinks a headlight, a hazard lamp, or a brake lamp one or more times, or a behavior in which a horn is sounded one or more times. An output device. In the output device according to any one of 1-10 to 1-1, the first acquisition unit can receive an input from the passenger to the first input unit. the first behavior is an input to the first input unit. An output device. 1-11. In the output device according to any one of 1-1. to 1-10., further comprising a detection unit that detects the behavior of the second moving body, the second acquisition unit acquires the second behavior information from the detection unit, the detection unit, acquires an image including at least a part of the second moving body, and uses the image to detect the second behavior and generate the second behavior information. An output device. 1-12. In the output device according to any one of 1-1. to 1-11., an output device further comprising a second output unit that outputs notification information for notifying the content of the control information to the passenger. 1-13. In the output device according to any one of 1-1. to 1-12., an output device further comprising a second input unit capable of receiving an input for changing the content of the control information. 1-14. In the output device according to any one of 1-1. to 1-13., an output device further comprising a third output unit that outputs request information for prompting the behavior of the passenger. 1-15. In the output device according to 1-14., the first acquisition unit acquires, as the first behavior information, information indicating the behavior of the passenger that occurred within a predetermined period from the output of the request information. An output device. 2-1. A first acquisition step of acquiring first behavior information indicating a first behavior of a passenger of a first moving body, a second acquisition step of acquiring second behavior information indicating a second behavior of a second moving body around the first moving body within a predetermined period from the occurrence of the first behavior, a first output step of outputting control information for controlling the running of the first moving body based on the first behavior information and the second behavior information, and an output method including. 2-2. In the output method according to 2-1., further acquiring third behavior information indicating a third behavior of the passenger within a predetermined period from the occurrence of the second behavior, In the first output step, an output method for outputting the control information based further on the third behavior information. In the output method according to 2-3., 2-1., or 2-2., further comprising a determination step of determining whether to output the control information based on the first behavior information and the second behavior information in the first output step, An output method in which, when it is determined in the determination step that the control information should be output, the control information is output in the first output step. In the output method according to 2-4., in the determination step, when the first moving body is located within a predetermined range from a crosswalk or an intersection, it is determined that the control information should be output based on the first behavior information and the second behavior information in the first output step. In the output method according to 2-5., 2-3., or 2-4., in the determination step, when the speed of the first moving body becomes equal to or lower than a predetermined speed, it is determined that the control information should be output based on the first behavior information and the second behavior information in the first output step. In the output method according to any one of 2-6. to 2-3., the first output step further includes a third acquisition step of acquiring information indicating a caution area where the control information should be output based on the first behavior information and the second behavior information, in the determination step, when the first moving body is located within the caution area, it is determined that the control information should be output based on the first behavior information and the second behavior information in the first output step. In the output method according to any one of 2-7. to 2-3., in the determination step, when the width of the road on which the first moving body travels satisfies a predetermined condition, it is determined that the control information should be output based on the first behavior information and the second behavior information in the first output step. In the output method according to any one of 2-8. to 2-1., The first behavior is an output method that is a gesture or vocalization by a passenger of the first moving body with respect to the second moving body. 2-9. In the output method according to any one of 2-1. to 2-8., The first moving body is a vehicle, and the second moving body is an oncoming vehicle of the first moving body, The second behavior is an output method that is a behavior in which the second moving body blinks its headlights, hazard lamps, or brake lamps one or more times, or a behavior in which the horn is sounded one or more times. 2-10. In the output method according to any one of 2-1. to 2-9., In the first acquisition step, it is possible to receive an input from the passenger to the first input unit, The first behavior is an input to the first input unit. 2-11. In the output method according to any one of 2-1. to 2-10., It further includes a detection step of detecting the behavior of the second moving body, In the second acquisition step, the second behavior information generated in the detection step is acquired, In the detection step, An image including at least a part of the second moving body is acquired, The second behavior is detected using the image, and the second behavior information is generated. 2-12. In the output method according to any one of 2-1. to 2-11., The output method further includes a step of outputting notification information for notifying the passenger of the content of the control information. 2-13. In the output method according to any one of 2-1. to 2-12., The output method further includes a step of being able to receive an input for changing the content of the control information. 2-14. In the output method according to any one of 2-1. to 2-13., The output method further includes a third output step of outputting request information for prompting the passenger to behave. In the output method according to 2-15 and 2-14, in the first acquisition step, an output method of acquiring, as the first behavior information, information indicating the behavior of the passenger that occurred within a predetermined period from the output of the request information. A computer program for realizing an output device, causing a computer to function as first acquisition means for acquiring first behavior information indicating the first behavior of a passenger of a first moving body, second acquisition means for acquiring second behavior information indicating the second behavior of a second moving body around the first moving body within a predetermined period from the occurrence of the first behavior, and first output means for outputting control information for controlling the travel of the first moving body based on the first behavior information and the second behavior information. A computer program that functions as In the computer program according to 3-1, the first acquisition means further acquires third behavior information indicating the third behavior of the passenger within a predetermined period from the occurrence of the second behavior, and the first output means outputs the control information further based on the third behavior information. A computer program. In the computer program according to 3-1 or 3-2, the computer is further caused to function as determination means for determining whether or not the first output means should output the control information based on the first behavior information and the second behavior information, and when the determination means determines that the control information should be output, the first output means outputs the control information. A computer program. In the computer program according to 3-3, the determination means determines that the first output means should output the control information based on the first behavior information and the second behavior information when the first moving body is located within a predetermined range from a crosswalk or an intersection. A computer program. In the computer program according to 3-3 or 3-4, The determination means is a computer program that determines that the first output means should output the control information based on the first behavior information and the second behavior information when the speed of the first moving body becomes equal to or lower than a predetermined speed. 3-6. In the computer program according to any one of 3-3. to 3-5., The computer is further caused to function as a third acquisition means for acquiring information indicating an area that requires attention, where the first output means should output the control information based on the first behavior information and the second behavior information. The determination means is a computer program that determines that the first output means should output the control information based on the first behavior information and the second behavior information when the first moving body is located within the area that requires attention. 3-7. In the computer program according to any one of 3-3. to 3-6., The determination means is a computer program that determines that the first output means should output the control information based on the first behavior information and the second behavior information when the width of the road on which the first moving body travels satisfies a predetermined condition. 3-8. In the computer program according to any one of 3-1. to 3-7., The first behavior is a gesture or vocalization by a passenger of the first moving body with respect to the second moving body. 3-9. In the computer program according to any one of 3-1. to 3-8., The first moving body is a vehicle, and the second moving body is an oncoming vehicle of the first moving body. The second behavior is a behavior in which the second moving body blinks its headlight, hazard lamp, or brake lamp one or more times, or sounds its horn one or more times. 3-10. In the computer program according to any one of 3-1. to 3-9., The first acquisition means is capable of receiving an input from the passenger to the input means. The first behavior is a computer program that is an input to the input means. 3-11. In the computer program according to any one of 3-1. to 3-10., the computer further functions as detection means for detecting the behavior of the second moving body, the second acquisition unit acquires the second behavior information from the detection means, the detection means, acquires an image including at least a part of the second moving body, detects the second behavior using the image, and generates the second behavior information, a computer program. 3-12. In the computer program according to any one of 3-1. to 3-11., a computer program for causing the second output means to output notification information for notifying the content of the control information to the passenger. 3-13. In the computer program according to any one of 3-1. to 3-12., a computer program capable of receiving an input to a second input means for changing the content of the control information. 3-14. In the computer program according to any one of 3-1. to 3-13., a computer program for causing the third output means to output request information for prompting the passenger to behave. 3-15. In the computer program according to 3-14., the first acquisition means acquires, as the first behavior information, information indicating the behavior of the passenger that occurred within a predetermined period from the output of the request information, a computer program.

Explanation of Reference Numerals

[0136] 10 Output device 40 Integrated circuit 110 First acquisition unit 112 First detection unit 120 Second acquisition unit 122 Second detection unit 130 Third acquisition unit 140 Determination unit 150 First output unit 152 Second output unit 153 Third output unit 160 First input unit 162 Second input unit 170 Memory unit 402 Bus 404 Processor 406 Memory 408 Storage device 410 Input / output interface 412 Network interface 420 First camera 421 Second camera 422 Monitor 423 Input panel 424 Microphone 425 Speaker

Claims

1. a first acquisition unit that acquires first behavior information indicating a first behavior of a passenger of a first moving object; a second acquisition unit that acquires second behavior information indicating a second behavior of a second moving object around the first moving object within a predetermined period from the occurrence of the first behavior; a first output unit that outputs control information for controlling traveling of the first moving object based on the first behavior information and the second behavior information; Equipped with The output device, wherein the second moving object is a pedestrian.

2. a first acquisition unit that acquires first behavior information indicating a first behavior of a passenger of a first moving object; a second acquisition unit that acquires second behavior information indicating a second behavior of a second moving object itself or a passenger of the second moving object in the vicinity of the first moving object within a predetermined period from the occurrence of the first behavior; a first output unit that outputs control information for controlling traveling of the first moving object based on the first behavior information and the second behavior information; Equipped with An output device, wherein the second behavior of the second moving body itself is a behavior of the second moving body flashing headlights or hazard lights one or more times, or a behavior of the second moving body honking a horn one or more times.

3. 3. The output device according to claim 1, The first acquisition unit further acquires third behavior information indicating a third behavior of a passenger of the first moving object within a predetermined period from the occurrence of the second behavior, The first output unit is an output device that outputs the control information further based on the third behavior information.

4. In the output device according to any one of claims 1 to 3, a determination unit that determines whether or not the first output unit should output the control information based on the first behavior information and the second behavior information, When the determination unit determines that the control information should be output, the first output unit outputs the control information.

5. 5. The output device according to claim 4, The determination unit is an output device that determines that the first output unit should output the control information based on the first behavior information and the second behavior information when the first moving body is located within a predetermined range from a crosswalk or an intersection.

6. 6. The output device according to claim 4, The determination unit is an output device that determines that the first output unit should output the control information based on the first behavior information and the second behavior information when the speed of the first moving body becomes equal to or lower than a predetermined speed.

7. In the output device according to any one of claims 4 to 6, a third acquisition unit configured to acquire information indicating a caution area in which the first output unit should output the control information based on the first behavior information and the second behavior information, The determination unit is an output device that determines that the first output unit should output the control information based on the first behavior information and the second behavior information when the first moving body is located within the caution area.

8. In the output device according to any one of claims 4 to 7, The determination unit is an output device that determines that the first output unit should output the control information based on the first behavior information and the second behavior information when the width of the road on which the first moving body travels satisfies a predetermined condition.

9. In the output device according to any one of claims 1 to 8, An output device, wherein the first behavior is a gesture or vocalization by an occupant of the first moving body toward the second moving body.

10. In the output device according to any one of claims 1 to 9, The first behavior is any one of a hand movement behavior, a head movement behavior, a voice behavior, a behavior of flashing the headlights or hazard lights of the first moving body one or more times so that the second moving body can directly recognize the first behavior, and a behavior of honking the horn of the first moving body one or more times.

11. In the output device according to any one of claims 1 to 10, the first acquisition unit is capable of accepting an input from a passenger of the first moving object to a first input unit; An output device, wherein the first behavior is an input to the first input section.

12. In the output device according to any one of claims 1 to 11, A detection unit that detects a behavior of the second moving object is further provided, The second acquisition unit acquires the second behavior information from the detection unit, The detection unit is acquiring an image including at least a portion of the second moving object; An output device that detects the second behavior using the image and generates the second behavior information.

13. In the output device according to any one of claims 1 to 12, The output device further includes a second output section that outputs notification information for notifying a passenger of the first moving body of the content of the control information.

14. In the output device according to any one of claims 1 to 13, The output device further includes a second input unit capable of accepting an input for changing content of the control information.

15. In the output device according to any one of claims 1 to 14, The output device further includes a third output unit that outputs request information for encouraging an occupant of the first moving object to take an appropriate action.

16. 16. The output device according to claim 15, The first acquisition unit is an output device that acquires, as the first behavior information, information indicating behavior of a passenger of the first moving object that occurred within a predetermined period from the output of the request information.

17. An output method realized by a computer functioning as a first acquisition means, a second acquisition means, and a first output means by executing a program, comprising: a first acquisition step in which the first acquisition means acquires first behavior information indicating a first behavior of a passenger of a first moving object; a second acquisition step in which the second acquisition means acquires second behavior information indicating a second behavior of a second moving object around the first moving object within a predetermined period from the occurrence of the first behavior; a first output step in which the first output means outputs control information for controlling traveling of the first moving object based on the first behavior information and the second behavior information; Including, An output method in which the second moving object is a pedestrian.

18. An output method realized by a computer functioning as a first acquisition means, a second acquisition means, and a first output means by executing a program, comprising: a first acquisition step in which the first acquisition means acquires first behavior information indicating a first behavior of a passenger of a first moving object; a second acquisition step in which the second acquisition means acquires second behavior information indicating a second behavior of a second moving body itself or a passenger of the second moving body in the vicinity of the first moving body within a predetermined period from the occurrence of the first behavior; a first output step in which the first output means outputs control information for controlling traveling of the first moving object based on the first behavior information and the second behavior information; Including, The second behavior of the second moving body itself is a behavior of the second moving body flashing headlights, hazard lights, or brake lights one or more times, or a behavior of honking the horn one or more times.

19. A computer program for implementing an output device, Computer, a first acquisition means for acquiring first behavior information indicating a first behavior of a passenger of the first moving object; a second acquisition means for acquiring second behavior information indicating a second behavior of a second moving object around the first moving object within a predetermined period from the occurrence of the first behavior; and a first output means for outputting control information for controlling traveling of the first moving object based on the first behavior information and the second behavior information; Functioning as a A computer program, wherein the second moving object is a pedestrian.

20. A computer program for implementing an output device, Computer, a first acquisition means for acquiring first behavior information indicating a first behavior of a passenger of the first moving object; a second acquisition means for acquiring second behavior information indicating a second behavior of a second moving object itself or a passenger of the second moving object in the vicinity of the first moving object within a predetermined period from the occurrence of the first behavior; and a first output means for outputting control information for controlling traveling of the first moving object based on the first behavior information and the second behavior information; Functioning as a A computer program in which the behavior of the second moving body itself is a behavior of the second moving body flashing headlights, hazard lights, or brake lights one or more times, or a behavior of honking the horn one or more times.

Images