Output device

Abstract

One example is facilitating smooth communication with other moving vehicles to achieve seamless autonomous driving. [Solution] The output device 10 comprises a first acquisition unit 110, a second acquisition unit 120, and a first output unit 150. The first acquisition unit 110 acquires first behavior information indicating the first behavior of a occupant of the first mobile body. The second acquisition unit 120 acquires second behavior information indicating the second behavior of a second mobile body around the first mobile body within a predetermined period from the occurrence of the first behavior. The first output unit 150 outputs control information for controlling the movement of the first mobile body based on the first behavior information and the second behavior information.

Description

Technical Field

[0001] The present invention relates to an output device, an output method, and a computer program.

Background Art

[0002] In autonomous driving, the next movement of the host vehicle is determined while estimating the next movements of vehicles, pedestrians, etc. traveling around using various sensing results.

[0003] Patent Document 1 describes that automatic steering control by a system is executed by referring to the driving operation of a driver after detecting a pedestrian or the like, thereby increasing the degree of driver involvement in vehicle driving.

[0004] Further, Patent Document 2 describes that when a specific message is received from another moving body, a response message including intention information indicating the intention to execute a specific behavior is transmitted to that moving body.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, with the technology of Patent Document 1, communication of intentions with pedestrians or the like could not be achieved.

[0007] Further, with the technology of Patent Document 2, since information transmission and reception intervene, the smoothness of communication of intentions with other moving bodies may be impaired. Also, when a message or the like is not transmitted from another moving body, communication of intentions could not be achieved.

[0008] One example of a problem that this invention aims to solve is to facilitate smooth communication with other moving objects and realize smooth autonomous driving. [Means for solving the problem]

[0009] The first invention is, A first acquisition unit that acquires first behavior information indicating the first behavior of the occupant of the first mobile body, A second acquisition unit acquires second behavior information indicating the second behavior of the second moving body around the first moving body within a predetermined period from the occurrence of the first behavior, A first output unit outputs control information for controlling the movement of the first mobile body based on the first behavior information and the second behavior information, It is an output device equipped with [a specific feature / feature].

[0010] The second invention is, A first acquisition step involves acquiring first behavior information that indicates the first behavior of the occupant of the first mobile body, A second acquisition step involves acquiring second behavior information indicating the second behavior of the second moving body around the first moving body within a predetermined period from the occurrence of the first behavior, A first output step that outputs control information for controlling the movement of the first mobile body based on the first behavior information and the second behavior information, This is an output method that includes [something].

[0011] The third invention is, A computer program for realizing an output device, Computers, A first acquisition means for acquiring first behavior information indicating the first behavior of a occupant of a first mobile body, A second acquisition means for acquiring second behavior information indicating the second behavior of the second moving body around the first moving body within a predetermined period from the occurrence of the first behavior, and A first output means that outputs control information for controlling the movement of the first mobile body based on the first behavior information and the second behavior information, It is a computer program that functions as such.

Brief Description of Drawings

[0012] [Figure 1] It is a block diagram illustrating the functional configuration of the output device according to the embodiment. [Figure 2] It is a flowchart illustrating the output method according to the embodiment. [Figure 3] It is a block diagram illustrating the functional configuration of the output device according to Example 1. [Figure 4] It is a diagram illustrating the hardware configuration of the output device. [Figure 5] It is a flowchart illustrating the processing performed by the output device according to Example 1. [Figure 6] It is a flowchart illustrating the processing performed by the output device according to Example 2. [Figure 7] It is a block diagram illustrating the functional configuration of the output device according to Example 5. [Figure 8] It is a block diagram illustrating the functional configuration of the output device 10 according to Example 6. [Figure 9] It is a flowchart illustrating the processing performed by the output device 10 according to Example 6.

Modes for Carrying Out the Invention

[0013] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same components are denoted by the same reference numerals, and the description will be omitted as appropriate.

[0014] In the following description, the first acquisition unit 110, the second acquisition unit 120, the first detection unit 112, the second detection unit 122, the third acquisition unit 130, the determination unit 140, the first output unit 150, the second output unit 152, the first input unit 160, the second input unit 162, and the storage unit 170 of the output device 10 do not represent hardware units but functional unit blocks. The first acquisition unit 110, the second acquisition unit 120, the first detection unit 112, the second detection unit 122, the third acquisition unit 130, the determination unit 140, the second output unit 152, the first output unit 150, the second input unit 162, the first input unit 160, and the storage unit 170 of the output device 10 are realized by any combination of hardware and software centered around the CPU, memory, program loaded into the memory, storage media such as a hard disk storing the program, and network connection interface of any computer. And there are various modification examples for the realization method and device.

[0015] FIG. 1 is a block diagram illustrating the functional configuration of the output device 10 according to the embodiment. The output device 10 according to the present embodiment includes a first acquisition unit 110, a second acquisition unit 120, and a first output unit 150. The first acquisition unit 110 acquires first behavior information indicating the first behavior of the passenger of the first moving body. The second acquisition unit 120 acquires second behavior information indicating the second behavior of the second moving body around the first moving body within a predetermined period T1 from the occurrence of the first behavior. The first output unit 150 outputs control information for controlling the travel of the first moving body based on the first behavior information and the second behavior information. This will be described in detail below.

[0016] In a moving body that travels using partial or fully automatic driving, the surrounding situation is judged by the control device of the automatic driving, and the operation of the moving body is controlled based on the judgment result. However, when it is difficult to predict the next behavior of other moving bodies existing around, it may not be possible to travel smoothly with automatic driving.

[0017] On the other hand, a person riding in an autonomously driven vehicle can communicate with, for example, the occupants of surrounding vehicles or pedestrians. According to this embodiment, the surrounding situation and the next actions of other vehicles are estimated from the behavior of the vehicle's occupants and the behavior of the occupants of surrounding vehicles and pedestrians, thereby maintaining smooth autonomous driving.

[0018] In this embodiment, the first mobile body is a vehicle such as an automobile, a motorized bicycle, or a light vehicle. The output device 10 is mounted on the first mobile body. The first mobile body is driven at least partially automatically by control information output from the first output unit 150. The first mobile body has at least one passenger.

[0019] The first behavior is the behavior of the occupant of the first mobile body, and is a gesture or vocalization directed at the second mobile body. Specifically, the first behavior includes actions such as extending the palm upward, turning the palm towards the second mobile body, bowing, communicating with voice, flashing the headlights, hazard lights, or brake lights of the first mobile body one or more times, honking the horn of the first mobile body one or more times, or decelerating the first mobile body. The first behavior information is an ID or the like that indicates which behavior the first behavior is. The first behavior may or may not be directed towards the second mobile body. Preferably, the first behavior is one that can be recognized directly or indirectly by the second mobile body.

[0020] The second mobile object is a mobile object located in the vicinity of the first mobile object. The vicinity of the first mobile object is, for example, the range in which the behavior of the second mobile object can be recognized and detected from the first mobile object. The vicinity of the first mobile object is not particularly limited, but for example, it is the range in which the distance from the first mobile object is 50m or less.

[0021] The second moving object is, for example, a pedestrian or a cyclist. In this case, the second behavior could be a bow, a stop, or a raised hand. The second behavior information is an ID or similar that indicates what the second behavior is. The second behavior may or may not be directed towards the first moving object. Preferably, the second behavior is one that can be recognized directly or indirectly by the first moving object.

[0022] The methods for generating and acquiring the first and second behavior information will be explained in detail in the examples described later.

[0023] For example, if a pedestrian on one side of a crosswalk is the second mobile object, the occupant of the first mobile object raises their palm upwards at face level (first behavior) to encourage the pedestrian to cross the crosswalk. The pedestrian then bows (second behavior) to indicate their willingness to accept the offer from the occupant of the first mobile object. Based on the first behavior information and the second behavior information, which represent these behaviors respectively, the first output unit 150 outputs control information to stop the first mobile object.

[0024] Furthermore, for example, if a pedestrian on one side of a crosswalk is the second mobile object, the occupant of the first mobile object raises their palm upwards at face level (first behavior) to encourage the pedestrian to cross the crosswalk. The pedestrian then shakes their head (second behavior) to indicate their refusal of the offer from the occupant of the first mobile object. Based on the first behavior information and second behavior information that represent these behaviors, the first output unit 150 outputs control information to drive the first mobile object. Note that the second behavior at this time may be a behavior in which the pedestrian waves their hand vertically from side to side, a behavior in which they move their hand horizontally in the direction of the vehicle's movement with their palm upwards, or a behavior in which they look towards the occupant but do not react.

[0025] The second moving object may be an oncoming vehicle of the first moving object. In that case, the second behavior may include the second moving object flashing its headlights, hazard lights, or brake lights once or more, honking its horn once or more, slowing down, or moving to the side of the road. Alternatively, the second behavior may be the behavior of the occupants of the oncoming vehicle, which is the second moving object. The behavior of the occupants of the oncoming vehicle may include bowing or raising one hand. The second behavior information is an ID or similar that indicates which behavior the second behavior is.

[0026] For example, if the oncoming vehicle on a narrow road is the second moving object, the occupant of the first moving object points their palm towards the second moving object to ask it to wait (first behavior). The second moving object then moves to the side of the road to yield the right of way (second behavior). Based on the first and second behavior information, which represent these behaviors, control information is output to allow the first moving object to pass the second moving object.

[0027] Other examples of situations in which the first output unit 150 outputs a control signal based on the first behavior information and the second behavior information include situations in which the first moving object or the second moving object is attempting to turn right or merge onto a road.

[0028] The second behavior occurs within a predetermined period T1 from the occurrence of the first behavior. The predetermined period T1 is predetermined based on the time it is assumed that the second moving object will need to react to the first behavior. The predetermined period T1 is, for example, within 5 seconds or within 3 seconds.

[0029] Figure 2 is a flowchart illustrating an output method according to an embodiment. This method includes a first acquisition step (S10), a second acquisition step (S20), and a first output step (S50). In the first acquisition step (S10), first behavior information indicating the first behavior of a passenger on the first mobile body is acquired. In the second acquisition step (S20), second behavior information indicating the second behavior of a second mobile body around the first mobile body within a predetermined period T1 from the occurrence of the first behavior is acquired. In the first output step (S50), control information for controlling the movement of the first mobile body is output based on the first behavior information and the second behavior information.

[0030] The output method according to this embodiment is implemented, for example, by the output device 10 described above.

[0031] As described above, according to this embodiment, the first output unit 150 outputs control information for controlling the movement of the first mobile body based on the first behavior information and the second behavior information. Therefore, smooth communication with other mobile bodies is achieved, and smooth automated driving is realized.

[0032] (Example 1) Figure 3 is a block diagram illustrating the functional configuration of the output device 10 according to Embodiment 1. The output device 10 according to this embodiment has the configuration of the output device 10 according to the embodiment. The output device 10 according to this embodiment further comprises a first input unit 160, a storage unit 170, a first detection unit 112, a second detection unit 122, and a determination unit 140. The functional configuration of the output device 10 will be described in detail later.

[0033] Figure 4 illustrates the hardware configuration of the output device 10. In this figure, the output device 10 is implemented using an integrated circuit 40. The integrated circuit 40 is, for example, a System On Chip (SoC).

[0034] The integrated circuit 40 includes a bus 402, a processor 404, a memory 406, a storage device 408, an input / output interface 410, and a network interface 412. The bus 402 is a data transmission path for the processor 404, memory 406, storage device 408, input / output interface 410, and network interface 412 to send and receive data to and from each other. However, the method of connecting the processor 404 and the others is not limited to bus connection. The processor 404 is an arithmetic processing unit implemented using a microprocessor or the like. The memory 406 is a memory implemented using RAM (Random Access Memory) or the like. The storage device 408 is a storage device implemented using ROM (Read Only Memory) or flash memory or the like.

[0035] The input / output interface 410 is an interface for connecting the integrated circuit 40 to peripheral devices. In this figure, the input / output interface 410 is connected to the first camera 420, the second camera 421, the monitor 422, the input panel 423, the microphone 424, and the speaker 425.

[0036] The network interface 412 is an interface for connecting the integrated circuit 40 to a communication network. This communication network is, for example, a CAN (Controller Area Network) communication network. The method by which the network interface 412 connects to the communication network may be wireless or wired.

[0037] The storage device 408 stores program modules for realizing the functions of the first input unit 160, the first detection unit 112, the second detection unit 122, the first acquisition unit 110, the second acquisition unit 120, the determination unit 140, and the first output unit 150. The processor 404 reads these program modules into memory 406 and executes them to realize the functions of the first input unit 160, the first detection unit 112, the second detection unit 122, the first acquisition unit 110, the second acquisition unit 120, the determination unit 140, and the first output unit 150.

[0038] The hardware configuration of the integrated circuit 40 is not limited to the configuration shown in this figure. For example, the program module may be stored in the memory 406. In this case, the integrated circuit 40 does not need to have a storage device 408.

[0039] Referring to Figure 3, the functional configuration of the output device 10 will be described in detail below. In the output device 10 according to this embodiment, the first acquisition unit 110 is capable of receiving input from a passenger to the first input unit 160. One example of the first behavior is input to the first input unit 160.

[0040] In the example in Figure 4, the input panel 423 and microphone 424 function as the first input unit 160. The first input unit 160 may also be a mechanical steering wheel, switch, pedal, etc. The first action may be, for example, an input operation on the touch panel or voice input, and the input from the passenger may be, for example, an operation to flash the headlights, hazard lights, or brake lights of the first mobile vehicle one or more times, an operation to sound the horn of the first mobile vehicle one or more times, or an operation to decelerate the first mobile vehicle. At least one of the passengers of the first mobile vehicle is in a position to access the first input unit 160 as the driver. For example, the passenger as the driver is in an operating position where the touch panel, microphone, etc., which constitute the first input unit 160 are located. The passenger as the driver is also holding the operating terminal on which the touch panel, microphone, etc., which constitute the first input unit 160 are located. When a passenger performs one of several input operations designated as candidates for a first behavior on the first input unit 160, the first acquisition unit 110 acquires information indicating the content of that operation as first behavior information. The first input unit 160 may also be able to accept input from the passenger indicating that a first behavior has been performed. For example, a screen display or voice output such as "Did you prompt the pedestrian to cross?" may be displayed, prompting the passenger to perform touch input or voice input.

[0041] Since the first behavior is an input to the first input unit 160, the first acquisition unit 110 can directly acquire the first behavior information, enabling high-precision recognition of the first behavior.

[0042] As described above, the output device 10 according to this embodiment includes a first detection unit 112. The first detection unit 112 detects the behavior of the first moving object. The first acquisition unit 110 acquires first behavior information from the first detection unit 112. The first detection unit 112 acquires an image that includes at least a portion of the occupant of the first moving object, detects the first behavior using the image, and generates first behavior information. The first detection unit 112 acquires an image captured by the first camera 420. The first camera 420 is provided to capture at least a portion of the occupant as the driver of the first moving object. Alternatively, instead of the first camera 420, an image may be acquired using a ToF (Time of Flight) camera or LiDAR (Laser Imaging Detection and Ranging, Laser Illuminated Detection and Ranging or LiDAR: Light Detection and Ranging).

[0043] The first detection unit 112 detects the first behavior of the passenger by, for example, extracting and analyzing feature points in the acquired image. Information indicating the pattern of feature points for the first behavior is pre-stored in the storage unit 170 for each type of behavior, and the first detection unit 112 can read it and use it for detection. When the first detection unit 112 detects the first behavior, it generates information indicating the content of that behavior as first behavior information. Once the first behavior information is generated by the first detection unit 112, the first acquisition unit 110 acquires that first behavior information. The first detection unit 112 may also detect the first behavior based on multiple images, such as a video.

[0044] Specifically, feature information representing the characteristics of a person's body is stored in the memory unit 170 beforehand. The first detection unit 112 uses the feature information from the image to identify the location of parts of the human body in the image. The occupant's posture is estimated from this identified location and the human body model information, and the behavior is estimated from the change in posture over time.

[0045] Furthermore, the first detection unit 112 may utilize deep learning. In this case, video data of the behavior and information indicating what kind of behavior the video data represents are stored in the storage unit 170 in advance. The first detection unit 112 estimates the behavior of the passenger based on the passenger's video data and the video data stored in advance.

[0046] Note that the output device 10 does not necessarily have to include either the first input unit 160 or the first detection unit 112. The first acquisition unit 110 only needs to be able to acquire the first behavior information from at least one of the first input unit 160 or the first detection unit 112.

[0047] As described above, the output device 10 according to this embodiment includes a second detection unit 122. The second detection unit 122 detects the behavior of the second moving object. The second acquisition unit 120 acquires second behavior information from the second detection unit 122. The second detection unit 122 acquires an image that includes at least a part of the second moving object, detects the second behavior using the image, and generates second behavior information.

[0048] The second detection unit 122 acquires images captured by the second camera 421. The second camera 421 is configured to capture images around the first moving object. The second detection unit 122 extracts and analyzes feature points in the images acquired from the second camera 421, for example, to extract a second moving object. For example, a pedestrian at the edge of a crosswalk or an oncoming vehicle located diagonally in front of the first moving object may be extracted as a second moving object.

[0049] The second detection unit 122 detects the second behavior of the second moving object by, for example, extracting and analyzing feature points in the acquired image. Information indicating the pattern of feature points for the second behavior is pre-stored in the storage unit 170 for each type of behavior, and the second detection unit 122 can read it and use it for detection. When the second detection unit 122 detects the second behavior, it generates information indicating the content of that behavior as first behavior information. Once the second behavior information is generated by the second detection unit 122, the second acquisition unit 120 acquires that second behavior information.

[0050] The second detection unit 122 may extract a second moving object and detect a second behavior based on multiple images, such as a video. Furthermore, the second detection unit 122 may detect the second behavior using a ToF camera or LiDAR, similar to the first detection unit 112, or it may utilize deep learning.

[0051] Here, the second acquisition unit 120 acquires second behavior information indicating the second behavior within a predetermined period T1 from the occurrence of the first behavior. The second detection unit 122 may, for example, detect only the behavior within the predetermined period T1 after the first detection unit 112 detects the first behavior as the second behavior.

[0052] Furthermore, if the second detection unit 122 detects the second behavior regardless of the timing, the second acquisition unit 120 does not need to acquire second behavior information indicating the second behavior if the second behavior occurs more than a predetermined period T1 after detecting the first behavior. In this case, the second acquisition unit 120 acquires both the second behavior information and information indicating the time when the second behavior indicated by the second behavior information occurred. In addition, the second acquisition unit 120 acquires information indicating the time when the first behavior occurred. Note that if the time when the first behavior occurred and the time when the first acquisition unit 110 acquires the first behavior information are approximately the same, and the time when the second behavior occurred and the time when the second acquisition unit 120 attempts to acquire the second behavior information are approximately the same, the second acquisition unit 120 may compare the time when the first acquisition unit 110 acquired the first behavior information with the time when the second acquisition unit 120 attempts to acquire the second behavior information to determine whether or not the second behavior occurred within the predetermined period T1 after detecting the first behavior.

[0053] Furthermore, the first acquisition unit 110 and the second acquisition unit 120 may acquire the first behavior information and the second behavior information, respectively, regardless of timing. This case will be described in detail later.

[0054] As described above, the output device 10 includes a determination unit 140. The determination unit 140 determines whether the first output unit 150 should output control information based on the first behavior information and the second behavior information. If the determination unit 140 determines that it should output control information, the first output unit 150 outputs control information based on the first behavior information and the second behavior information.

[0055] Based on the determination result of the determination unit 140, the output device 10 switches between a standby state and a non-standby state. That is, if the determination unit 140 determines that the first output unit 150 should output control information based on the first behavior information and the second behavior information, the output device 10 enters a standby state. If the determination unit 140 determines that the first output unit 150 does not need to output control information based on the first behavior information and the second behavior information, the output device 10 enters a non-standby state. When in the standby state, the first output unit 150 outputs control information based on the first behavior information and the second behavior information. On the other hand, when in the non-standby state, the first output unit 150 does not output control information based on the first behavior information and the second behavior information.

[0056] For example, in a non-standby state, the first detection unit 112 and the second detection unit 122 do not need to detect behavior. Also, in a non-standby state, the first acquisition unit 110 and the second acquisition unit 120 do not need to acquire first behavior information and second behavior information, respectively. Furthermore, the first output unit 150 does not need to generate control information based on the first behavior information and second behavior information. Note that in a non-standby state, the first output unit 150 may generate and output other control information that is not based on the first behavior information and second behavior information.

[0057] The first output unit 150 includes a determination unit 140 that determines whether or not to output control information based on the first behavior information and the second behavior information. This ensures that control information is generated based on the first behavior information and the second behavior information only when necessary, thereby reducing the processing load on the output device 10. For example, if detection is not performed by the first detection unit 112 and the second detection unit 122 in a non-standby state, the first detection unit 112 and the second detection unit 122 do not need to continue processing images.

[0058] In this embodiment, the determination unit 140 determines that when the speed of the first moving body falls below a predetermined speed V, the first output unit 150 should output control information based on the first behavior information and the second behavior information. The speed V is not particularly limited, but for example, it is between 2 km / h and 20 km / h. The reduction in the speed of the first moving body may be due to deceleration by automatic driving, or deceleration due to instructions or operations by the driver. For example, if the control device for automatic driving cannot determine the next behavior of the second moving body based solely on the behavior of the second moving body, deceleration may occur. Also, deceleration may occur if the occupant of the first moving body spots the second moving body and thinks that communication is necessary, or if the surrounding environment requires caution, such as a narrow road. Information indicating the predetermined speed V is stored in the storage unit 170, and the determination unit 140 can read it and use it for determination.

[0059] Furthermore, the determination unit 140 may determine that the first output unit 150 should output control information based on the first behavior information and the second behavior information when the acceleration of the first moving body falls below a predetermined acceleration A. By defining a negative acceleration as acceleration A, the determination unit 140 can put the output device 10 into standby mode when the first moving body decelerates.

[0060] When the determination unit 140 determines that the first output unit 150 should output control information based on the first behavior information and the second behavior information, and the output device 10 enters a standby state, the fact that it is in standby mode may be indicated to the occupant of the first mobile body by a display on a monitor 422 or by sound from the speaker 425. Upon receiving this indication, the occupant can perform the first action.

[0061] Furthermore, when the output device 10 is in standby mode, the first output unit 150 may output control information to slow down or stop the first moving object. Also, when the output device 10 is in standby mode and the presence of a second moving object is detected, the first output unit 150 may output control information to slow down or stop the first moving object. The second moving object can be detected by the second detection unit 122 in the same manner as described above.

[0062] Furthermore, the output device 10 may enter a standby state when the determination unit 140 determines that the first output unit 150 should output control information based on the first behavior information and the second behavior information, and the second detection unit 122 detects the presence of the second moving object. In this case, the second detection unit 122 may perform the process of attempting to detect the second moving object while the output device 10 is running, or it may start after the first output unit 150 determines that it should output control information based on the first behavior information and the second behavior information.

[0063] Furthermore, if the first mobile body is equipped with an automatic driving system, when the first mobile body receives auxiliary request information from the automatic driving system, the determination unit 140 may determine that the first output unit 150 should output control information based on the first behavior information and the second behavior information, and the output device 10 may enter a standby state. The automatic driving system outputs auxiliary request information when it is difficult to determine what action the first mobile body should perform in automatic driving control.

[0064] The determination unit 140 makes a determination at predetermined time intervals T3. The time interval T3 is not particularly limited, but for example, it is between 1 second and 1 minute. When the speed of the first moving body exceeds a predetermined speed V, the determination unit 140 determines that the first output unit 150 does not need to output control information based on the first behavior information and the second behavior information, and puts the output device 10 into a non-standby state.

[0065] The first output unit 150 outputs control information based on the first behavior information and the second behavior information. In the standby state, when the first acquisition unit 110 acquires the first behavior information and the second acquisition unit 120 acquires the second behavior information, the first output unit 150 generates and outputs control information based on the combination of the acquired first behavior information and the second behavior information. The relationship between the combination of the first behavior information and the second behavior information and the control information to be generated is predetermined based on the expected communication pattern between the first mobile body and the second mobile body. Reference information showing the relationship between the combination of the first behavior information and the second behavior information and the control information to be generated is stored in the storage unit 170 in advance, and the first output unit 150 can read the reference information from the storage unit 170 and use it to generate the control information.

[0066] If the first acquisition unit 110 and the second acquisition unit 120 acquire the first behavior information and the second behavior information respectively, regardless of timing, the first output unit 150 may determine whether or not the second behavior occurred within a predetermined period T1 after detecting the first behavior. The first output unit 150 can make this determination using at least one of the times when the first behavior information and the second behavior information were generated, or the times when the first behavior information and the second behavior information were acquired by the first acquisition unit 110 and the second acquisition unit 120, respectively. The first output unit 150 generates and outputs control information based on the first behavior information and the second behavior information only if the second behavior information indicating the second behavior within a predetermined period T1 from the occurrence of the first behavior is acquired by the second acquisition unit 120.

[0067] The first mobile unit is controlled by control information output from the first output unit 150. The control information includes, for example, information for stopping the first mobile unit, information for starting the first mobile unit, and so on.

[0068] Figure 5 is a flowchart illustrating the processing performed by the output device 10 according to Embodiment 1. Below, we will describe an example in which the first output unit 150 determines whether a second behavior occurred within a predetermined period T1 after detecting a first behavior, but the processing performed by the output device 10 is not limited to this example. The output device 10 is switched between a standby state and a non-standby state by the determination unit 140. First, in step S100, the first output unit 150 checks whether the output device 10 is in a standby state. The first output unit 150 can, for example, obtain information indicating the state at that time from the determination unit 140. When the output device 10 is not in a standby state, i.e., in a non-standby state, the first output unit 150 performs step S100 again after a predetermined time T4 has elapsed. On the other hand, when the output device 10 is in a standby state, the processing moves to step S200. The time T4 is not particularly limited, but for example, it is between 1 second and 1 minute.

[0069] In step S200, the first output unit 150 checks whether the first behavior information is acquired by the first acquisition unit 110. If the first behavior information is not acquired by the first acquisition unit 110 during the predetermined waiting time T5, the process returns to step S100. On the other hand, if the first behavior information is acquired by the first acquisition unit 110 during the predetermined waiting time T5, the process moves to step S300. The waiting time T5 is not particularly limited, but for example, it is between 1 second and 1 minute.

[0070] In step S300, the first output unit 150 checks whether the second behavior information is acquired by the second acquisition unit 120. If the second behavior information is not acquired by the second acquisition unit 120 within a predetermined period T1 after the first behavior information is acquired, the process returns to step S100. On the other hand, if the second behavior information is acquired by the second acquisition unit 120 within a predetermined period T1 after the first behavior information is acquired, the process moves to step S500. In this example, it is assumed that the time of occurrence of the first behavior and the time when the first acquisition unit 110 acquires the first behavior information are approximately the same, and that the time of occurrence of the second behavior and the time when the second acquisition unit 120 acquires the second behavior information are approximately the same.

[0071] In step S500, the first output unit 150 generates control information using the acquired first behavior information and second behavior information, along with the reference information, and outputs the generated control information. Then, the process returns to step S100.

[0072] As described above, according to this embodiment, similar to the embodiment, the first output unit 150 outputs control information for controlling the movement of the first mobile body based on the first behavior information and the second behavior information. Therefore, smooth communication with other mobile bodies is achieved, and smooth automated driving is realized.

[0073] Furthermore, according to this embodiment, the first output unit 150 outputs control information when the determination unit 140 determines that it should output. Therefore, control information is generated based on the first behavior information and the second behavior information only when necessary, reducing the processing load on the output device 10.

[0074] (Example 2) Figure 6 is a flowchart illustrating the processing performed by the output device 10 according to Embodiment 2. The output device 10 according to this embodiment is the same as the output device 10 according to Embodiment 1, except for the points described below. In the output device 10 according to this embodiment, the first acquisition unit 110 further acquires third behavior information indicating the third behavior of the passenger within a predetermined period T2 from the occurrence of the second behavior. The first output unit 150 outputs control information based on the third behavior information. This will be explained in detail below.

[0075] The third behavior is the behavior of the occupant of the first mobile body in response to the second behavior performed by the second mobile body. The occupant of the first mobile body in response to the second behavior may, for example, show gratitude to the second mobile body or take action toward the next movement of the first mobile body in response to the second behavior. The third behavior may include, for example, the occupant's actions such as raising one hand or bowing, actions related to the occupant's gaze such as checking the surroundings for safety or keeping an eye on pedestrians, or actions such as flashing the headlights, hazard lights, or brake lights of the first mobile body one or more times, or actions such as sounding the horn of the first mobile body one or more times.

[0076] In the output device 10 according to this embodiment, the first output unit 150 outputs control information based on third behavior information in addition to first behavior information and second behavior information. Therefore, the results of communication between the occupant of the first mobile body and the second mobile body can be grasped with greater accuracy, and the first mobile body can be controlled accordingly.

[0077] In the output device 10 according to this embodiment, the first acquisition unit 110 acquires the third behavior in the same way as the first behavior. The third behavior information is an ID or the like that indicates what kind of behavior the third behavior is. The third behavior may or may not be directed towards the second moving object. Preferably, the third behavior is a behavior that can be recognized directly or indirectly by the second moving object.

[0078] Furthermore, the third behavior may be an input to the first input unit 160, similar to the first behavior. For example, when a passenger performs one of the multiple input operations designated as candidates for the third behavior to the first input unit 160, the first acquisition unit 110 acquires information indicating the content of that operation as third behavior information.

[0079] Furthermore, the third behavior may also be a behavior detected by the first detection unit 112, similar to the first behavior. The first detection unit 112 can detect the third behavior in the same manner as detecting the first behavior. Information indicating the pattern of feature points for the third behavior is pre-stored in the storage unit 170 for each type of behavior, and the first detection unit 112 can read it and use it for detection. When the first detection unit 112 detects the third behavior, it generates information indicating the content of that behavior as third behavior information. Once the first detection unit 112 generates the third behavior information, the first acquisition unit 110 acquires that third behavior information.

[0080] Here, the first acquisition unit 110 acquires third behavior information indicating the third behavior within a predetermined period T2 from the occurrence of the second behavior. The predetermined period T1 between the occurrence of the first behavior and the occurrence of the second behavior and the predetermined period T2 between the occurrence of the second behavior and the occurrence of the third behavior may be the same or different. The predetermined period T2 is predetermined based on the time it is assumed that it will take for the occupant of the first mobile body to react to the second behavior. The predetermined period T2 may be, for example, within 5 seconds or within 3 seconds.

[0081] The first detection unit 112 may, for example, detect only the behavior within a predetermined period T2 after the second detection unit 122 has detected the second behavior as the third behavior.

[0082] Furthermore, if the first detection unit 112 detects the first behavior regardless of the timing, the first acquisition unit 110 does not need to acquire third behavior information indicating the third behavior if the third behavior occurs more than a predetermined period T2 after detecting the second behavior. In this case, the second acquisition unit 120 acquires both the third behavior information and information indicating the time when the third behavior indicated by the third behavior information occurred. In addition, the second acquisition unit 120 acquires information indicating the time of occurrence of the third behavior. Note that if the time of occurrence of the second behavior and the time when the second acquisition unit 120 acquires the second behavior information are approximately the same, and the time of occurrence of the third behavior and the time when the first acquisition unit 110 attempts to acquire the third behavior information are approximately the same, the first acquisition unit 110 may compare the time when the second acquisition unit 120 acquires the second behavior information with the time when the first acquisition unit 110 attempts to acquire the third behavior information to determine whether or not the third behavior occurred within the predetermined period T2 after detecting the second behavior.

[0083] Furthermore, the second acquisition unit 120 and the first acquisition unit 110 may acquire the second behavior information and the third behavior information, respectively, regardless of timing. In this case, the first output unit 150 may determine whether or not the third behavior occurred within a predetermined period T2 after detecting the second behavior. The first output unit 150 can make this determination using at least one of the times when the second behavior information and the third behavior information were generated, or the times when the second behavior information and the third behavior information were acquired by the second acquisition unit 120 and the first acquisition unit 110, respectively. The first output unit 150 generates and outputs control information based on the first behavior information, the second behavior information, and the third behavior information only if the first acquisition unit 110 has acquired third behavior information indicating the third behavior within a predetermined period T2 from the occurrence of the second behavior.

[0084] Referring to Figure 6, the output method according to this embodiment will be described below. In this embodiment, steps S100 to S300 are performed in the same manner as in Embodiment 1. In this embodiment, in step S300, if the second behavior information is acquired by the second acquisition unit 120 within a predetermined period T1 after the first behavior information is acquired, the process moves to step S400.

[0085] In step S400, the first output unit 150 checks whether the third behavior information is acquired by the first acquisition unit 110. If the third behavior information is not acquired by the first acquisition unit 110 within a predetermined period T2 after the second behavior information is acquired, the process returns to step S100. On the other hand, if the third behavior information is acquired by the first acquisition unit 110 within a predetermined period T2 after the second behavior information is acquired, the process moves to step S500. In this example, it is assumed that the time of occurrence of the second behavior and the time when the second acquisition unit 120 acquires the second behavior information are approximately the same, and that the time of occurrence of the third behavior and the time when the first acquisition unit 110 acquires the third behavior information are approximately the same.

[0086] In step S500, the first output unit 150 generates control information using the acquired first behavior information, second behavior information, and third behavior information, along with the reference information, and outputs the generated control information. Then, the process returns to step S100.

[0087] In this embodiment, the reference information is information that shows the relationship between the combination of the first behavior information, the second behavior information, and the third behavior information, and the control information to be generated.

[0088] As described above, according to this embodiment, similar to the embodiment, the first output unit 150 outputs control information for controlling the movement of the first mobile body based on the first behavior information and the second behavior information. Therefore, smooth communication with other mobile bodies is achieved, and smooth automated driving is realized.

[0089] Furthermore, according to this embodiment, the first output unit 150 outputs control information based on the third behavior information. Therefore, the first mobile body is controlled by more accurately understanding the results of communication between the occupant of the first mobile body and the second mobile body.

[0090] (Example 3) The output device 10 according to Embodiment 3 is the same as the output device 10 according to at least one of Embodiment 1 or Embodiment 2, except that the determination unit 140 determines that the first output unit 150 should output control information based on the first behavior information and the second behavior information when the first moving body is located within a predetermined range from a pedestrian crossing or intersection. A detailed explanation follows below.

[0091] Near pedestrian crossings and intersections, communication between the passenger of the first mobile vehicle and the second mobile vehicle is highly likely to be necessary. Therefore, when the first mobile vehicle is located within a predetermined range from a pedestrian crossing or intersection, the determination unit 140 puts the output device 10 into standby mode, enabling smooth automated driving near pedestrian crossings and intersections.

[0092] The determination unit 140 in this embodiment acquires location information of the first mobile object and map information of at least the area where the first mobile object is located. The first mobile object is equipped with, for example, a GPS (Global Positioning System) receiver, and the determination unit 140 acquires location information indicating the position of the first mobile object from the GPS device mounted on the first mobile object. Map information is also stored in advance in the storage unit 170, and the determination unit 140 can read it and use it. Alternatively, the determination unit 140 may acquire map information from an external server via a communication network. The map information includes information indicating the location of pedestrian crossings and information indicating the location of intersections.

[0093] The determination unit 140 uses the acquired location information and map information to determine whether the first moving object is located within a predetermined range from a pedestrian crossing or intersection. The predetermined range is defined based on the range in which communication between the occupant of the first moving object and the second moving object is necessary. The predetermined range indicates, for example, that the distance between the first moving object and the pedestrian crossing or intersection is within L [m]. L is, for example, between 2 and 100. The determination unit 140 may also determine that the first moving object is located within the predetermined range from a pedestrian crossing or intersection only if the pedestrian crossing or intersection is located in front of the direction of travel of the first moving object.

[0094] The determination unit 140 determines that the first output unit 150 should output control information based on the first behavior information and the second behavior information when the first moving object is located within a predetermined range from the pedestrian crossing or intersection. In other words, it puts the output device 10 into standby mode. On the other hand, when the first moving object is not located within a predetermined range from the pedestrian crossing or intersection, the determination unit 140 determines that there is no need for the first output unit 150 to output control information based on the first behavior information and the second behavior information. In other words, it puts the output device 10 into non-standby mode.

[0095] Furthermore, in this embodiment, the determination unit 140 may prepare reference information for each fact that serves as the basis for the first output unit 150 to determine that it should output control information based on the first behavior information and the second behavior information. In this case, when the determination unit 140 determines that the first output unit 150 should output control information based on the first behavior information and the second behavior information, it generates information indicating the fact that serves as the basis for that determination (for example, "being located within a predetermined range from a pedestrian crossing" or "the speed of the first moving object becoming below a predetermined speed").

[0096] In step S500, the first output unit 150 obtains information from the determination unit 140 indicating the facts that form the basis of the determination, and reads reference information from the storage unit 170 corresponding to the facts indicated by that information. The first output unit 150 then generates control information using the read reference information and outputs it.

[0097] The content of communication between the occupant of the first mobile body and the second mobile body may vary depending on the situation in which the communication takes place. Therefore, the first output unit 150 outputs control information using reference information prepared for each factual basis, thereby enabling smooth automated driving in each situation.

[0098] As described above, according to this embodiment, similar to the embodiment, the first output unit 150 outputs control information for controlling the movement of the first mobile body based on the first behavior information and the second behavior information. Therefore, smooth communication with other mobile bodies is achieved, and smooth automated driving is realized.

[0099] Furthermore, according to this embodiment, the determination unit 140 determines that when the first moving object is located within a predetermined range from a pedestrian crossing or intersection, the first output unit 150 should output control information based on the first behavior information and the second behavior information. Therefore, smooth automated driving is performed near pedestrian crossings and intersections.

[0100] (Example 4) In the output device 10 according to Embodiment 4, the determination unit 140 is the same as the output device 10 according to at least one of Embodiments 1 to 3, except that when the width of the path on which the first moving body travels satisfies predetermined conditions, the first output unit 150 determines that control information should be output based on the first behavior information and the second behavior information. This will be explained in detail below.

[0101] In this embodiment, the first moving object is a vehicle, and the second moving object is an oncoming vehicle of the first moving object. When two vehicles pass each other on a narrow road, the two oncoming vehicles may communicate and yield the right of way. Therefore, when the width of the road on which the first moving object is traveling meets predetermined conditions, the determination unit 140 puts the output device 10 into standby mode, enabling smooth automatic driving even on narrow roads.

[0102] The determination unit 140 can make a determination using, for example, the road width included in the map information. Alternatively, the determination unit 140 may make a determination using the results of actual measurements of the circumference of the first moving object.

[0103] An example of the determination unit 140 making a determination using the road width indicated in the map information will be described. The determination unit 140 acquires map information and the position information of the first moving object, as in Embodiment 3. In this example, the map information includes information indicating the width of each road. The determination unit 140 then uses the acquired position information and map information to determine whether the width of the road on which the first moving object is traveling satisfies a predetermined condition. This condition indicates that the road is narrow enough that caution is required when two moving objects pass each other. Specifically, for example, if the width of the road on which the first moving object is traveling is W [m] or less, the determination unit 140 determines that the road width satisfies the predetermined condition. W is, for example, 7 or more and 10 or less.

[0104] The following describes the case in which the determination unit 140 makes a determination using the results of actual measurements of the area around the first moving body. In this example, the first moving body is equipped with a sensor that measures the distance between the side surface of the first moving body and a structure facing the side surface. The determination unit 140 then obtains the distance between the side surface of the first moving body and the structure from the sensor. The structure is, for example, a building, wall, or guardrail located on the side of a road. If the obtained distance is less than or equal to a predetermined reference distance, the determination unit 140 determines that the width of the road on which the first moving body travels satisfies predetermined conditions.

[0105] The determination unit 140 determines that the first output unit 150 should output control information based on the first behavior information and the second behavior information when the width of the path on which the first mobile body travels satisfies predetermined conditions. In other words, it puts the output device 10 into standby mode. On the other hand, when the width of the path on which the first mobile body travels does not satisfy predetermined conditions, the determination unit 140 determines that there is no need for the first output unit 150 to output control information based on the first behavior information and the second behavior information. In other words, it puts the output device 10 into non-standby mode.

[0106] In this embodiment as well, similar to Embodiment 3, reference information may be prepared for each fact that serves as the basis for the determination unit 140 to determine that the first output unit 150 should output control information based on the first behavior information and the second behavior information. The first output unit 150 may then generate and output control information using the reference information corresponding to the fact that serves as the basis for the determination.

[0107] As described above, according to this embodiment, similar to the embodiment, the first output unit 150 outputs control information for controlling the movement of the first mobile body based on the first behavior information and the second behavior information. Therefore, smooth communication with other mobile bodies is achieved, and smooth automated driving is realized.

[0108] Furthermore, according to this embodiment, the determination unit 140 determines that the first output unit 150 should output control information based on the first behavior information and the second behavior information when the width of the road on which the first moving body travels satisfies predetermined conditions. Therefore, smooth automatic driving can be performed even on narrow roads.

[0109] (Example 5) Figure 7 is a block diagram illustrating the functional configuration of the output device 10 according to Example 5. The output device 10 according to this example is the same as the output device 10 according to at least one of Examples 1 to 4, except for the points described below.

[0110] The output device 10 according to this embodiment further includes a third acquisition unit 130. The third acquisition unit 130 acquires information indicating a region requiring attention, which the first output unit 150 should output based on the first behavior information and the second behavior information. The determination unit 140 then determines that when the first moving object is located within the region requiring attention, the first output unit 150 should output control information based on the first behavior information and the second behavior information. This will be explained in detail below.

[0111] Places where communication between the passengers of the first vehicle and the second vehicle may be necessary, such as intersections, pedestrian crossings, narrow roads, and merging points at highway entrances and exits to shops, can be predicted to some extent in advance. By designating such places as areas requiring attention, the area in which autonomous driving can proceed smoothly can be expanded.

[0112] The attention zone is the area where control information should be output based on the first and second behavior information. Examples of attention zones include areas near intersections and pedestrian crossings, narrow roads, and areas near merging points at highways and store entrances / exits. Multiple areas where communication between the occupant of the first mobile body and the second mobile body is expected to occur can be pre-determined as attention zones. Information indicating the attention zone may be, for example, a list showing the range of each area in terms of latitude and longitude. Alternatively, the information indicating the attention zone may be linked to map information, showing areas on a map.

[0113] The third acquisition unit 130 may acquire information indicating areas requiring attention from an external server, for example, via a communication network. Alternatively, the third acquisition unit 130 may read and acquire information indicating areas requiring attention that is pre-stored in the storage unit 170.

[0114] The determination unit 140 acquires the position information of the first moving object in the same manner as in Embodiment 3. Then, using the position information and the information indicating the area requiring attention acquired by the third acquisition unit 130, it determines whether or not the first moving object is located within the area requiring attention. If the first moving object is located within the area requiring attention, the first output unit 150 determines that it should output control information based on the first behavior information and the second behavior information. In other words, it puts the output device 10 into a standby state. On the other hand, if the first moving object is not located within the area requiring attention, the first output unit 150 determines that it does not need to output control information based on the first behavior information and the second behavior information. In other words, it puts the output device 10 into a non-standby state.

[0115] In this embodiment as well, similar to Embodiment 3, reference information may be prepared for each fact that serves as the basis for the determination unit 140 to determine that the first output unit 150 should output control information based on the first behavior information and the second behavior information. The first output unit 150 may then generate and output control information using the reference information corresponding to the fact that serves as the basis for the determination.

[0116] As described above, according to this embodiment, similar to the embodiment, the first output unit 150 outputs control information for controlling the movement of the first mobile body based on the first behavior information and the second behavior information. Therefore, smooth communication with other mobile bodies is achieved, and smooth automated driving is realized.

[0117] Furthermore, according to this embodiment, the determination unit 140 determines that when the first moving object is located within the attention zone, the first output unit 150 should output control information based on the first behavior information and the second behavior information. Therefore, the area in which smooth automatic driving can be performed can be expanded.

[0118] (Example 6) Figure 8 is a block diagram illustrating the functional configuration of the output device 10 according to Embodiment 6. The output device 10 according to this embodiment is the same as the output device 10 according to at least one of Embodiments 1 to 5, except for the points described below.

[0119] The output device 10 according to this embodiment further includes a second output unit 152 and a third output unit 153 for providing notifications to the passenger. The output device 10 according to this embodiment further includes a second input unit 162 capable of receiving input from the passenger. In the example in Figure 4, the monitor 422 and speaker 425 function as the second output unit 152. The monitor 422 and speaker 425 also function as the third output unit 153. The input panel 423 and microphone 424 function as the second input unit 162. The outputs of the second output unit 152 and the third output unit 153 may be sound, screen display, AR (Augmented Reality) display, etc.

[0120] In this embodiment, the second output unit 152 outputs notification information to inform the passenger of the contents of the control information generated by the first output unit 150. The third output unit 153 outputs information to prompt the passenger of the first mobile body to take action. The second input unit 162 can also accept input to change the contents of the control information. If the passenger of the first mobile body has a problem with the next action indicated by the control information generated by the first output unit 150, they can input to the second input unit 162 to cancel that action. This allows for smooth mutual yielding between the first mobile body and the second mobile body. This will be explained in detail below.

[0121] In this embodiment, the first mobile vehicle is equipped with an automatic driving system. For example, when the first mobile vehicle approaches a crosswalk and detects the presence of a pedestrian (the second mobile vehicle) near the crosswalk, the first mobile vehicle stops before the stop line near the crosswalk to wait for the pedestrian to cross, based on the automatic driving system.

[0122] If the pedestrian does not begin to cross the crosswalk, the automated driving system outputs assistance request information. The determination unit 140 then puts the output device 10 into standby mode.

[0123] In this embodiment, when the output device 10 is in standby mode, the third output unit 153 outputs information to the occupant of the first mobile body prompting communication with the second mobile body. The information prompting communication will be referred to below as "request information." Request information is information that prompts the occupant of the first mobile body to take action. As output of request information, for example, an audio announcement or screen display such as "Pedestrian detected near the sidewalk. Could you please ask the pedestrian to cross?" may be provided. The third output unit 153 may also output information indicating the status of the first mobile body, such as "The vehicle will be stopped to wait for pedestrians to cross." Alternatively, the output of the third output unit 153 may be an AR display that emphasizes the second mobile body.

[0124] Based on the request information, the passenger of the first mobile body communicates with the second mobile body. The output device 10 then detects the first behavior and the second behavior. Here, the first acquisition unit 110 may acquire information indicating the passenger's behavior that occurred within a predetermined period from the output of the request information as first behavior information.

[0125] Next, the first output unit 150 generates control information for controlling the movement of the first mobile body based on the first behavior information and the second behavior information.

[0126] Figure 9 is a flowchart illustrating the processing performed by the output device 10 according to Example 6. The processing performed by the output device 10 is the same as the processing performed by the output device 10 according to at least one of Examples 1 to 5, except for the points described below.

[0127] In this embodiment, if the output device 10 is in standby mode in step S100, in step S150, the third output unit 153 presents request information to the passenger of the first mobile body. Then, communication takes place between the passenger of the first mobile body and the second mobile body, and steps S200 and S300 are performed in the same manner as in Embodiment 1. However, in step S200, the first acquisition unit 110 may acquire information indicating the passenger's behavior that occurred within a predetermined period from the output of the request information as first behavior information.

[0128] When the second behavior information is acquired in step S300 (Y in step S300), in step S501 the first output unit 150 generates control information in the same manner as in step S500 of Embodiment 1. Then, in step S502 the second output unit 152 outputs notification information to inform the passenger of the contents of the control information generated by the first output unit 150.

[0129] Next, in step S503, the passenger inputs to the second input unit 162 whether or not to cancel the next action indicated in the output notification information. If the second input unit 162 receives an input indicating cancellation (Y in step S503), the first output unit 150 stops outputting control information (step S505) and returns to step S100. In this case, the second output unit 152 may further notify the passenger to input an alternative action. Upon receiving this notification, the passenger inputs an alternative action to the second input unit 162, and the first output unit 150 outputs control information based on the input to the second input unit 162.

[0130] On the other hand, if an input indicating that cancellation is not to be performed is received in the second input unit 162 (N in step S503), the first output unit 150 outputs the generated control information (step S504). Then, the process returns to step S100.

[0131] In step S503, an example was described in which the passenger inputs to the second input unit 162 whether or not to cancel the next action indicated in the outputted notification information. However, the passenger may also input to the second input unit 162 whether or not to approve the next action indicated in the outputted notification information.

[0132] As described above, according to this embodiment, similar to the embodiment, the first output unit 150 outputs control information for controlling the movement of the first mobile body based on the first behavior information and the second behavior information. Therefore, smooth communication with other mobile bodies is achieved, and smooth automated driving is realized.

[0133] In addition, according to this embodiment, the second output unit 152 outputs notification information to inform the passenger of the contents of the control information generated by the first output unit 150. Therefore, the first mobile body is controlled by more accurately understanding the results of communication between the passenger of the first mobile body and the second mobile body.

[0134] The embodiments and examples described above with reference to the drawings are illustrative examples of the present invention, and various other configurations can be adopted. For example, although the sequence diagrams and flowcharts used in the above description show multiple processes in order, the execution order of the processes performed in each embodiment is not limited to the order in which they are described. In each embodiment, the order of the illustrated processes can be changed to the extent that it does not impede the content. Furthermore, the embodiments and examples described above can be combined to the extent that their content does not conflict.

[0135] Examples of reference formats are provided below. 1-1. A first acquisition unit that acquires first behavior information indicating the first behavior of the occupant of the first mobile body, A second acquisition unit acquires second behavior information indicating the second behavior of the second moving body around the first moving body within a predetermined period from the occurrence of the first behavior, A first output unit outputs control information for controlling the movement of the first mobile body based on the first behavior information and the second behavior information, An output device equipped with the following features. 1-2. In the output device described in 1-1, The first acquisition unit further acquires third behavior information indicating the third behavior of the passenger within a predetermined period from the occurrence of the second behavior, The first output unit is an output device that outputs the control information based on the third behavior information. 1-3. In the output device described in 1-1 or 1-2, The first output unit further comprises a determination unit that determines whether or not to output the control information based on the first behavior information and the second behavior information, The first output unit is an output device that outputs the control information when the determination unit determines that it should output the information. 1-4. In the output device described in 1-3, The determination unit is an output device that determines, based on the first behavior information and the second behavior information, that the first output unit should output the control information when the first moving body is located within a predetermined range from a pedestrian crossing or intersection. 1-5. In the output device described in 1-3 or 1-4, The determination unit is an output device that determines, when the speed of the first moving body falls below a predetermined speed, that the first output unit should output the control information based on the first behavior information and the second behavior information. 1-6. In any one of the output devices described in 1-3 to 1-5, The first output unit further comprises a third acquisition unit that acquires information indicating a region requiring attention, on which the control information should be output based on the first behavior information and the second behavior information. The determination unit is an output device that determines, based on the first behavior information and the second behavior information, that the first output unit should output the control information when the first moving body is located within the area requiring attention. 1-7. In any one of the output devices described in 1-3 to 1-6, The determination unit is an output device that determines that the first output unit should output the control information based on the first behavior information and the second behavior information when the width of the path on which the first moving body travels satisfies predetermined conditions. 1-8. In any one of the output devices described in 1-1 to 1-7, The first behavior is an output device in which the occupant of the first mobile body makes a gesture or vocalization toward the second mobile body. 1-9. In any one of the output devices described in 1-1 to 1-8, The first moving object is a vehicle, and the second moving object is a vehicle facing the first moving object. The second behavior is an output device in which the second moving body flashes its headlights, hazard lights, or brake lights one or more times, or sounds its horn one or more times. 1-10. In any one of the output devices described in 1-1 to 1-9, The first acquisition unit is capable of receiving input from the passenger to the first input unit, The first behavior is an output device which is an input to the first input unit. 1-11. In any one of the output devices described in 1-1 to 1-10, The system further includes a detection unit for detecting the behavior of the second moving object, The second acquisition unit acquires the second behavior information from the detection unit, The detection unit is An image is obtained that includes at least a portion of the second moving object. An output device that detects the second behavior using the aforementioned image and generates the second behavior information. 1-12. In any one of the output devices described in 1-1 to 1-11, An output device further comprising a second output unit that outputs notification information for informing the passenger of the contents of the control information. 1-13. In any one of the output devices described in 1-1 to 1-12, An output device further comprising a second input section capable of receiving input to change the content of the aforementioned control information. 1-14. In any one of the output devices described in 1-1 to 1-13, An output device further comprising a third output unit that outputs request information to prompt the aforementioned passenger to take action. 1-15. In the output device described in 1-14, The first acquisition unit is an output device that acquires information indicating the behavior of the passenger that occurred within a predetermined period from the output of the request information, as the first behavior information. 2-1. A first acquisition step of acquiring first behavior information that indicates the first behavior of the occupant of the first mobile body, A second acquisition step involves acquiring second behavior information indicating the second behavior of the second moving body around the first moving body within a predetermined period from the occurrence of the first behavior, A first output step that outputs control information for controlling the movement of the first mobile body based on the first behavior information and the second behavior information, Output methods including [specific methods / methods]. 2-2. In the output method described in 2-1, Further information is obtained indicating the third behavior of the passenger within a predetermined period following the occurrence of the second behavior, The first output step is an output method that outputs the control information based on the third behavior information. 2-3. In the output method described in 2-1 or 2-2, The first output step further includes a determination step of determining whether or not to output the control information based on the first behavior information and the second behavior information, An output method in which, if it is determined in the determination step that the control information should be output, the control information is output in the first output step. 2-4. In the output method described in 2-3, The determination step is an output method which determines in the first output step that the control information should be output based on the first behavior information and the second behavior information when the first moving body is located within a predetermined range from a pedestrian crossing or intersection. 2-5. In the output method described in 2-3 or 2-4, The determination step is an output method which determines that when the speed of the first moving body falls below a predetermined speed, the control information should be output in the first output step based on the first behavior information and the second behavior information. 2-6. In any of the output methods described in 2-3 to 2-5, The first output step further includes a third acquisition step of acquiring information indicating a region requiring attention, for which control information should be output based on the first behavior information and the second behavior information, The determination step is an output method which determines that when the first moving body is located within the area requiring attention, the control information should be output in the first output step based on the first behavior information and the second behavior information. 2-7. In any of the output methods described in 2-3 to 2-6, The determination step is an output method which determines that the control information should be output in the first output step based on the first behavior information and the second behavior information when the width of the path on which the first moving body travels satisfies predetermined conditions. 2-8. In any of the output methods described in 2-1 to 2-7, The first behavior is an output method which is a gesture or vocalization made by the occupant of the first mobile body toward the second mobile body. 2-9. In any of the output methods described in 2-1 to 2-8, The first moving object is a vehicle, and the second moving object is a vehicle facing the first moving object. The second behavior is an output method in which the second moving body flashes its headlights, hazard lights, or brake lights one or more times, or sounds its horn one or more times. 2-10. In any of the output methods described in 2-1 to 2-9, In the first acquisition step, the input of the passenger to the first input unit can be accepted. The first behavior is an output method which is an input to the first input unit. 2-11. In any of the output methods described in 2-1 to 2-10, The system further comprises a detection step for detecting the behavior of the second moving object, In the second acquisition step, the second behavior information generated in the detection step is acquired. In the aforementioned detection step, An image is obtained that includes at least a portion of the second moving object. An output method for detecting the second behavior using the aforementioned image and generating the second behavior information. 2-12. In any of the output methods described in 2-1 to 2-11, An output method further comprising the step of outputting notification information for informing the passenger of the contents of the control information. 2-13. In any of the output methods described in 2-1 to 2-12, An output method further comprising the step of being able to accept an input that modifies the content of the control information. 2-14. In any of the output methods described in 2-1 to 2-13, An output method further comprising a third output step of outputting request information to prompt the aforementioned passenger to take action. 2-15. In the output method described in 2-14, The first acquisition step is an output method that acquires information indicating the behavior of the passenger that occurred within a predetermined period from the output of the request information as the first behavior information. 3-1. A computer program for realizing an output device, Computers, A first acquisition means for acquiring first behavior information indicating the first behavior of a occupant of a first mobile body, A second acquisition means for acquiring second behavior information indicating the second behavior of the second moving body around the first moving body within a predetermined period from the occurrence of the first behavior, and A first output means that outputs control information for controlling the movement of the first mobile body based on the first behavior information and the second behavior information, A computer program that functions as such. 3-2. In the computer program described in 3-1, The first acquisition means further acquires third behavior information indicating the third behavior of the passenger within a predetermined period from the occurrence of the second behavior, The first output means is a computer program that outputs the control information based on the third behavior information. 3-3. In the computer program described in 3-1 or 3-2, The computer is further configured to function as a determination means for determining whether the first output means should output the control information based on the first behavior information and the second behavior information. The first output means is a computer program that outputs the control information when the determination means determines that it should output. 3-4. In the computer program described in 3-3, The determination means is a computer program that determines that the first output means should output the control information based on the first behavior information and the second behavior information when the first moving body is located within a predetermined range from a pedestrian crossing or intersection. 3-5. In the computer programs described in 3-3 or 3-4, The determination means is a computer program that determines, when the speed of the first moving body falls below a predetermined speed, that the first output means should output the control information based on the first behavior information and the second behavior information. 3-6. In any one of the computer programs described in 3-3 to 3-5, The computer is further configured to function as a third acquisition means for acquiring information indicating a region requiring attention, on which the first output means should output the control information based on the first behavior information and the second behavior information. The determination means is a computer program that determines that when the first moving body is located within the area requiring attention, the first output means should output the control information based on the first behavior information and the second behavior information. 3-7. In any one of the computer programs described in 3-3 to 3-6, The determination means is a computer program that determines that the first output means should output the control information based on the first behavior information and the second behavior information when the width of the road on which the first moving body travels satisfies predetermined conditions. 3-8. In any one of the computer programs described in 3-1 to 3-7, A computer program in which the first behavior is a gesture or vocalization made by the occupant of the first mobile body toward the second mobile body. 3-9. In any one of the computer programs described in 3-1 to 3-8, The first moving object is a vehicle, and the second moving object is a vehicle facing the first moving object. The second behavior is a computer program in which the second moving object flashes its headlights, hazard lights, or brake lights one or more times, or sounds its horn one or more times. 3-10. In any one of the computer programs described in 3-1 to 3-9, The first acquisition means is capable of receiving input from the passenger to the input means, The first behavior described above is a computer program which is input to the input means. 3-11. In any of the computer programs described in 3-1 to 3-10, The computer is further configured to function as a detection means for detecting the behavior of the second moving object. The second acquisition unit acquires the second behavior information from the detection means, The aforementioned detection means is An image is obtained that includes at least a portion of the second moving object. A computer program that detects the second behavior using the aforementioned image and generates the second behavior information. 3-12. In any of the computer programs described in 3-1 to 3-11, A computer program that causes a second output means to output notification information for informing the passenger of the contents of the control information. 3-13. In any of the computer programs described in 3-1 to 3-12, A computer program capable of receiving input to a second input means for changing the content of the control information. 3-14. In any one of the computer programs described in 3-1 to 3-13, A computer program that causes the aforementioned passenger to output request information to a third output means to prompt a specific action. 3-15. In the computer program described in 3-14, The first acquisition means is a computer program that acquires information indicating the behavior of the passenger that occurred within a predetermined period from the output of the request information as the first behavior information. [Explanation of Symbols]

[0136] 10 Output device 40 Integrated Circuits 110 First acquisition part 112 First detection unit 120 Second acquisition part 122 Second detection unit 130 Third acquisition part 140 Judgment section 150 First Output Section 152 Second Output Section 153 Third Output Section 160 First Input Section 162 Second Input Section 170 Storage section Bus 402 404 Processor 406 memory 408 storage devices 410 Input / Output Interfaces 412 Network Interfaces 420 Camera 1 421 Second Camera 422 monitors 423 Input Panel 424 Mike 425 Speaker

Claims

[Claim 1] A first acquisition unit that acquires first behavior information indicating the first behavior of a passenger of a first mobile body, A second acquisition unit acquires second behavior information indicating the second behavior of the second moving body around the first moving body within a predetermined period from the occurrence of the first behavior, A first output unit outputs control information for controlling the movement of the first mobile body based on the first behavior information and the second behavior information, An output device equipped with the following features.

Images