Mobile Action Planning Device

Abstract

When a moving object passes through a pedestrian crossing, even if an obstruction partially blocks the field of view of a roadside sensor, it is required to generate a movement plan for the moving object without requiring another roadside sensor. [Solution] The mobile body action planning device acquires roadside sensor information from roadside sensors that detect surrounding obstacles, acquires the position information of the mobile body, acquires map information of the path to be moved, detects the state of pedestrian crossings present in the path of the mobile body based on the position information of the mobile body, map information and roadside sensor information, determines whether there are any obstructions that block the field of view of the roadside sensors, tracks obstacles around the pedestrian crossings if obstructions exist, estimates the future position of the obstacles, and generates an action plan for the mobile body.

Description

Technical Field

[0004] , , , ,

[0005] , , ,

[0003] , ,

[0001] This application relates to a mobile behavior planning device.

Background Art

[0002] In recent years, the introduction of automatic driving technology to mobile bodies has been desired. Regarding vehicles traveling on roads, technologies have been disclosed that utilize information obtained by roadside sensors of roadside monitoring devices provided along the road to perform driving support for the vehicles and realize automatic driving. Even when there are obstacles on the road or when traveling on a road with poor visibility, it is possible to select a roadside sensor that enables driving support and generate a behavior plan for the mobile body using the information of the roadside sensor (for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the infrastructure sensor management device described in Patent Document 1, for an autonomous vehicle or a driver-operated vehicle, a roadside sensor that detects an area where detection is required based on the vehicle's position information, information on obstacles, map information, information on the presence or absence of crosswalks, etc. is selected. A first roadside sensor that can detect a first range including an obstacle and the area on the opposite side separated from the obstacle, and a second roadside sensor that can detect the remaining range are selected. By providing information from each roadside sensor, it is possible to obtain information around the crosswalk through which the vehicle passes and perform driving support.

[0005] However, in the technology described in Patent Document 1, when a moving object passes through a pedestrian crossing, multiple roadside sensors are required to compensate for blind spots caused by obstructions blocking the field of view of the roadside sensors. This increases the number of roadside sensors required when generating the moving object's action plan, necessitating costly infrastructure development. As a result, the system cost for assisting the driving of moving objects increases.

[0006] The present disclosure aims to provide a mobile object behavior planning device that can generate a behavior plan for a mobile object without requiring another roadside sensor, even when an obstruction blocks part of the field of view of a roadside sensor when the mobile object passes through a pedestrian crossing. [Means for solving the problem]

[0007] The mobile action planning device relating to this disclosure is A roadside sensor information acquisition unit that acquires roadside sensor information from roadside sensors that detect obstacles around the roadside monitoring device. A unit that acquires the location information of a moving object. Map information acquisition unit that acquires map information of the path a moving object is taking. A pedestrian crossing state detection unit detects the state of a pedestrian crossing interference area that interferes with the pedestrian crossing's path, and the state of a pedestrian crossing waiting area adjacent to the interference area, based on the pedestrian crossing position information acquired by the pedestrian crossing position information acquisition unit, the map information acquired by the map information acquisition unit, and the roadside sensor information acquired by the roadside sensor information acquisition unit. An obstruction presence / absence determination unit determines whether there is an obstruction that obstructs the field of view of the roadside sensor in at least one of the pedestrian crossing interference area and the pedestrian crossing waiting area detected by the pedestrian crossing state detection unit. An obstacle tracking and estimation unit that tracks obstacles around a pedestrian crossing when an obstruction is present and estimates the future position of the obstacle, and The system includes a behavior planning unit that generates a behavior plan for a moving object based on the state of the pedestrian crossing detected by the pedestrian crossing state detection unit, the presence or absence of an obstruction determined by the obstruction presence / absence determination unit, and, if an obstruction is present, the estimated position of the obstruction tracked by the obstacle tracking estimation unit. [Effects of the Invention]

[0008] The mobile vehicle action planning device according to this disclosure acquires roadside sensor information from roadside sensors that detect surrounding obstacles, and even if an obstruction in the field of view of the roadside sensor obstructs part of the roadside sensor's field of view at a pedestrian crossing in the mobile vehicle's path, the obstacle tracking and estimation unit tracks the surrounding obstacles and estimates the position of the obstacles, thereby enabling the generation of a mobile vehicle action plan without requiring additional roadside sensors. As a result, the number of required roadside sensors can be reduced, which can lead to lower infrastructure development costs. In other words, the system cost for assisting the driving of mobile vehicles can be reduced. [Brief explanation of the drawing]

[0009] [Figure 1] This is a diagram showing the configuration of the mobile action planning device according to Embodiment 1. [Figure 2] This is a hardware configuration diagram of the mobile action planning device according to Embodiment 1. [Figure 3] This is a plan view showing a pedestrian crossing and roadside monitoring device according to Embodiment 1. [Figure 4] This is a first example of a detection image from a roadside sensor according to Embodiment 1. [Figure 5] This is a second example of a detection image from a roadside sensor according to Embodiment 1. [Figure 6] This is a perspective view showing the case where there is an obstruction at a pedestrian crossing according to Embodiment 1. [Figure 7] This is a plan view showing the case where there is an obstruction at the pedestrian crossing according to Embodiment 1. [Figure 8] This is a first flowchart illustrating the processing of the action planning device according to Embodiment 1. [Figure 9] It is a second flowchart showing the processing of the action plan device according to Embodiment 1. [Figure 10] It is a third flowchart showing the processing of the action plan device according to Embodiment 1. [Figure 11] It is a configuration diagram of the mobile body action plan device according to Embodiment 2. [Figure 12] It is a first plan view showing the case where there is an obstacle on the crosswalk according to Embodiment 2. [Figure 13] It is a second plan view showing the case where there is an obstacle on the crosswalk according to Embodiment 2. [Figure 14] It is a first flowchart showing the processing of the action plan device according to Embodiment 2. [Figure 15] It is a second flowchart showing the processing of the action plan device according to Embodiment 2. [Figure 16] It is a third flowchart showing the processing of the action plan device according to Embodiment 2. [Figure 17] It is a fourth flowchart showing the processing of the action plan device according to Embodiment 2. [Figure 18] It is a first perspective view showing the case where there is an obstacle on the crosswalk according to Embodiment 2. [Figure 19] It is a second perspective view showing the case where there is an obstacle on the crosswalk according to Embodiment 2. [Figure 20] It is a perspective view showing the case where there is no obstacle on the crosswalk according to Embodiment 1.

Embodiments for Carrying Out the Invention

[0010] The embodiments will be described in detail below with reference to the drawings. Note that the drawings are schematic representations, and for the sake of clarity, some components may be omitted or simplified as appropriate. Furthermore, the relative sizes and positions of components shown in different drawings are not necessarily precisely represented and may be modified as appropriate. In the following description, similar components will be denoted by the same reference numerals, and their names and functions will also be the same. Therefore, detailed explanations of these components may be omitted to avoid redundancy.

[0011] 1. Embodiment 1 <Configuration of a mobile action planning device> Embodiment 1 describes the invention assuming a vehicle traveling on a road as the mobile entity. The mobile entity can be an autonomous vehicle. However, the mobile entity is not limited to vehicles traveling on public roads. The mobile entity may also be a transport vehicle, robot, etc., that travels within a limited facility such as a factory or shopping mall. Furthermore, it may be applied to other types of mobile entities.

[0012] Figure 1 is a configuration diagram of a mobile object action planning device 100 according to Embodiment 1. The mobile object action planning device 100 includes a roadside sensor information acquisition unit 110, a mobile object position acquisition unit 130, a map information acquisition unit 140, a pedestrian crossing state detection unit 150, an obstacle presence / absence determination unit 160, an obstacle tracking estimation unit 170, and an action planning unit 180.

[0013] <Roadside monitoring device> The roadside sensor information acquisition unit 110 acquires roadside sensor information from the roadside sensors 11 of the roadside monitoring device 10 installed on the roadside of the road. The roadside sensor information may be transmitted by radio wave communication, or by optical communication or wired communication.

[0014] The roadside monitoring device 10 is sometimes referred to as an RSU (Road Side Unit). Multiple roadside monitoring devices 10 are installed along the roadside and detect obstacles in the surrounding area. Multiple roadside monitoring devices 10 may be clustered together in the same location and each be responsible for monitoring the entire circumferential direction. Alternatively, multiple roadside monitoring devices 10 may be placed at different distances to monitor roads, intersections, etc., from different directions.

[0015] The roadside monitoring device 10 may be installed in areas with particularly heavy traffic, such as intersections, pedestrian crossings, corners, and roads with poor visibility. Furthermore, the monitoring ranges may overlap to ensure continuous monitoring of the road and surrounding areas.

[0016] <Mobile Unit Control Section> The action planning unit 180 of the mobile body action planning device 100 transmits the action plan to the mobile body control unit 200. The mobile body control unit 200 controls the mobile body based on the action plan. The mobile body control unit 200 controls the actions of the mobile body (acceleration, deceleration, braking, change of course, etc.) according to the action plan. Specifically, the mobile body control unit 200 performs actions such as driving the wheels, steering, braking, and operating the transmission. Information between the mobile body action planning device 100 and the mobile body control unit 200 may be transmitted by radio wave communication, optical communication, or wired communication.

[0017] <Roadside Sensor> The roadside sensor 11 is a sensor for understanding the external environment around the roadside monitoring device 10, and can be configured using an image sensor, radio wave sensor, optical sensor, ultrasonic sensor, etc., either individually or in combination. The roadside sensor 11 can detect obstacles such as vehicles with three or more wheels, motorcycles, bicycles, pedestrians, and other objects.

[0018] Image sensors, such as those found in surveillance cameras, capture images of objects and calculate the distance to those objects from the image data captured within a certain field of view. The image data can also provide information about the object's size, direction of movement, speed, and type. Visible light cameras and infrared cameras can be used as image sensors.

[0019] Radio wave sensors can include millimeter-wave radar (MMWR) and other types that utilize the 24-79 GHz frequency band. These sensors can detect the position of objects within the radio wave range and, using the Doppler effect, the speed at which the objects move.

[0020] Optical sensors such as laser radar and LiDAR (Light Detection and Ranging) can be used. By irradiating a laser beam within a certain field of view and detecting point cloud data obtained from the reflection of the laser beam from an object, the position and shape of the object can be determined. Ultrasonic sensors are sensors that detect obstacles at relatively close range by emitting ultrasonic waves in a predetermined angular range and detecting objects from the reflected waves.

[0021] The roadside sensor 11 may use all of the following: image sensors, radio wave sensors, optical sensors, and ultrasonic sensors, or it may use only some of these sensors. In addition, sensors other than those mentioned above may be used to understand the external environment.

[0022] The roadside sensor 11 can utilize sensor fusion technology to detect objects by combining this information. By combining multiple types of sensor information, noise information can be removed, enabling reliable distance measurement, speed detection, and attribute identification of objects. Furthermore, obstacles may be identified based on reinforcement learning such as deep learning.

[0023] All of the information from these sensors may be transmitted to the roadside sensor information acquisition unit 110 and processed by the mobile object action planning device 100. Alternatively, information processing may be performed for each sensor, processing the data acquired by various sensors and transmitting only the position, shape, speed, and type information of the identified object to the roadside sensor information acquisition unit 110. In this way, the processing of sensor information can be distributed, thereby reducing the amount of information that the mobile object action planning device 100 processes individually. The roadside sensor information acquisition unit 110 transmits the roadside sensor information to the pedestrian crossing state detection unit 150, the presence or absence of obstacles determination unit 160, the obstacle tracking estimation unit 170, and the action planning unit 180.

[0024] The mobile vehicle action planning device 100 can be incorporated into the roadside monitoring device 10, or installed alongside it at the same location. Alternatively, the mobile vehicle action planning device 100 may be mounted on the mobile vehicle. Furthermore, a control server may be installed between the roadside monitoring device 10 and the mobile vehicle to manage information, and the mobile vehicle action planning device 100 may be installed on the control server.

[0025] <Hardware configuration of the mobile action planning device> Figure 2 is a hardware configuration diagram of the mobile object action planning device. The hardware configuration shown in Figure 2 can also be individually applied to the roadside monitoring device 10 and the mobile object control unit 200. Here, we will describe the case where it is applied to the mobile object action planning device 100. In this embodiment, the mobile object action planning device 100 is an electronic control device installed to generate an action plan for a mobile object. Each function of the mobile object action planning device 100 is realized by the processing circuit provided in the mobile object action planning device 100. Specifically, the mobile object action planning device 100 includes, as a processing circuit, a arithmetic processing unit 90 (computer) such as a CPU (Central Processing Unit), a storage device 91 that exchanges data with the arithmetic processing unit 90, an input circuit 92 that inputs external signals to the arithmetic processing unit 90, and an output circuit 93 that outputs signals from the arithmetic processing unit 90 to the outside. Each piece of hardware, such as the arithmetic processing unit 90, storage device 91, input circuit 92, and output circuit 93, is connected to each other by a wired network such as a bus or a wireless network.

[0026] The arithmetic processing unit 90 may include an ASIC (Application Specific Integrated Circuit), an IC (Integrated Circuit), a DSP (Digital Signal Processor), a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), various logic circuits, and various signal processing circuits. Furthermore, multiple arithmetic processing units 90 of the same or different types may be provided, with each unit performing a portion of the processing. The storage device 91 may include a RAM (Random Access Memory) configured to read and write data from the arithmetic processing unit 90, or a ROM (Read Only Memory) configured to read data from the arithmetic processing unit 90. The storage device 91 may use non-volatile or volatile semiconductor memory such as flash memory, an SSD (Solid State Drive), an EPROM, or an EEPROM. The input circuit 92 is connected to various sensors, switches, and communication lines, and includes an A / D converter, communication circuits, etc., which input the output signals and communication information of these sensors and switches to the arithmetic processing unit 90. The input circuit 92 may be connected to a mobile sensor 21. The output circuit 93 includes a drive circuit, a communication circuit, etc., that outputs control signals from the arithmetic processing unit 90. The output circuit 93 may be connected to the mobile unit control unit 200. In addition to the input circuit 92 and the output circuit 93, the arithmetic processing unit 90 may be directly connected to a communication device 94 to communicate with, for example, the roadside monitoring device 10.

[0027] Each function of the mobile object action planning device 100 is realized by the arithmetic processing unit 90 executing software (programs) stored in a storage device 91 such as a ROM, and cooperating with other hardware of the mobile object action planning device 100, such as the storage device 91, input circuit 92, and output circuit 93. Setting data such as thresholds and judgment values ​​used by the mobile object action planning device 100 are stored in the storage device 91 such as a ROM as part of the software (program). Each function of the mobile object action planning device 100 may be composed of software modules, or it may be composed of a combination of software and hardware.

[0028] <Functions of the Mobile Action Planning Device> The functions of the mobile object movement planning device 100 are shown in Figure 1. The mobile object movement planning device 100 includes a mobile object position acquisition unit 130 that acquires the position of a mobile object. The mobile object position acquisition unit 130 may acquire positioning information from a GNSS (Global Navigation Satellite System) that detects the position of a mobile object. Alternatively, the position of the mobile object may be calculated using a distance sensor that detects the rotation speed of the mobile object's wheels, a gyro sensor that detects the acceleration, velocity, angular acceleration, and angular velocity of the mobile object, etc. Furthermore, the position of the mobile object may be determined by receiving information from tags embedded in the road using a short-range wireless communication technology such as NFC (Near Field Communication). In addition, the position of the mobile object may be determined by identifying landmark buildings and signs using a mobile object sensor 21 (the mobile object sensor 21 is shown in Figure 11).

[0029] The mobile object position acquisition unit 130 may acquire the position of the mobile object by combining these methods. That is, it can acquire position information using GNSS while correcting position information errors using NFC and landmark identification. Furthermore, in situations where GNSS radio waves cannot be received, the position of the mobile object may be determined by supplementing this with autonomous navigation using a distance sensor and a gyro sensor.

[0030] The mobile object movement planning device 100 includes a map information acquisition unit 140 that acquires map information of the path the mobile object will travel. Based on the mobile object's position and travel path acquired by the mobile object position acquisition unit 130, the map information acquisition unit 140 acquires map information of the area around the current position and travel path from a map database. The map database may be installed on the mobile object, or it may consist of data stored on a server outside the mobile object. It may also be a dynamic map dynamically created and updated by the mobile object.

[0031] Map information allows for the identification of intersections, pedestrian crossings, road shapes, and the placement and detection areas of roadside monitoring devices along the mobile vehicle's route. When the mobile vehicle travels along a designated route, the mobile vehicle planning device 100 selects a roadside monitoring device 10 capable of monitoring the necessary area from among multiple roadside monitoring devices 10 installed around the route. Then, the roadside sensor information detected by the roadside sensor 11 installed on the selected roadside monitoring device 10 is acquired by the roadside sensor information acquisition unit 110.

[0032] Figure 3 is a plan view showing a pedestrian crossing and roadside monitoring device 10 according to Embodiment 1. Here, the mobile unit MO is shown approaching the pedestrian crossing. Pedestrian crossings are installed at intersections with traffic lights, intersections without traffic lights, and on roads with a high volume of pedestrians. Pedestrian crossings often have white striped patterns (CR) painted on the pavement surface to make them easily visible to both pedestrians and vehicles. A stop line (SL) is provided before the striped CR where pedestrians walk.

[0033] When there are pedestrians crossing, vehicles must stop before the stop line SL. In Figure 3, the area of ​​the road defined by the stop line SL and the striped pattern CR is defined as the pedestrian crossing interference area CNF. This is the area that interferes with the vehicle's path when it is traveling on road R. The adjacent area where pedestrians may enter the pedestrian crossing interference area CNF is defined as the pedestrian crossing waiting area WA. If pedestrians are present in the pedestrian crossing waiting area, they may proceed to cross the roadway.

[0034] Figures 4 and 5 show the first and second examples of detection images from a roadside sensor according to Embodiment 1. Figure 4 shows an image of the area around a pedestrian crossing detected by the roadside sensor 11. The striped pattern CR of the pedestrian crossing, the stop line SL, and the moving object MO are shown. Figure 5 shows an obstruction S placed in the pedestrian crossing waiting area WA in the same image of the area around the pedestrian crossing (shown with a white oval in Figure 5). The obstruction S is luggage placed on the sidewalk. In this case, the field of view of the roadside sensor 11 is limited by the obstruction, and it is not possible to detect obstacles in the area hidden by the obstruction S.

[0035] The mobile vehicle action planning device 100 includes a pedestrian crossing state detection unit 150. The pedestrian crossing state detection unit 150 detects the state of pedestrian crossings present in the mobile vehicle's path based on the mobile vehicle's position information acquired by the mobile vehicle position acquisition unit 130, map information acquired by the map information acquisition unit 140, and roadside sensor information acquired by the roadside sensor information acquisition unit 110. Specifically, it detects the state of pedestrian crossing interference regions CNF that interfere with the mobile vehicle's path, and the state of pedestrian crossing waiting regions WA adjacent to the pedestrian crossing interference regions CNF.

[0036] The mobile action planning device 100 includes an obstacle presence / absence determination unit 160. The obstacle presence / absence determination unit 160 determines whether there is an obstacle S that obstructs the field of view of the roadside sensor 11 in at least one of the pedestrian crossing interference region CNF and the pedestrian crossing waiting region WA detected by the pedestrian crossing state detection unit 150.

[0037] Figure 6 is a perspective view showing a case where an obstruction S is present at a pedestrian crossing according to Embodiment 1. In Figure 6, an example is shown where the obstruction S is located in the pedestrian crossing waiting area WA. In Figure 6, a pedestrian, acting as an obstacle O, enters the blind spot of the roadside sensor 11 caused by the obstruction S. It is also possible that the obstruction S is located in a place other than the pedestrian crossing interference area CNF and the pedestrian crossing waiting area WA. The obstruction presence / absence determination unit 160 identifies an object that obstructs the field of view of the roadside sensor 11 with respect to the pedestrian crossing interference area CNF or the pedestrian crossing waiting area WA as an obstruction S and determines the presence or absence of the obstruction.

[0038] The mobile action planning device 100 includes an obstacle tracking and estimation unit 170. The obstacle tracking and estimation unit 170 tracks obstacles O around the pedestrian crossing when an obstruction S is present. The obstacle tracking and estimation unit 170 then estimates the future position of the obstacles O. In Figure 6, the obstacles O are tracked, and the future position of the obstacles O after it is hidden behind the obstruction S is estimated. It is then possible to estimate that a pedestrian, who is the obstacle O, will enter the pedestrian crossing interference area CNF.

[0039] Figure 7 is a plan view showing a case where there is an obstruction S at a pedestrian crossing according to Embodiment 1. In Figure 7, the road on which the moving object is traveling and the opposing road are collectively represented as Road R. Figure 7 shows how the obstacle tracking and estimation unit 170 tracks a pedestrian, who is an obstacle O, walking in an area around the pedestrian crossing but not in the pedestrian crossing interference area CNF or the pedestrian crossing waiting area WA. The obstacle tracking and estimation unit 170 can track the state in which the pedestrian, as an obstacle O, enters the blind spot of the roadside sensor 11 created by the obstruction S, and estimate the future position of the obstacle O.

[0040] The mobile body action planning device 100 includes an action planning unit 180. The action planning unit 180 gives instructions to the mobile body control unit 200 to make the mobile body MO travel along a specified route.

[0041] If an obstacle O is present in the pedestrian crossing interference area CNF at an intersection in the path of the mobile body MO, the action planning unit 180 generates an action plan to stop the mobile body MO at the pedestrian crossing stop line SL. If the obstacle O enters the blind spot area of ​​the roadside sensor 11 and its presence can be estimated to be in the pedestrian crossing interference area CNF, the action planning unit 180 similarly generates an action plan to stop the mobile body MO at the pedestrian crossing stop line SL, assuming that the obstacle O is present in the pedestrian crossing interference area CNF. After the obstacle O has left the pedestrian crossing interference area CNF, the action planning unit 180 can generate an action plan to resume the movement of the mobile body MO.

[0042] If an obstacle O is present in the pedestrian crossing waiting area WA at an intersection in the path of the moving object MO, the action planning unit 180 determines whether the obstacle O enters the pedestrian crossing interference area CNF. If the action planning unit 180 determines that the obstacle O enters the pedestrian crossing interference area CNF, it generates an action plan to stop the moving object MO at the pedestrian crossing stop line SL. If the obstacle O enters the blind spot area of ​​the roadside sensor 11 and its presence can be estimated to be in the pedestrian crossing waiting area WA, it is assumed that the obstacle O enters the pedestrian crossing interference area CNF, and similarly an action plan to stop the moving object MO at the pedestrian crossing stop line SL is generated. The behavior of the obstacle O can be estimated according to its speed and direction of movement.

[0043] If it can be determined that the obstacle O is passing through the pedestrian crossing interference area CNF and crossing the road, and then leaving the pedestrian crossing waiting area WA on the sidewalk, or if it can be determined that the obstacle O is not heading towards the pedestrian crossing interference area CNF but is leaving the pedestrian crossing waiting area WA on the sidewalk, the action planning unit 180 generates an action plan to have the moving object MO proceed slowly through the pedestrian crossing.

[0044] Thus, if the action planning unit 180 determines that the obstacle O is present in the pedestrian crossing waiting area WA and will not enter the pedestrian crossing interference area CNF, it generates an action plan to have the mobile body MO proceed slowly through the pedestrian crossing. Even if, contrary to the determination, the obstacle O begins to enter the pedestrian crossing interference area CNF, if the mobile body is already proceeding slowly, it can immediately generate an action plan to stop the mobile body and respond accordingly. If there is no obstacle O in the pedestrian crossing interference area CNF or the pedestrian crossing waiting area WA on the path of the mobile body MO, the action planning unit 180 generates an action plan for the mobile body MO to proceed through the pedestrian crossing.

[0045] If an obstacle O enters the blind spot area of ​​the roadside sensor 11 and is then detected again by the roadside sensor 11, the position and behavior of the obstacle O can be accurately determined, allowing for the generation of a new action plan. In this way, the mobile action planning device 100 can achieve both proper passage through intersections and a reduction in the time spent stopping and waiting at the stop line SL of a pedestrian crossing.

[0046] By generating action plans in this manner, the action planning unit 180 enables the mobile body action planning device 100 to generate appropriate action plans for pedestrian crossings in the path of the mobile body MO. Furthermore, since it does not require information from other roadside sensors when generating action plans, the number of required roadside sensors can be reduced, thereby lowering the cost of infrastructure development. In other words, the system cost for assisting the driving of mobile bodies can be reduced.

[0047] The obstacle tracking and estimation unit 170 tracks obstacles O around the pedestrian crossing when an obstruction S is present. Therefore, if no obstruction S is present, there is no need to track obstacles O around the pedestrian crossing. Consequently, when no obstruction S is present, the processing load of tracking obstacles O can be reduced, thus avoiding an excessive processing load. For this reason, there is no need to set the processing capacity of the mobile object action planning device 100 to be excessively large, and an excessively fast and expensive processing device is not required. This contributes to reducing the cost of the mobile object action planning device 100.

[0048] Furthermore, it was explained that the obstruction presence / absence determination unit 160 identifies an object that obstructs the field of view of the roadside sensor 11 to the pedestrian crossing interference area CNF or the pedestrian crossing waiting area WA as an obstruction S, and determines the presence or absence of an obstruction. However, it is also possible to identify an object as an obstruction S only if it exists obstructing the view for more than a predetermined determination time Ts (determination time Ts is not shown). Doing so would eliminate false detections due to noise in the roadside sensor 11 or the influence of transient obstructions. This would eliminate the need to track unnecessary obstacles and reduce the processing load of tracking obstacles O.

[0049] Furthermore, an object may be identified as an obstruction S only if it is a fixed object whose position does not change beyond a predetermined determination time Ts. Objects moving on the road R, including the pedestrian crossing interference area CNF, and in the pedestrian crossing waiting area WA should be identified and noted as obstacles O. If moving obstacles O are not identified as obstructions S, it becomes unnecessary to track obstacles outside the road R (on the sidewalk) in areas other than the pedestrian crossing interference area CNF and the pedestrian crossing waiting area WA, thereby reducing the processing load required for tracking.

[0050] <Processing by the mobile object action planning device> Figures 8 to 10 are the first to third flowcharts showing the processing of the mobile body action planning device 100 according to Embodiment 1. Figure 9 is a flowchart continuing from Figure 8. Figure 10 is a flowchart continuing from Figures 8 and 9.

[0051] The process shown in Figure 8 may be executed at predetermined intervals (for example, every 1 ms). Alternatively, it may be executed each time the mobile body MO travels a predetermined distance. Or, it may be executed in response to events such as when the roadside sensor information acquisition unit 110 of the mobile body action planning device 100 acquires roadside sensor information, or when the mobile body position acquisition unit 130 acquires the position of the mobile body.

[0052] The process shown in Figure 8 is initiated, and in step S101, the mobile object position acquisition unit 130 acquires the position information of the mobile object MO. In step S102, the map information acquisition unit 140 acquires map information of the current location and the area around the travel path from the map database.

[0053] In step S103, a pedestrian crossing along the route is selected. At this time, the pedestrian crossing closest to the current position of the moving object MO may be selected, and an action plan may be generated. Alternatively, action plans may be created in parallel by sequentially selecting pedestrian crossings along the route. Here, we will explain with an example the case in which an action plan is generated by selecting the pedestrian crossing closest to the current position.

[0054] In step S104, the optimal roadside monitoring device 10 for monitoring the selected pedestrian crossing is selected. At this time, the roadside monitoring device 10 may be selected based on the direction of travel of the moving body MO, the condition around the pedestrian crossing, the presence or absence of obstructions S, etc. In step S105, the roadside sensor information detected by the roadside sensor 11 of the selected roadside monitoring device 10 is acquired by the roadside sensor information acquisition unit 110.

[0055] In step S107, the pedestrian crossing state detection unit 150 detects the state of the pedestrian crossing. It detects the state of the pedestrian crossing selected based on the location information of the moving object, map information, and roadside sensor information. Specifically, it detects the state of the pedestrian crossing interference area CNF and the state of the pedestrian crossing waiting area WA.

[0056] In step S108, the obstacle presence / absence determination unit 160 determines whether or not an obstacle S exists at the selected intersection. An obstacle S may be identified as an obstacle S only if it is a fixed object that obstructs the field of view of the roadside sensor 11 to part or all of the area of ​​the pedestrian crossing interference region CNF or the pedestrian crossing waiting region WA, and whose position does not change beyond a predetermined determination time Ts.

[0057] In step S109, the presence or absence of an obstruction S is determined. If an obstruction exists (determination is YES), proceed to step S121 in Figure 9. If there is no obstruction (determination is NO), proceed to step S111 in Figure 10.

[0058] In step S121 of Figure 9, the obstacle tracking estimation unit 170 identifies and tracks the position, speed, and direction of movement of the obstacle O around the selected pedestrian crossing. Then, in step S122, the future position of the obstacle O is estimated if it is hidden behind the shield S.

[0059] In step S123, the possibility that obstacle O is behind shielding S is estimated. In step S124, the possibility is determined. If there is a possibility that obstacle O is behind shielding S (determination is YES), proceed to step S119 in Figure 10. If there is no possibility that obstacle O is behind shielding S (determination is NO), proceed to step S111 in Figure 10.

[0060] In step S111 of Figure 10, the presence or absence of an obstacle O in the pedestrian crossing interference area CNF is checked based on roadside sensor information. In step S112, it is determined whether or not an obstacle O is present. If an obstacle O is present (determination is YES), proceed to step S119. Here, if the obstacle O has entered the blind spot area of ​​the roadside sensor 11 and its presence can be estimated to be in the pedestrian crossing interference area CNF, it is considered that an obstacle O exists in the pedestrian crossing interference area CNF and proceed to step S119. If no obstacle O is present in step S112 (determination is NO), proceed to step S113.

[0061] In step S113, the presence or absence of an obstacle O in the pedestrian crossing waiting area WA is checked based on roadside sensor information. In step S114, it is determined whether or not an obstacle O exists. If an obstacle O exists (determination is YES), proceed to step S116. Here, if it can be estimated that the obstacle O has entered the blind spot area of ​​the roadside sensor 11 and is located in the pedestrian crossing waiting area WA, it is considered that an obstacle O exists in the pedestrian crossing waiting area WA and proceed to step S116. If in step S114 there is no obstacle O (determination is NO), proceed to step S115.

[0062] In step S115, the action planning unit 180 creates an action plan for the mobile body MO to pass through the crosswalk interference area CNF of the designated crosswalk. Then the process ends.

[0063] In step S116, it is checked whether the obstacle O present in the pedestrian crossing waiting area WA is attempting to enter the pedestrian crossing interference area CNF. If it can be determined that the obstacle O is attempting to leave the pedestrian crossing waiting area WA by crossing the road via the pedestrian crossing interference area CNF, or if it can be determined that the obstacle O is attempting to leave the pedestrian crossing waiting area WA by leaving the pedestrian crossing waiting area WA by crossing the sidewalk without heading towards the pedestrian crossing interference area CNF, then it can be determined that the obstacle O is not attempting to enter the pedestrian crossing interference area CNF. Otherwise, it should be determined that the obstacle O is attempting to enter the pedestrian crossing interference area CNF. Furthermore, if the obstacle O enters the blind spot area of ​​the roadside sensor 11 and its presence can be estimated to be in the pedestrian crossing waiting area WA, then it can be considered that the obstacle O is attempting to enter the pedestrian crossing interference area CNF.

[0064] In step S117, it is determined whether the obstacle O located in the pedestrian crossing waiting area WA is attempting to enter the pedestrian crossing interference area CNF. If the obstacle O is attempting to enter the pedestrian crossing interference area CNF (determination is YES), proceed to step S119. If the obstacle O is not attempting to enter the pedestrian crossing interference area CNF (determination is NO), proceed to step S118.

[0065] In step S118, the action planning unit 180 generates an action plan for the moving object MO to proceed slowly and pass through the crosswalk interference area CNF of the designated crosswalk. Then the process ends.

[0066] In step S119, the action planning unit 180 generates an action plan for the mobile body MO to stop before the stop line SL of the designated pedestrian crossing. Then the process ends.

[0067] 2. Embodiment 2 <Configuration of a mobile action planning device> Figure 11 is a configuration diagram of the mobile object action planning device 100 according to Embodiment 2. The difference between Figure 11 according to Embodiment 2 and Figure 1 according to Embodiment 1 is that a mobile object sensor information acquisition unit 120 is added as a component of the mobile object action planning device 100, and it acquires mobile object sensor information output by the mobile object sensor 21. The hardware configuration diagram in Figure 2 can also be applied to the mobile object action planning device 100 of Embodiment 2. Here, we will mainly explain the differences from Embodiment 1.

[0068] <Mobile Sensor> The mobile unit MO is equipped with a mobile sensor 21 that detects obstacles around the mobile unit MO. The mobile sensor 21 is a sensor that perceives the external environment and, like the roadside sensor 11, can be configured by combining image sensors, radio wave sensors, optical sensors, ultrasonic sensors, etc. Obstacles that the mobile sensor 21 should detect include vehicles with three or more wheels, motorcycles, bicycles, pedestrians, and other objects.

[0069] The mobile sensor information acquisition unit 120 receives information from the image sensor, radio wave sensor, optical sensor, and ultrasonic sensor of the mobile sensor 21, which is a sensor that grasps the external environment. The mobile sensor 21 may use all of the image sensor, radio wave sensor, optical sensor, and ultrasonic sensor, or it may use only some of the sensors. In addition, other sensors may be used to grasp the external environment.

[0070] The mobile sensor information acquisition unit 120 can use sensor fusion technology to detect objects by combining this information. By combining multiple types of sensor information, noise information can be removed, enabling highly reliable object distance measurement, velocity detection, and attribute identification. Furthermore, obstacles may be identified based on reinforcement learning such as deep learning.

[0071] All of the information from these sensors may be processed by the mobile sensor information acquisition unit 120, or information processing may be performed for each sensor, processing the data acquired by various sensors and transmitting only the position, shape, speed, and type information of the identified object to the mobile sensor information acquisition unit 120. In this way, the processing of sensor information can be distributed and executed, so the amount of information that the mobile sensor information acquisition unit 120 processes alone can be reduced. The mobile sensor information acquisition unit 120 transmits the mobile sensor information to the pedestrian crossing state detection unit 150 and the action planning unit 180.

[0072] <Functions of the Mobile Action Planning Device> The difference in the function of the mobile object action planning device 100 according to Embodiment 2 compared to the mobile object action planning device 100 according to Embodiment 1 is that the pedestrian crossing state detection unit 150 and the action planning unit 180 additionally use mobile object sensor information detected by the mobile object sensor 21, in addition to the roadside sensor information detected by the roadside sensor 11, to perform processing. The functions of the obstacle presence / absence determination unit 160 and the obstacle tracking estimation unit 170 in the mobile object action planning device 100 in Embodiment 2 are the same as in Embodiment 1.

[0073] Figure 12 is a first plan view showing the case where there is an obstruction S at the pedestrian crossing according to Embodiment 2. The difference between Figure 12 according to Embodiment 2 and Figure 7 according to Embodiment 1 is that the mobile sensor 21 is mounted on the mobile body MO. In Figure 12, the detection range of the mobile sensor 21 is illustrated by a dashed line. There is an obstruction S that blocks the field of view of the roadside sensor 11 provided on the roadside monitoring device 10. However, the portion of the field of view that is blocked from the roadside sensor 11 can be detected by the mobile sensor 21 when the mobile body MO approaches the pedestrian crossing.

[0074] The action planning unit 180 of the mobile body action planning device 100 gives instructions to the mobile body control unit 200 to make the mobile body MO travel along a specified path. If an obstacle O exists in the crosswalk interference area CNF of a crosswalk in the path of the mobile body MO, the action planning unit 180 generates an action plan to stop the mobile body MO at the crosswalk's stop line SL.

[0075] <Example 1> If it is estimated that the obstacle O has entered the blind spot area of ​​the roadside sensor 11, the action planning unit 180 considers that the obstacle O is in the pedestrian crossing interference area CNF and similarly generates an action plan to stop the mobile body MO at the pedestrian crossing stop line SL. If it is determined that the obstacle O is not in the blind spot area of ​​the roadside sensor 11, the mobile body sensor 21 is used to check whether the obstacle O can be detected. If detection is not possible, the action planning unit 180 considers that the obstacle O is in the pedestrian crossing interference area CNF and generates an action plan to stop the mobile body MO at the pedestrian crossing stop line SL.

[0076] If an obstacle O can be detected using the mobile sensor 21, the roadside sensor information and mobile sensor information detected by the roadside sensor 11 and the mobile sensor 21 are used to determine whether there is an obstacle in the pedestrian crossing interference area CNF or the pedestrian crossing waiting area WA. The action planning unit 180 then generates an action plan to stop the mobile vehicle MO, pass through the pedestrian crossing, or proceed slowly through the pedestrian crossing. In particular, it is possible to generate a highly reliable action plan without providing a high-precision mobile sensor 21.

[0077] <Example 2> Alternatively, if it is estimated that the obstacle O has entered the blind spot area of ​​the roadside sensor 11, the mobile sensor 21 is used to check whether the obstacle O can be detected. If it cannot be detected, the action planning unit 180 assumes that the obstacle O is in the pedestrian crossing interference area CNF and generates an action plan to stop the mobile body MO at the pedestrian crossing stop line SL.

[0078] If an obstacle O can be detected using the mobile sensor 21, the roadside sensor information and mobile sensor information detected by the roadside sensor 11 and the mobile sensor 21 are used to determine whether there is an obstacle in the pedestrian crossing interference area CNF or the pedestrian crossing waiting area WA. The action planning unit 180 then generates an action plan to stop the mobile body MO, pass through the pedestrian crossing, or proceed slowly through the pedestrian crossing.

[0079] As described above, an action plan can be generated using the roadside sensor information and mobile sensor information detected by the roadside sensor 11 and mobile sensor 21. This makes it possible to generate a more appropriate action plan. Even if an obstacle O enters the blind spot area of ​​the roadside sensor 11, an action plan can be generated that allows the mobile object MO to quickly pass through the pedestrian crossing based on the information detected by the mobile sensor 21.

[0080] Figure 13 is a second plan view showing the case where there is an obstruction S at a pedestrian crossing according to Embodiment 2. Compared to Figure 12 according to Embodiment 2, this shows the case where a large truck is parked as a second mobile body MO2 on the opposite lane side of the mobile body MO. In the case shown in Figure 13, the obstruction S creates a pedestrian crossing interference region CNF and a pedestrian crossing waiting region WA where the roadside sensor 11 cannot detect the obstacle O, which is a pedestrian. Furthermore, the mobile body sensor 21 also has a pedestrian crossing interference region CNF and a pedestrian crossing waiting region WA where the field of view is obstructed by the large truck as the second mobile body MO2, making it impossible to detect the obstacle O.

[0081] In such cases, the action planning unit 180 assumes that the obstacle O is in the pedestrian crossing interference area CNF and generates an action plan to stop the moving body MO at the pedestrian crossing stop line SL. This makes it possible to generate a more appropriate action plan. Furthermore, if the second moving body MO2, which is a large truck, moves, and if it is determined that the obstacle O appears in an area detectable by the roadside sensor 11 or the moving body sensor 21 and does not enter the pedestrian crossing interference area CNF, the action planning unit 180 can generate an action plan to have the moving body MO pass through the pedestrian crossing.

[0082] <Processing of the mobile object action planning device (1)> Figures 14 to 16 are the first to third flowcharts showing Example 1 of the processing of the mobile body action planning device 100 according to Embodiment 2. Figure 15 is a flowchart continuing from Figure 14. Figure 16 is a flowchart continuing from Figures 14 and 15.

[0083] The process shown in Figure 14 may be executed at predetermined intervals (for example, every 1 ms). Alternatively, it may be executed each time the mobile body MO travels a predetermined distance. Or, it may be executed in response to events such as the roadside sensor information acquisition unit 110 of the mobile body action planning device 100 acquiring roadside sensor information, or the mobile body position acquisition unit 130 acquiring the position of the mobile body.

[0084] The only difference between the process in Figure 14 and Figure 8 relating to Embodiment 1 is the addition of step S106 between steps S105 and S107. Only the added part will be explained. In the added step S106, the mobile sensor information detected by the mobile sensor 21 is acquired by the mobile sensor information acquisition unit 120.

[0085] The only difference between the process in Figure 15 and Figure 9 in Embodiment 1 is that step S124 has been changed to steps S134, S125, and S126. Only the changed part will be explained.

[0086] If in step S134 there is no possibility that the obstacle O is behind the shield S (judgment is NO), proceed to step S125. If there is a possibility that the obstacle O is behind the shield S (judgment is YES), proceed to step S119 in Figure 16. In step S125, check whether the obstacle O behind the shield S can be detected by the mobile sensor 21.

[0087] In step S126, it is determined whether the obstacle O behind the shield S can be detected by the mobile sensor 21. If it can be detected by the mobile sensor 21 (determination is YES), proceed to step S131 in Figure 16. If it cannot be detected by the mobile sensor 21 (determination is NO), proceed to step S119 in Figure 16.

[0088] In the process shown in Figure 16, the only difference from Figure 10 relating to Embodiment 1 is that steps S111 and S113 have been changed to steps S131 and S133. Only the changed part will be explained.

[0089] In step S131, the verification is performed based on both roadside sensor information and mobile sensor information. Based on the roadside sensor information and mobile sensor information, the presence or absence of obstacles O in the pedestrian crossing interference area CNF is checked.

[0090] In step S133, the confirmation is performed based on both roadside sensor information and mobile object sensor information. Based on the roadside sensor information and mobile object sensor information, the presence or absence of obstacles O in the pedestrian crossing waiting area WA is checked.

[0091] By processing in this manner, if there is a possibility that an obstacle O is behind the obstruction S that blocks the field of view of the roadside sensor 11, an action plan is generated to carefully stop the moving body MO at the stop line SL of the pedestrian crossing. Then, only when there is no possibility that an obstacle O is behind the obstruction S that blocks the field of view of the roadside sensor 11, the detection information from the moving body sensor 21 is used to determine whether there is an obstacle O behind the obstruction S that blocks the field of view of the roadside sensor 11. This makes it possible to prepare for situations where an obstacle O, such as a pedestrian, suddenly appears from behind the obstruction S.

[0092] <Processing by the mobile object action planning device (2)> Figures 14, 17, and 16 are the first, fourth, and third flowcharts showing the processing of the mobile body action planning device 100 according to Embodiment 2. Figure 17 is a flowchart continuing from Figure 14. Figure 16 is a flowchart continuing from Figures 14 and 17.

[0093] In the processing of the mobile body action planning device 100 according to Embodiment 2, the only difference from Embodiment 1 is that Figure 15 is changed to Figure 17. Figure 17 differs from Figure 15 only in that step S134 is changed to step S144. Only the differences will be explained below.

[0094] In step S123, the possibility that obstacle O is behind shielding S is estimated. In step S144, the possibility is determined. If there is a possibility that obstacle O is behind shielding S (determination is YES), proceed to step S125. If there is no possibility that obstacle O is behind shielding S (determination is NO), proceed to step S131 in Figure 16.

[0095] By processing in this manner, even if there is a possibility that an obstacle O is behind an obstruction S that blocks the field of view of the roadside sensor 11, the presence or absence of the obstacle O can be confirmed using the detection information from the mobile sensor 21. With this configuration, the mobile sensor 21 can detect the area behind the obstruction S that blocks the field of view of the roadside sensor 11, and an appropriate action plan for passing through the pedestrian crossing can be generated. This makes it possible to generate an appropriate action plan that corresponds to the condition of the pedestrian crossing without delay.

[0096] Figure 18 is a first perspective view showing the case where there is an obstruction S at a pedestrian crossing according to Embodiment 2. It can be seen that the back side of the obstruction S that blocks the field of view of the roadside sensor 11 is detected by the mobile sensor 21, and an action plan for appropriately passing through the pedestrian crossing can be generated.

[0097] Figure 19 is a second perspective view showing a case where there is an obstruction S at a pedestrian crossing according to Embodiment 2. In Figure 19, similar to Figure 13, a large truck is shown as a second mobile object MO2 parked on the opposite lane side of the mobile object MO.

[0098] Due to the obstruction S, there are crosswalk interference regions CNF and crosswalk waiting regions WA in which the roadside sensor 11 cannot detect the obstacle O, which is a pedestrian. Furthermore, from the perspective of the mobile sensor 21, there are also crosswalk interference regions CNF and crosswalk waiting regions WA in which the detection of the obstacle O is impossible due to the obstruction caused by a large truck acting as a second mobile object MO2.

[0099] In such cases, the action planning unit 180 assumes that the obstacle O is in the pedestrian crossing interference area CNF and generates an action plan to stop the moving object MO at the pedestrian crossing stop line SL. By doing so, it becomes possible to generate a more reliable action plan.

[0100] Figure 20 is a perspective view showing the case where there is no obstruction S at the pedestrian crossing according to Embodiment 1. Figure 20 can also be applied to Embodiment 2. When there is no obstruction S, the roadside sensor 11 of the roadside monitoring device 10 can detect the entire area of ​​the pedestrian crossing interference region CNF and the pedestrian crossing waiting region WA, and can appropriately generate an action plan for the mobile body MO. In such a case, there is no need to track obstacles O around the pedestrian crossing, so the processing load of the mobile body action planning device 100 can be reduced. For this reason, there is no need to set the processing capacity of the mobile body action planning device 100 to be excessively large, and an excessively high-speed and expensive processing device is not required. This contributes to reducing the cost of the mobile body action planning device 100.

[0101] While this application describes various exemplary embodiments and examples, the various features, aspects, and functions described in one or more embodiments are not limited to the application of a particular embodiment, but are applicable individually or in various combinations to the embodiments. Accordingly, countless variations not illustrated are envisioned within the scope of the art disclosed herein. These include, for example, modifying, adding or omitting at least one component, or even extracting at least one component and combining it with a component from another embodiment.

[0102] The various aspects of this disclosure are summarized below as an appendix.

[0103] (Note 1) A roadside sensor information acquisition unit that acquires roadside sensor information from roadside sensors that detect obstacles around the roadside monitoring device. A unit that acquires the location information of a moving object. A map information acquisition unit acquires map information of the path along which the moving object travels. A pedestrian crossing state detection unit detects the state of a pedestrian crossing interference area that interferes with the path of the mobile body on the path of the mobile body, and the state of a pedestrian crossing waiting area adjacent to the pedestrian crossing interference area, based on the position information of the mobile body acquired by the mobile body position information acquisition unit, the map information acquired by the map information acquisition unit, and the roadside sensor information acquired by the roadside sensor information acquisition unit. An obstruction presence / absence determination unit determines whether there is an obstruction that obstructs the field of view of the roadside sensor in at least one of the areas of the pedestrian crossing interference area and the pedestrian crossing waiting area detected by the pedestrian crossing state detection unit. Obstacle tracking and estimation unit that tracks the obstacles around the pedestrian crossing when the aforementioned obstruction is present and estimates the future position of the obstacles, and A mobile body action planning device comprising an action planning unit that generates an action plan for the mobile body based on the state of the crosswalk detected by the crosswalk state detection unit, the presence or absence of an obstruction determined by the obstruction presence / absence determination unit, and, if an obstruction exists, the estimated position of the obstruction tracked by the obstacle tracking estimation unit. (Note 2) The mobile body action planning device according to Appendix 1, wherein the obstruction presence / absence determination unit determines that an obstruction exists if an obstruction that obstructs the field of view of the roadside sensor for at least one of the pedestrian crossing interference area and the pedestrian crossing waiting area continues to exist for a predetermined determination time. (Note 3) The mobile body action planning device according to Appendix 1 or 2, wherein the obstruction presence determination unit determines that an obstruction exists when a fixed object obstructs the field of view of the roadside sensor for at least one of the pedestrian crossing interference area and the pedestrian crossing waiting area. (Note 4) The system includes a mobile sensor information acquisition unit that acquires mobile sensor information detected by a mobile sensor installed on the mobile body that detects objects around the mobile body, A mobile body action planning device according to any one of the appendices 1 to 3, wherein the pedestrian crossing state detection unit detects the state of a pedestrian crossing interference region of a pedestrian crossing in the path of the mobile body and a pedestrian crossing waiting region adjacent to the pedestrian crossing interference region, based on the position information of the mobile body acquired by the mobile body position information acquisition unit, the map information acquired by the map information acquisition unit, the roadside sensor information acquired by the roadside sensor information acquisition unit, and the mobile body sensor information acquired by the mobile body sensor information acquisition unit. (Note 5) The mobile body action planning device according to Appendix 4, wherein if the presence or absence of an obstruction is determined by the obstruction presence / absence determination unit, the action planning unit determines whether there is an obstruction in the area obstructed by the obstruction based on the estimated position of the obstruction tracked by the obstacle tracking estimation unit, and if it is determined that there is no obstruction in the area obstructed by the obstruction, the mobile body action planning device generates an action plan for the mobile body passing through the pedestrian crossing interference area based on the mobile body sensor information acquired by the mobile body sensor information acquisition unit, where it is possible to detect an obstacle in the area obstructed by the obstruction and no obstacle is detected in the pedestrian crossing interference area and the pedestrian crossing waiting area. (Note 6) The mobile body action planning device according to Appendix 4, wherein if the presence or absence of an obstruction is determined by the obstruction presence / absence determination unit, the action planning unit estimates the position of the obstruction tracked by the obstacle tracking estimation unit and determines whether the obstruction is in the area obstructed by the obstruction, generates an action plan for the mobile body to pass through the pedestrian crossing interference area if it is determined that there is no obstruction in the area obstructed by the obstruction and no obstruction is detected in the pedestrian crossing interference area and the pedestrian crossing waiting area, and if it is not determined that there is no obstruction in the area obstructed by the obstruction, the action planning unit generates an action plan for the mobile body to pass through the pedestrian crossing interference area if it is possible to detect an obstruction in the area obstructed by the obstruction based on the mobile body sensor information acquired by the mobile body sensor information acquisition unit and no obstruction is detected in the pedestrian crossing interference area and the pedestrian crossing waiting area. (Note 7) The mobile body action planning device according to any one of the appendices 4 to 6, wherein the action planning unit determines that there is an obstruction, and based on the mobile body sensor information acquired by the mobile body sensor information acquisition unit, it generates an action plan to stop the mobile body before the pedestrian crossing interference area if it is not possible to detect an obstacle in the area obstructed by the obstruction. (Note 8) The mobile body action planning device according to any one of the appendices 4 to 7, wherein the action planning unit determines that there is an obstruction, and based on the mobile body sensor information acquired by the mobile body sensor information acquisition unit, it generates an action plan to stop the mobile body before the pedestrian crossing interference area if an obstacle is detected in the area obstructed by the obstruction. (Note 9) The mobile body action planning device according to any one of the appendices 1 to 8, wherein the action planning unit, when it is determined by the obstruction presence / absence determination unit that an obstruction is present, estimates the position of the obstruction tracked by the obstacle tracking estimation unit, determines whether the obstruction is in the area blocked by the obstruction, and generates an action plan to stop the mobile body before the pedestrian crossing interference area when it is determined that the obstruction is in the area blocked by the obstruction. (Note 10) The mobile body action planning device according to any one of the appendices 1 to 9, wherein the action planning unit stops processing by the obstacle tracking estimation unit when the presence or absence of an obstacle is determined to be absent, and generates an action plan for the mobile body based on the state of the crosswalk detected by the crosswalk state detection unit. [Explanation of symbols]

[0104] 10 Roadside monitoring device, 11 Roadside sensor, 21 Mobile object sensor, 100 Mobile object action planning device, 110 Roadside sensor information acquisition unit, 120 Mobile object sensor information acquisition unit, 130 Mobile object position acquisition unit, 140 Map information acquisition unit, 150 Pedestrian crossing state detection unit, 160 Obstacle presence / absence determination unit, 170 Obstacle tracking estimation unit, 180 Action planning unit, CNF Pedestrian crossing interference area, MO Mobile object, O Obstacle, S Obstacle, WA Pedestrian crossing waiting area

Claims

1. A roadside sensor information acquisition unit that acquires roadside sensor information from roadside sensors that detect obstacles around the roadside monitoring device. A unit that acquires the location information of a moving object. A map information acquisition unit acquires map information of the path along which the moving object travels. A pedestrian crossing state detection unit detects the state of a pedestrian crossing interference area that interferes with the path of the mobile body on the path of the mobile body, and the state of a pedestrian crossing waiting area adjacent to the pedestrian crossing interference area, based on the position information of the mobile body acquired by the mobile body position information acquisition unit, the map information acquired by the map information acquisition unit, and the roadside sensor information acquired by the roadside sensor information acquisition unit. An obstruction presence / absence determination unit determines whether there is an obstruction that obstructs the field of view of the roadside sensor in at least one of the areas of the pedestrian crossing interference area and the pedestrian crossing waiting area detected by the pedestrian crossing state detection unit. Obstacle tracking and estimation unit that tracks the obstacles around the pedestrian crossing when the aforementioned obstruction is present and estimates the future position of the obstacles, and A mobile body action planning device comprising an action planning unit that generates an action plan for the mobile body based on the state of the crosswalk detected by the crosswalk state detection unit, the presence or absence of an obstruction determined by the obstruction presence / absence determination unit, and, if an obstruction exists, the estimated position of the obstruction tracked by the obstacle tracking estimation unit.

2. The mobile body action planning device according to claim 1, wherein the obstruction presence / absence determination unit determines that an obstruction is present if an obstruction that obstructs the field of view of the roadside sensor for at least one of the pedestrian crossing interference area and the pedestrian crossing waiting area continues to exist for a predetermined determination time.

3. The mobile body action planning device according to claim 1, wherein the obstruction presence determination unit determines that an obstruction exists when a fixed object obstructs the field of view of the roadside sensor for at least one of the pedestrian crossing interference area and the pedestrian crossing waiting area.

4. The system includes a mobile sensor information acquisition unit that acquires mobile sensor information detected by a mobile sensor installed on the mobile body that detects objects around the mobile body, The mobile body action planning device according to claim 1, wherein the pedestrian crossing state detection unit detects the state of a pedestrian crossing interference region of a pedestrian crossing in the path of the mobile body and a pedestrian crossing waiting region adjacent to the pedestrian crossing interference region, based on the position information of the mobile body acquired by the mobile body position information acquisition unit, the map information acquired by the map information acquisition unit, the roadside sensor information acquired by the roadside sensor information acquisition unit, and the mobile body sensor information acquired by the mobile body sensor information acquisition unit.

5. The mobile body action planning device according to claim 4, wherein if the presence or absence of an obstruction is determined by the obstruction presence / absence determination unit, the action planning unit determines whether there is an obstruction in the area obstructed by the obstruction based on the estimated position of the obstruction tracked by the obstacle tracking estimation unit, and if it is determined that there is no obstruction in the area obstructed by the obstruction, the action planning unit generates an action plan for the mobile body passing through the pedestrian crossing interference area based on the mobile body sensor information acquired by the mobile body sensor information acquisition unit, and no obstruction is detected in the pedestrian crossing interference area or the pedestrian crossing waiting area.

6. If the presence or absence determination unit determines that there is an obstruction, the action planning unit estimates the position of the obstruction tracked by the obstacle tracking estimation unit and determines whether there is an obstruction in the area obstructed by the obstruction; if it determines that there is no obstruction in the area obstructed by the obstruction and no obstruction is detected in the pedestrian crossing interference area and the pedestrian crossing waiting area, it generates an action plan for the moving body to pass through the pedestrian crossing interference area; if it does not determine that there is no obstruction in the area obstructed by the obstruction, it generates an action plan for the moving body to pass through the pedestrian crossing interference area based on the moving body sensor information acquired by the moving body sensor information acquisition unit, and no obstruction is detected in the pedestrian crossing interference area and the pedestrian crossing waiting area.

7. The mobile body action planning device according to claim 4, wherein if the action planning unit determines that there is an obstruction, the unit generates an action plan to stop the mobile body before the pedestrian crossing interference area if it is not possible to detect an obstacle in the area obstructed by the obstruction, based on the mobile body sensor information acquired by the mobile body sensor information acquisition unit.

8. The mobile body action planning device according to claim 4, wherein if the action planning unit determines that there is an obstruction, the unit generates an action plan to stop the mobile body before the pedestrian crossing interference area if an obstacle is detected in the area obstructed by the obstruction, based on the mobile body sensor information acquired by the mobile body sensor information acquisition unit.

9. The mobile body action planning device according to any one of claims 1 to 8, wherein the action planning unit, when it is determined by the obstruction presence / absence determination unit that an obstruction is present, estimates the position of the obstruction tracked by the obstacle tracking estimation unit, determines whether the obstruction is in the area blocked by the obstruction, and generates an action plan to stop the mobile body before the pedestrian crossing interference area if it is determined that the obstacle is in the area blocked by the obstruction.

10. The mobile body action planning device according to any one of claims 1 to 8, wherein if the presence or absence of an obstruction is determined to be absent by the obstruction presence / absence determination unit, the action planning unit stops processing by the obstacle tracking estimation unit and generates an action plan for the mobile body based on the state of the crosswalk detected by the crosswalk state detection unit.

Images