Mobile object control system, control method thereof, mobile object, program and recording medium

Abstract

To provide a moving object control system that can generate a map with the presence of dynamic obstruction taken into account.SOLUTION: A moving object control system according to the present disclosure is a moving object control system controlling operation of a moving object, the system comprises: acquisition means for acquiring information from a sensor to recognize a surrounding of the moving object; map generation means for generating a dynamic prediction map which includes information for indicating a position of a static obstruction recognized based on information from a sensor, and information for indicating a position of dynamic obstruction that changes over time and is recognized based on the information from the sensor; and trajectory generation means for generating a trajectory that serves as a target when controlling a movement of the moving object using the generated dynamic prediction map.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a mobile object control system, a control method thereof, a mobile object, a program, and a recording medium. [Background technology]

[0002] In recent years, there has been an increasing demand for ultra-small mobile vehicles (micromobility) to support people's mobility within small areas. Micromobility can travel in both areas where automobiles and pedestrians are mobile, so it requires autonomous driving technology for traveling on roadways as well as autonomous mobility technology for free spaces such as sidewalks. In the direction of travel of micromobility, it is expected that there will be static obstacles that do not move, as well as obstacles whose position changes dynamically, such as bicycles. For this reason, micromobility must be able to travel along a highly flexible trajectory that can flexibly avoid various obstacles.

[0003] Patent Document 1 describes a technique for determining the situation (scene) in which a vehicle is placed based on a prediction of future movement of an obstacle, and for determining the behavior of the vehicle. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication 2022 / 070303 Summary of the Invention [Problem to be solved by the invention]

[0005] However, if pre-generated high-precision map information is not used, micromobility vehicles must generate their own maps of the surrounding area while traveling. In order to flexibly avoid dynamic obstacles whose positions change, the issue is how to reflect the dynamic obstacles in the maps generated by the vehicle itself. Patent Document 1 did not take such an issue into consideration.

[0006] The present invention has been made in view of the above-mentioned problems, and its object is to realize a technology that can generate a map that takes into account the presence of dynamic obstacles. [Means for solving the problem]

[0007] According to the present invention, A mobile object control system for controlling the operation of a mobile object, an acquisition means for acquiring information from a sensor for recognizing the surroundings of the moving object; a map generating means for generating a dynamic predictive map including information indicating the positions of static obstacles recognized based on the information from the sensors and information indicating the positions of dynamic obstacles that change over time recognized based on the information from the sensors; A mobile object control system is provided, which is characterized by having a trajectory generation means that uses the generated dynamic predictive map to generate a trajectory that serves as a target when controlling the travel of the mobile object. [Effects of the Invention]

[0008] According to the present invention, a technique is realized that can generate a map that takes into account the presence of dynamic obstacles. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a moving body according to an embodiment of the present invention. [Figure 2] FIG. 1 is a block diagram showing an example of the configuration of a control system of a moving body according to an embodiment; [Figure 3] FIG. 1 is a block diagram showing an example of a functional configuration related to a control unit of a moving body according to an embodiment; [Figure 4] FIG. 1 is a diagram illustrating an example of generating a dynamic prediction map according to an embodiment. [Figure 5] FIG. 2 is a diagram illustrating an example of generating a dynamic prediction map according to an embodiment. [Figure 6] FIG. 1 is a diagram for schematically explaining a local path (trajectory) of a moving body according to an embodiment; [Figure 7]FIG. 1 is a diagram illustrating an example in which a local path (trajectory) of a moving body according to an embodiment is configured by combining a clothoid curve and a curvature change point. [Figure 8] FIG. 1 is a diagram illustrating a cost function according to an embodiment. [Figure 9] 1 is a flowchart showing a series of operations in a travel control process for a moving body according to an embodiment; [Figure 10] 1 is a flowchart showing a series of operations for generating a grid map according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention as claimed, and not all combinations of features described in the embodiments are necessarily essential to the invention. Two or more of the features described in the embodiments may be arbitrarily combined. Furthermore, the same reference numerals are used for the same or similar components, and redundant explanations will be omitted.

[0011] In the following embodiments, an ultra-compact electric vehicle with a passenger capacity of about one person will be described as an example of a mobile body that is micromobility. However, micromobility may also include vehicles that travel with people while carrying luggage instead of a person on board. Furthermore, this embodiment is not limited to examples in which the mobile body is an electric vehicle, and can also be applied to mobile bodies other than electric vehicles. Furthermore, in the following description, a mobile body having one driven wheel will be described as an example, but it does not necessarily have to have a driven wheel, and the number of driven wheels is not limited to one, but may be two or more.

[0012] For mobile vehicles such as the micromobility mentioned above, it would be useful to realize autonomous driving that takes into account the presence of passengers, the frequent change of target location, and the lack of high-precision maps. Micromobility vehicles do not necessarily travel along specific, fixed routes. Furthermore, because they can travel in both areas where automobiles and pedestrians are present, they must be able to navigate appropriately even in areas where high-precision maps are not available. Furthermore, they must be able to navigate while appropriately avoiding obstacles in situations where multiple obstacles are present at irregular intervals, such as when traveling through shopping malls or event venues. In addition to static obstacles, micromobility vehicles are likely to encounter obstacles whose positions change dynamically, such as pedestrians, bicycles, and other vehicles, along their travel path. In other words, micromobility vehicles must be able to navigate along a highly flexible trajectory that can flexibly avoid various obstacles. However, because micromobility vehicles may have passengers, the trajectory used to avoid obstacles must also take into account the comfort of the ride.

[0013] The mobile body 100 according to this embodiment autonomously travels toward a target position while avoiding obstacles without using a high-precision map. To enable autonomous travel without using a high-precision map, the area in which the mobile body 100 can travel is identified using information recognized from the output of a detection unit (described later). As described later, the mobile body 100 generates a grid map that indicates areas in which the mobile body 100 can travel and areas in which the mobile body 100 cannot travel. In this embodiment, a grid map that can take into account both static and dynamic obstacles is generated and used to generate a trajectory for the mobile body 100. Furthermore, the mobile body 100 generates a trajectory with a high degree of freedom and that takes into consideration a comfortable ride, using a clothoid curve with a point where the curvature changes nonlinearly (hereinafter simply referred to as a curvature change point). The mobile body 100 controls its drivetrain to travel along the generated trajectory.

[0014] <Configuration of moving body> The configuration of the moving body 100 will be described with reference to Fig. 1. Fig. 1(A) shows a side view of the moving body 100 according to this embodiment, and Fig. 1(B) shows the internal configuration of the moving body 100. In the figure, arrow X indicates the front-to-rear direction of the moving body 100, with F indicating the front and R indicating the rear. Arrows Y and Z indicate the width direction (left-to-right direction) and up-down direction of the moving body 100.

[0015] The mobile object 100 is an electric autonomous vehicle equipped with a propulsion unit 112 and using a battery 113 as a main power source. The battery 113 is, for example, a secondary battery such as a lithium-ion battery, and the mobile object 100 is self-propelled by the propulsion unit 112 using power supplied from the battery 113. The propulsion unit 112 is in the form of a tricycle equipped with a pair of left and right drive wheels 120 that are front wheels, and one driven wheel 121 that is a rear wheel. As described above, the rear wheels may be drive wheels. Note that the propulsion unit 112 may be in the form of a four-wheeled vehicle or other form. The mobile object 100 is equipped with, for example, a single seat 111.

[0016] The propulsion unit 112 includes a drive mechanism 122. The drive mechanism 122 is a mechanism that uses motors 122a and 122b as drive sources to rotate the corresponding drive wheels 120. The drive mechanism 122 can move the mobile body 100 forward or backward by rotating each of the drive wheels 120. The drive mechanism 122 can also change the direction of travel of the mobile body 100 by generating a rotation difference between the motors 122a and 122b. The propulsion unit 112 includes a driven wheel 121. The driven wheel can rotate around the Z direction as its rotation axis.

[0017] The moving body 100 is equipped with detection units 114 to 116 that detect targets around the moving body 100. The detection units 114 to 116 are a group of external sensors that monitor the periphery of the moving body 100. In the present embodiment, the detection units 114 to 116 are all imaging devices that capture images of the periphery of the moving body 100, and include, for example, an optical system such as a lens and an image sensor. However, instead of or in addition to the imaging devices, radar or lidar (Light Detection and Ranging) may be used.

[0018] The detection units 114 are arranged, for example, in pairs at the front of the moving body 100, spaced apart in the Y direction, and are mainly used to detect targets ahead of the moving body 100. The detection units 115 are arranged on the left and right sides of the moving body 100, respectively, and are mainly used to detect targets on the sides of the moving body 100. The detection unit 116 is arranged at the rear of the moving body 100, and is mainly used to detect targets behind the moving body 100.

[0019] 2 is a block diagram of a control system of the mobile object 100. The mobile object 100 includes a control unit (ECU) 130. The control unit 130 includes one or more processors such as a CPU, a memory device such as a semiconductor memory, an interface with an external device, etc. The memory device stores programs executed by the processor and data used by the processor for processing, etc. Multiple sets of processors, memory devices, and interfaces may be provided for different functions of the mobile object 100 and configured to be able to communicate with each other.

[0020] The control unit 130 acquires the outputs (e.g., image information) of the detection units 114 to 116, input information from the operation unit 131, and audio information input from the audio input device 133, and executes processing according to the respective pieces of information. The control unit 130 controls the motors 122a and 122b (controls the driving of the driving unit 112), controls the display of the display panel included in the operation unit 131, and notifies and outputs information to the occupants of the moving object 100 by audio. The control unit 130 may execute processing using a machine learning model for image recognition (e.g., a deep neural network) on the outputs (e.g., image information) from the detection units 114 to 116. The control unit 130 may also execute processing using a machine learning model for voice recognition (e.g., a deep neural network) on the outputs (e.g., audio information) from the audio input device 133.

[0021] The voice input device 133 includes, for example, a microphone, and picks up the voice of the occupant of the mobile object 100. The control unit 130 can recognize the input voice and execute corresponding processing. The GNSS (Global Navigation Satellite system) sensor 134 receives GNSS signals and detects the current position of the mobile object 100.

[0022] The storage device 135 includes a non-volatile recording medium that stores various data. The storage device 135 may also store programs executed by the processor, data used by the processor for processing, etc. The storage device 135 may also store various parameters of machine learning models for speech recognition and image recognition executed by the control unit 130 (for example, trained parameters and hyperparameters of a deep neural network, etc.).

[0023] The communication device 136 is a communication device that can communicate with an external device (for example, an external server or a communication terminal 140 owned by a user) via wireless communication such as Wi-Fi or fifth generation mobile communication.

[0024] Next, an example of the functional configuration of the control unit 130 will be described with reference to Fig. 3. The user instruction acquisition unit 301 acquires a user instruction input via the operation unit 131 or the voice input device 133. The user instruction includes, for example, a specification of a final target position where the moving object 100 should arrive. The final target position may be the position of a target designated by a spoken voice among targets recognized in the images output by the detection units 114 to 116. The user instruction may also include an instruction to change the traveling trajectory of the moving object 100, such as turning right or left, while the moving object 100 is traveling.

[0025] The image information processing unit 302 recognizes the position, shape, etc. of the roadway and obstacles based on the outputs (e.g., image information) of the detection units 114 to 116. Recognition of the position, shape, etc. of the roadway and obstacles ahead of the mobile object 100 is performed, for example, by using stereo images obtained from the two detection units 114 to calculate the depth distance from the mobile object 100. The image information processing unit 302 also recognizes dynamic obstacles such as traffic participants, for example, using monocular images. In the following description, unless otherwise specified, recognition of the roadway, recognition of static obstacles, and recognition of dynamic obstacles will also be simply referred to as obstacle recognition.

[0026] Furthermore, the image information processing unit 302 can predict the movement trajectory of a dynamic obstacle such as a traffic participant, for example, using (e.g., monocular) images acquired in time series. The image information processing unit 302, for example, predicts the position, movement speed, and / or acceleration of the dynamic obstacle, and outputs information on the predicted time-series position of the dynamic obstacle. In the following description, unless otherwise specified, the prediction of the position, movement speed, and / or acceleration of a dynamic obstacle is also simply referred to as movement prediction. For obstacle recognition and movement prediction, the image information processing unit 302 can use, for example, a machine learning model for image recognition (e.g., a deep neural network) that has been trained in advance.

[0027] The grid map generation unit 303 generates a grid map that indicates areas in the vicinity of the moving object 100 that the moving object 100 can travel in and areas that the moving object 100 cannot travel in, based on the results of obstacle recognition and movement prediction by the image information processing unit 302. The grid map generation unit 303 shifts and updates the grid map as the moving object 100 moves so that the moving object is positioned at the center of the grid map.

[0028] 4 and 5 show schematic diagrams of an example of generating a grid map according to this embodiment. The grid map generation unit 303 generates a static grid map 401 using the results of obstacle recognition. The static grid map 401 is information that indicates the positions of static obstacles on a grid-like map. The grid map generation unit 303 assigns the undrivable areas 404 to the corresponding grids of the static grid map 401 according to the recognition results of the image information processing unit 302. For example, if the grid map generation unit 303 recognizes that an object exists at a predetermined height from the ground surface (e.g., a height at which the mobile body 100 cannot proceed), it designates the area corresponding to the position where the object is recognized as an undrivable area. The grid map generation unit 303 generates a static accumulation map 410 by accumulating the undrivable areas 401 as the mobile body 100 moves. The grid map generation unit 303 sets a small forgetting rate for the undrivable areas 404, so that the undrivable areas 404 remain at the corresponding positions on the grid even after a predetermined time (e.g., several minutes to several hours) has passed.

[0029] Furthermore, the grid map generation unit 303 generates a dynamic grid map 402 using the result of the obstacle recognition. The dynamic grid map 402 is information that shows the positions of dynamic obstacles on a grid-like map. The grid map generation unit 303 assigns the positions of the dynamic obstacles to corresponding grids on the dynamic grid map 402 as undriveable areas 406 in accordance with the recognition results of the image information processing unit 302. For example, if the grid map generation unit 303 recognizes the presence of a pedestrian 405, which is a traffic participant, it designates the area corresponding to the position where the object is recognized as the undriveable area 406. Because the position of a dynamic obstacle may change over time, the grid map generation unit 303 sets a large forgetting rate for the undriveable area 406. The grid map generation unit 303 may be configured to erase the undriveable area 406 after, for example, several tens of milliseconds have passed. If the dynamic obstacle moves, after, for example, several tens of milliseconds have passed, the undriveable area 406 corresponding to the position of the dynamic obstacle will move to a nearby grid (the dynamic obstacle will appear at the position of the nearby grid).

[0030] The grid map generation unit 303 generates prediction information 403 based on the result of the movement prediction. The prediction information 403 includes a movement trajectory 407 for each dynamic obstacle (for example, a pedestrian 405 who is a traffic participant). The movement trajectory 407 is information on the predicted time-series position of the dynamic obstacle. The movement trajectory 407 is, for example, information on time change (t1, t2, t3, . . . t n ) can be expressed as a position (x, y) on a two-dimensional grid.

[0031] In this way, the grid map generation unit 303 generates a static grid map (static grid map 401 or static accumulation map 410) that indicates the positions of static obstacles recognized based on image information, a dynamic grid map (dynamic grid map 402) that indicates the positions of dynamic obstacles recognized based on image information, and information on the time-series positions of predicted dynamic obstacles (prediction information 403).The grid map generation unit 303 then uses this information to generate a dynamic prediction map 430 shown in FIG.

[0032] The dynamic prediction map 430 is configured by associating grid maps (constructed on an xy plane) for different times with each other in a time axis direction (direction of t) perpendicular to the xy plane. At this time, the grid map generation unit 303 combines the static accumulation map 410 and the dynamic grid map 402 at each time. As a result, in the dynamic prediction map 430, the grid map at time t1 includes an untravelable area 404 and an untravelable area 406 (t1, t2, and t3 are not shown). Furthermore, the grid map at time t2 includes an untravelable area 404 (whose position has not changed) and an untravelable area 422. The grid map generation unit 303 can determine the untravelable area 422 based on the movement trajectory 407. Similarly, the grid map at time t3 includes an untravelable area 404 (whose position has not changed) and an untravelable area 423. The grid map generation unit 303 can determine the untravelable area 422 based on the movement trajectory 407. The positions of the non-drivable area 406, the non-drivable area 422, and the non-drivable area 423 change over time. A movement trajectory 421 indicates the movement trajectory 421 of the pedestrian 405 within the three-dimensional space of the dynamic predictive map.

[0033] In this way, the grid map generation unit 303 includes information indicating the positions of static obstacles recognized based on image information and information indicating the time-varying positions of dynamic obstacles recognized based on image information in the dynamic predictive map 430. Generating the dynamic predictive map 430 in this way allows the positions of static obstacles and dynamic obstacles to be handled in an integrated manner. That is, it becomes possible to generate a map that takes into account the presence of dynamic obstacles, thereby enabling control of a mobile object that takes into account the presence of dynamic obstacles. The information indicating the time-varying positions of dynamic obstacles includes, for example, information on the predicted time-series positions of the dynamic obstacles (the positions of the impassable regions 406, 422, and 423 associated with t1, t2, and t3). Using the predicted time-series positions of the dynamic obstacles makes it possible to generate a trajectory that avoids the dynamic obstacles entering the position to which the dynamic obstacle is moving.

[0034] The path generation unit 304 generates a path (travel path) along which the mobile object 100 will travel by executing processes for trajectory generation and control variable determination, which will be described later. The path generated by the path generation unit 304 in this embodiment is called a local path, as opposed to a global path, which will be described later. The path generation unit 304 can generate a path based on a target position by referencing the global path. The global path is a reference path for moving toward the target position, roughly determining the target path of the mobile object. The path generation unit 304 can generate a global path so as not to interfere with untravelable areas shown in the static accumulation map 410, for example. In this way, by using the static accumulation map 410, in which the update rate of untravelable areas is slow, a stable path can be generated. The target position in this embodiment is different from a final target position specified by the user, but is a temporary position to be reached when the trajectory along which the mobile object 100 will travel is determined at regular intervals. The target position is set on the global path. The maximum distance from the position of the moving body to the target position may be set according to the range within which an obstacle can be detected by the detection unit, such as 6 meters. That is, the path generation unit 304 can limit the traveling trajectory to a range within which the moving body can adequately detect obstacles, etc. Furthermore, the maximum distance from the position of the moving body to the target position may be set according to, for example, a braking distance within which the moving body 100 can control its traveling in an emergency. That is, the path generation unit 304 can limit the traveling trajectory to a range within which the moving body 100 can be adequately controlled to stop, etc. Furthermore, the target position may be set closer to the moving body as the final target position (e.g., the position of a target specified by a user) approaches.

[0035] To generate a traveling trajectory, the path generation unit 304 generates a trajectory composed of a combination of multiple clothoid curves and curvature change points whose curvature changes nonlinearly. A clothoid curve is a curve whose trajectory curvature changes linearly with distance. A clothoid curve is also generally known as the trajectory drawn by a vehicle equipped with a steering wheel when the vehicle is traveling at a constant speed and the steering wheel is turned at a constant rate. In this embodiment, by combining multiple trajectories whose curvatures change linearly and curvature change points whose curvatures change nonlinearly, a complex trajectory with a high degree of freedom, such as snaking left and right, can be generated. In other words, such a trajectory makes it possible to avoid obstacles by snaking left and right, even when dynamic obstacles are present.

[0036] Furthermore, by configuring the above trajectory, it is possible to generate a trajectory for a vehicle equipped with a steering wheel, for example, by turning the steering wheel at a constant rate, passing through a curvature change point, and then turning the steering wheel in a different constant manner. In other words, even when generating a trajectory with a high degree of freedom, the curvature of the trajectory changes smoothly, ensuring a comfortable ride for passengers riding in the moving body. By adjusting the curvature at three points on the trajectory, the path generation unit 304 can suppress frequent changes in the turning direction of the moving body 100 and further consider motion constraints of the moving body 100, such as the minimum turning radius.

[0037] FIG. 6 schematically shows an example of a trajectory 603 traveled by a moving object 601. An obstacle 602 exists ahead of the moving object 601 in the traveling direction (X direction). The Y direction indicates the left-right direction with respect to the traveling direction of the moving object. The trajectory 603 is made up of multiple clothoid curves and multiple curvature change points 604 to 606. As described above, the multiple clothoid curves and curvature change points enable the path generation unit 304 to generate a trajectory 603 that does not deviate significantly from a global path 607 (shown as a point cloud) even if the global path 607 is a complex path. S1 to S3 indicate the distance between adjacent curvature change points among the curvature change points 604 to 606. This distance may be at regular intervals, or an optimal distance determined in advance through experiments or the like may be set.

[0038] Fig. 7 shows the relationship between the distance from the moving object and the curvature of the trajectory for generating the trajectory 603. As shown in Fig. 7, the trajectory 603 generated in this embodiment has a constant curvature (i.e., a linear change) up to the maximum or minimum point of the curvature (the point where the curvature changes).

[0039] In this embodiment, an example will be described in which the curvatures of the trajectory (curvatures κ1, κ2, and κ3) at the positions of three curvature change points on the trajectory are optimized. While it is possible to use more curvature change points, increasing the number of curvature change points to be adjusted can dramatically increase the amount of calculation required for optimization using a cost function, which will be described later. In other words, the amount of calculation increases significantly compared to the improvement in the degree of freedom of the resulting trajectory. By using three curvature change points, the path generation unit 304 can generate a trajectory with a high degree of freedom while reducing the amount of calculation required for optimization. By appropriately reducing the number of curvature change points, it becomes possible to repeat generation of the trajectory in a shorter cycle (e.g., in real time).

[0040] An outline of the trajectory generation process by the path generation unit 304 will be described. The path generation unit 304 substitutes the trajectory in which the curvature at three points is changed into a cost function, and finds the curvature at the curvature change point that reduces the cost according to the cost function. The cost function is, for example, C poserr is the deviation cost from the global path, C obstacle is the approach and collision cost to the obstacle, C oscillationIf κ is the cost of difference from the trajectory generated immediately before (e.g., one time before), it can be expressed as in Equation (1). The cost of deviation from the global trajectory refers to the global trajectory, and the cost increases as the generated trajectory deviates from the global trajectory. Therefore, even when generating a trajectory with a high degree of freedom, it is possible to generate a trajectory that does not deviate significantly from the global trajectory. Furthermore, the cost of difference from the trajectory generated immediately before (e.g., one time before) decreases as the difference from the trajectory generated immediately before (e.g., one time before) decreases. For example, the trajectory generation unit 304 refers to the curvature of the trajectory generated a predetermined time ago, and uses a cost function in which the cost increases as the curvature of the trajectory to be generated changes significantly from the curvature of the trajectory generated immediately before. Therefore, by using the cost of difference from the trajectory generated immediately before, it is possible to suppress sudden trajectory changes. The trajectory generation unit 304 selects the curvature κ={κ1, κ2, κ3} that minimizes the cost function of Equation (1). For example, the curvature κ1 corresponds to the curvature at the curvature change point 701, the curvature κ2 corresponds to the curvature at the curvature change point 702, and the curvature κ3 corresponds to the curvature at the curvature change point 703. Then, each cost (C poserr , C obstacle , C oscillation ) is calculated according to equations (2) to (4).

[0041]

number

[0042]

number

[0043]

number

[0044]

number

[0045] FIG. 8 shows a schematic diagram of the cost function according to the present embodiment shown in Equation (1). For example, C poserr The deviation cost from the global path, C, is calculated by calculating the trajectory to be generated and the value obtained by assigning the potential of the global path to the grid. For example, the potential of the global path is assigned such that the farther the grid is from the global path, the higher the cost. By using such a potential, the curvature change point is optimized so that the trajectory to be generated approaches the global path. obstacle The approach / collision cost with an obstacle, which is κ, is calculated by calculating the trajectory to be generated and a value obtained by assigning the potential of the obstacle to a grid. For example, the closer the obstacle is to an untravelable area of ​​the grid map, the higher the cost assigned to the potential of the obstacle. By using such potential, the curvature change point is optimized so that the trajectory to be generated avoids the obstacle. When varying the curvature of the curvature change point, the path generation unit 304 performs optimization using Adam with a loss gradient to determine the curvature of the curvature change point (next in the iterative calculation). The path generation unit 304 repeatedly calculates the cost shown in Equation 1 using the trajectory generated by varying the curvature of the curvature change point, and determines κ = {κ1, κ2, κ3} that minimizes the cost L. In this way, the path generation unit 304 generates a trajectory (local path) based on the trajectory of the global path and the time-varying positions of dynamic obstacles included in the dynamic predictive map. This makes it possible to accurately and stably generate a trajectory that avoids entering a position where a dynamic obstacle is about to move.

[0046] In the example shown in FIG. 8, the potential of a global path and the potential of an obstacle are expressed using a cost grid. The cost grid may be composed of the same number of grids as the grid map. As described above, each grid of the cost grid is associated with a cost value according to the distance from the global path and the distance from the obstacle. Multiple cost grids can be taken into account by, for example, adding up the values ​​of the corresponding grids. In a cost grid that handles the potential of a global path, for example, when the global path is projected onto a grid surface, a lower cost value is set for grids closer to the global path.

[0047] Furthermore, the path generation unit 304 performs a process of determining control variables using the determined trajectory to determine the speed v and angular velocity ω, which are control variables for controlling the traveling of the moving object. The process of determining the control variables is a process of determining the speed v and angular velocity ω so as to satisfy predetermined constraints when traveling along the generated trajectory. The predetermined constraints are, for example, constraints that allow the moving object 100 to safely turn a curve or ensure a stable and comfortable ride.

[0048] The traveling control unit 305 controls the traveling of the moving body 100 in accordance with the control amount determined by the path generation unit 304 (for example, controls the motors 122a and 122b).

[0049] Next, a series of operations in the travel control process of the moving body 100 will be described with reference to Fig. 9. This process is realized by the control unit 130 expanding a program stored in the storage device 135 into the memory device of the control unit 130 and executing it. At the start of this process, a final destination position has been set in accordance with a user instruction or the like.

[0050] In S901, the image information processing unit 302 of the control unit 130 uses the outputs (image information) of the detection units 114 to 116 to perform the above-mentioned obstacle recognition and movement prediction using, for example, a deep neural network.

[0051] In S902, the grid map generating unit 303 of the control unit 130 performs grid map generation processing to generate the dynamic prediction map described with reference to Figures 4 and 5. The grid map generation processing will be described later with reference to Figure 10.

[0052] In S903, the route generation unit 304 of the control unit 130 determines a global route. As described above, the route generation unit 304 determines the global route using the static accumulated map. The route generation unit 304 can determine a global route that does not interfere with obstacles (non-travelable areas) on the static accumulated map using, for example, the A* algorithm, but the global route may also be determined using other methods.

[0053] In S904, the path generation unit 304 of the control unit 130 generates a trajectory (local path) along which the moving body travels, including a clothoid curve, as described above. Specifically, the path generation unit 304 first calculates the initial curvature of the curvature change point of the clothoid curve using the global path. Specifically, the path generation unit 304 first determines a path that is close to the global path as the initial path, without considering the presence of an obstacle. This path is determined by calculating C poserr The path generation unit 304 performs optimization calculations using only C poserr and C oscillation By performing the processing of this step, it is possible to reduce the possibility that the curvature of the local route determined by the optimization calculation falls into a local solution, and it is possible to obtain an optimal solution with higher accuracy. Next, the route generation unit 304 calculates the curvature change points of the clothoid curve using the global route, the dynamic predictive map, and the past route (performs the optimization calculation). That is, the route generation unit 304 calculates the curvature change points of the clothoid curve (performs the optimization calculation) using the above-mentioned method. poserr , C obstacle , C oscillation ) is used to find the curvature at, for example, three curvature change points (to generate a trajectory).

[0054] In S905, the path generation unit 304 of the control unit 130 generates control variables (velocity v and angular velocity ω) for the moving object 100 based on the generated trajectory. Any method may be used for the process of determining the control variables based on the generated trajectory.

[0055] In S906, the traveling control unit 305 of the control unit 130 controls the motors 122a and 122b using the control amount (speed v and angular velocity ω) determined by the path generation unit 304, and controls the traveling of the moving body 100.

[0056] In S907, the control unit 130 determines whether the final target position has been reached. If the control unit 130 determines that the final target position has not been reached, the process returns to S901 and the process is repeated. If the control unit 130 determines that the final target position has been reached, the control unit 130 ends this series of processes.

[0057] Next, we will explain the series of operations of the grid map generation process executed by the grid map generation unit 303. Note that this process is realized by the control unit 130 expanding a program stored in the storage device 135 into the memory device of the control unit 130 and executing it. Note that this process starts when the above-mentioned S902 is executed.

[0058] In S1001, the grid map generation unit 303 generates a static grid map using the positions of static obstacles obtained by obstacle recognition, as described with reference to Fig. 4. The grid map generation unit 303 generates a static accumulation map by accumulating the positions of static obstacles over time. To generate the static accumulation map, the grid map generation unit 303 uses a forgetting rate that is smaller than the forgetting rate used in the dynamic grid map.

[0059] In S1002, the grid map generation unit 303 generates a dynamic grid map using the dynamic obstacle positions obtained by obstacle recognition, as described with reference to Fig. 4. In addition, in S1003, the grid map generation unit 303 generates prediction information using the result of movement prediction, as described with reference to Fig. 4.

[0060] In S1004, the grid map generation unit 303 generates a dynamic prediction map using the static accumulation map, the dynamic grid map, and the prediction information, as described with reference to Fig. 5. After generating the dynamic prediction map, the grid map generation unit 303 returns the process to the caller (ends this process).

[0061] As described above, in the above-described embodiment, the mobile body 100 generates a dynamic predictive map including information indicating the positions of static obstacles recognized based on sensor information (e.g., image information) and information indicating the time-varying positions of dynamic obstacles recognized based on the sensor information. The generated dynamic predictive map is then used to generate a trajectory that serves as a target when controlling the travel of the mobile body. This makes it possible to generate a map that takes into account the presence of dynamic obstacles, thereby enabling control of the mobile body that takes into account the presence of dynamic obstacles. Furthermore, the mobile body 100 generates the dynamic predictive map using a first map that indicates the positions of static obstacles recognized based on sensor information, a second map that indicates the positions of dynamic obstacles recognized based on sensor information, and information on the time-series positions of the predicted dynamic obstacles. When generating a map that takes into account the presence of dynamic obstacles, it is possible to easily integrate the position information of the static obstacles, the position information of the dynamic obstacles, and the predicted trajectories of the dynamic obstacles. Furthermore, it is possible to generate a trajectory that avoids entering a position to which the dynamic obstacle is attempting to move.

[0062] This makes it possible to control the vehicle while taking into consideration safety and riding comfort, and reduces the load on the occupant due to acceleration and turning.

[0063] The configuration of the control unit 130 described above may function in various forms as a mobile object control system. For example, a mobile object control system may be configured in a form in which at least a part of the control unit 130 described above is configured on a device external to the mobile object 100, such as an external server. Alternatively, the mobile object control system may be the mobile object 100, or may be configured to be incorporated inside the mobile object 100 (i.e., the control unit 130). Furthermore, the computer program that operates the mobile object 100 described above may be a computer program that causes one or more computers to function as each means of the mobile object control system.

[0064] <Summary of the embodiment> The above description includes the following embodiments of a mobile object control system, a mobile object, a control method for a mobile object control system, a program, and a storage medium.

[0065] (Item 1) A mobile object control system (e.g., 100) for controlling the operation of a mobile object, Acquisition means (e.g., 302) for acquiring information from a sensor for recognizing the surroundings of the moving object; A map generating means (e.g., 303) that generates a dynamic predictive map (e.g., 430) including information indicating the positions of static obstacles recognized based on the information from the sensors and information indicating the positions of dynamic obstacles that change over time recognized based on the information from the sensors; A mobile object control system characterized by having a trajectory generation means (e.g., 304) that uses the generated dynamic predictive map to generate a target trajectory when controlling the travel of the mobile object.

[0066] According to this embodiment, it is possible to generate a map that takes into account the presence of dynamic obstacles, and also to control a moving object that takes into account the presence of dynamic obstacles.

[0067] (Item 2) 2. The mobile object control system according to claim 1, wherein the map generation means includes information on the time-series positions of the dynamic obstacles that are predicted in the dynamic predictive map as information indicating the positions of the dynamic obstacles that change over time.

[0068] According to this embodiment, it is possible to generate a trajectory that avoids entering a position where a dynamic obstacle is about to move.

[0069] (Item 3) The mobile object control system described in item 1 is characterized in that the map generation means generates the dynamic predictive map using a first map (e.g., 410) showing the positions of static obstacles recognized based on information from the sensor, a second map (e.g., 402) showing the positions of dynamic obstacles recognized based on information from the sensor, and information (e.g., 403) on the predicted time-series positions of the dynamic obstacles.

[0070] According to this embodiment, the positions of static obstacles, the positions of dynamic obstacles, and the predicted positions of dynamic obstacles can be handled in an integrated manner.

[0071] (Item 4) 2. The mobile object control system according to item 1, wherein the trajectory generation means generates, as the target trajectory, a trajectory that is constructed by combining a plurality of trajectories, each of which has a curvature that changes linearly with distance.

[0072] According to this embodiment, even if a dynamic obstacle is present, it is possible to avoid the obstacle by meandering left and right.

[0073] (Item 5) 4. The mobile body control system according to item 3, wherein the trajectory generation means generates a first trajectory of the mobile body, which is a reference path, and generates a second trajectory, which is configured by combining a plurality of trajectories based on the first trajectory, as the target trajectory.

[0074] According to this embodiment, it is possible to generate a second trajectory with a high degree of freedom using curvature with high accuracy and stability.

[0075] (Item 6) 6. The mobile object control system according to item 5, wherein the trajectory generation means generates a first trajectory for the mobile object based on the positions of static obstacles shown on the first map.

[0076] According to this embodiment, a stable route can be generated by using the static accumulation map 410 in which the positions of obstacles are stable.

[0077] (Item 7) 6. The mobile object control system according to item 5, wherein the trajectory generation means generates the second trajectory based on the first trajectory and a position of the dynamic obstacle included in the dynamic predictive map that changes over time.

[0078] According to this embodiment, it is possible to generate a trajectory that avoids entering a position to which a dynamic obstacle is about to move with high accuracy and stability.

[0079] (Item 8) 5. The mobile body control system according to item 4, wherein the trajectory generation means generates, as the target trajectory, a trajectory from the position of the mobile body to the target position, which is composed of a plurality of trajectories, each of which has a curvature that changes linearly with distance, and a curvature change point, the curvature of which changes nonlinearly.

[0080] According to this embodiment, even when a trajectory with a high degree of freedom is generated, the curvature of the trajectory changes smoothly, making it possible to ensure a comfortable ride for passengers riding on the moving body.

[0081] (Item 9) 9. The mobile object control system according to item 8, wherein the trajectory along which the mobile object should travel has three curvature change points.

[0082] According to this embodiment, by using three curvature change points, it is possible to generate a traveling trajectory with a high degree of freedom while suppressing the amount of calculation required for optimization.

[0083] (Item 11) A mobile object, an acquisition means for acquiring information from a sensor for recognizing the surroundings of the moving object; a map generating means for generating a dynamic predictive map including information indicating the positions of static obstacles recognized based on the information from the sensors and information indicating the positions of dynamic obstacles that change over time recognized based on the information from the sensors; a trajectory generation means for generating a trajectory to be used as a target when controlling the travel of the moving object, using the generated dynamic predictive map; a control means for controlling a drive device of the moving body so as to make the moving body travel along the target trajectory. A moving object characterized by having:

[0084] According to this embodiment, it is possible to generate a map that takes into account the presence of dynamic obstacles, and also to control a moving object that takes into account the presence of dynamic obstacles.

[0085] (Item 12) A control method for a mobile object control system that controls the operation of a mobile object, comprising: acquiring information from a sensor for recognizing the surroundings of the moving object; generating a dynamic predictive map including information indicating the positions of static obstacles recognized based on the information from the sensors and information indicating the positions of dynamic obstacles recognized based on the information from the sensors that change over time; and generating a trajectory that serves as a target when controlling the traveling of the moving body using the generated dynamic predictive map.

[0086] According to this embodiment, it is possible to generate a map that takes into account the presence of dynamic obstacles, and also to control a moving object that takes into account the presence of dynamic obstacles.

[0087] (Item 13) A program for causing a computer to function as each means of the mobile object control system according to any one of items 1 to 10.

[0088] According to this embodiment, it is possible to generate a map that takes into account the presence of dynamic obstacles, and also to control a moving object that takes into account the presence of dynamic obstacles.

[0089] (Item 14) A recording medium storing a program for causing a computer to function as each of the means of the mobile object control system according to any one of items 1 to 10.

[0090] According to this embodiment, it is possible to generate a map that takes into account the presence of dynamic obstacles, and also to control a moving object that takes into account the presence of dynamic obstacles.

[0091] The invention is not limited to the above-described embodiment, and various modifications and variations are possible within the scope of the gist of the invention. [Explanation of symbols]

[0092] 100... Mobile body, 120... Driving wheel, 121... Driven wheel, 130... Control unit, 303... Grid map generating unit, 304... Route generating unit, 305... Travel control unit

Claims

1. A mobile object control system for controlling the operation of a mobile object, an acquisition means for acquiring information from a sensor for recognizing the surroundings of the moving object; a map generating means for generating a dynamic predictive map including information indicating the positions of static obstacles recognized based on the information from the sensors and information indicating the positions of dynamic obstacles that change over time recognized based on the information from the sensors; and a trajectory generation means for generating a trajectory that serves as a target when controlling the travel of the moving body, using the generated dynamic predictive map.

2. 2. The mobile object control system according to claim 1, wherein the map generation means includes information on the time-series positions of the dynamic obstacles that are predicted in the dynamic prediction map as information indicating the positions of the dynamic obstacles that change over time.

3. 2. The mobile body control system according to claim 1, wherein the map generation means generates the dynamic predictive map using a first map indicating positions of static obstacles recognized based on information from the sensor, a second map indicating positions of dynamic obstacles recognized based on information from the sensor, and information on predicted positions of the dynamic obstacles over time.

4. 2. The mobile object control system according to claim 1, wherein the trajectory generation means generates, as the target trajectory, a trajectory that is configured by combining a plurality of trajectories, each of which has a curvature that changes linearly with distance.

5. 4. The mobile body control system according to claim 3, wherein the trajectory generation means generates a first trajectory of the mobile body, which is a reference path, and generates a second trajectory, which is constructed by combining a plurality of trajectories based on the first trajectory, as the target trajectory.

6. 6. The mobile object control system according to claim 5, wherein said trajectory generation means generates a first trajectory of said mobile object based on the positions of static obstacles shown on said first map.

7. 6. The mobile object control system according to claim 5, wherein the trajectory generation means generates the second trajectory based on the first trajectory and a time-varying position of the dynamic obstacle included in the dynamic predictive map.

8. 5. The mobile body control system according to claim 4, wherein the trajectory generation means generates, as the target trajectory, a trajectory from the position of the mobile body to the target position, which is constructed by combining a plurality of trajectories, each of which has a curvature that changes linearly with distance, and a curvature change point, the curvature of which changes nonlinearly.

9. 9. The mobile object control system according to claim 8, wherein the trajectory along which the mobile object is to travel has three curvature change points.

10. the sensor information is image information captured by an imaging means of the moving body, 2. The mobile object control system according to claim 1, further comprising a prediction means for predicting a position of said dynamic obstacle that changes over time based on said image information.

11. A mobile object, an acquisition means for acquiring information from a sensor for recognizing the surroundings of the moving object; a map generating means for generating a dynamic predictive map including information indicating the positions of static obstacles recognized based on the information from the sensors and information indicating the positions of dynamic obstacles that change over time recognized based on the information from the sensors; a trajectory generation means for generating a trajectory to be used as a target when controlling the travel of the moving object, using the generated dynamic predictive map; a control means for controlling a drive device of the moving body so as to make the moving body travel along the target trajectory. A moving object characterized by having:

12. A control method for a mobile object control system for controlling the operation of a mobile object, comprising: acquiring information from a sensor for recognizing the surroundings of the moving object; generating a dynamic predictive map including information indicating the positions of static obstacles recognized based on the information from the sensors and information indicating the positions of dynamic obstacles recognized based on the information from the sensors that change over time; and generating a trajectory that serves as a target when controlling the traveling of the moving body using the generated dynamic predictive map.

13. A program for causing a computer to function as each of the means of the mobile object control system according to any one of claims 1 to 10.

14. A recording medium storing a program for causing a computer to function as each of the means of the mobile object control system according to any one of claims 1 to 10.

Images