Mobile object control system, control method thereof, program, and mobile object

Abstract

To appropriately generate a course of a mobile without using a high-precision map.SOLUTION: A mobile control system acquires a picked-up image, detects an obstacle included in the picked-up image, divides regions around a mobile, and generates an occupation map indicating the occupation of the obstacle which is detected for each divided region. The mobile control system generates a macro course, avoiding the detected obstacle, from a current location to a target location based on a first cost in which a cost becomes higher as a distance from the current location in the occupation map is longer, a second cost in which a cost becomes higher as a distance from the target location in the occupation map is longer, and a third cost in which a cost becomes higher as a distance from a past course in the occupation map is longer.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to a mobile object control system, a control method therefor, a program, and a mobile object. [Background technology]

[0002] In recent years, small mobile objects such as electric vehicles with a seating capacity of one to two people, called ultra-compact mobility (also called micromobility), and mobile robots that provide various services to people have become known. Some of these mobile objects can travel autonomously while periodically generating a route to a destination.

[0003] Patent Document 1 discloses generating a first trajectory based on map and route information, optimizing the route and speed of the first trajectory based on obstacle information, etc., and generating a second trajectory for controlling an autonomous vehicle based on the optimized route and speed. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-128962 Summary of the Invention [Problem to be solved by the invention]

[0005] In the above-mentioned conventional technology, route planning is performed based on highly accurate map information provided by a server. However, small mobile objects have limited hardware resources, making it difficult to secure an area to store such highly accurate map information or a communication device that can quickly acquire large amounts of map information. Furthermore, in areas where small mobile objects travel, highly accurate maps may not be available, so it is necessary to generate routes without using such map information.

[0006] On the other hand, it is known to use cost functions based on the distance to the destination and the distance to obstacles to optimize routes. While such cost functions can generate routes that avoid obstacles, they have the problem of requiring a wide search range and increasing the amount of processing. For small mobile devices with limited hardware resources, it is extremely important to reduce the processing load when generating routes periodically in real time.

[0007] The present invention has been made in view of the above-mentioned problems, and aims to preferably generate a route for a moving object without using a high-precision map. [Means for solving the problem]

[0008] According to the present invention, for example, a mobile body is characterized by comprising an acquisition means for acquiring an image, a detection means for detecting obstacles included in the image, a map generation means for dividing an area around the mobile body and generating an occupancy map showing the occupancy of the obstacles detected by the detection means for each divided area, and a route generation means for generating a global route from the current position to the target position that avoids the detected obstacles based on a first cost, which increases the cost the farther the distance from the current position in the occupancy map, a second cost, which increases the cost the farther the distance from the target position in the occupancy map, and a third cost, which increases the cost the farther the distance from a past route in the occupancy map. [Effects of the Invention]

[0009] According to the present invention, it is possible to preferably generate a route for a moving object without using a high-precision map, thereby reducing the amount of processing. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing an example of the hardware configuration of a mobile body according to an embodiment of the present invention; [Figure 2] FIG. 1 is a block diagram showing a control configuration of a moving body according to an embodiment of the present invention. [Figure 3] FIG. 1 is a block diagram showing the functional configuration of a moving body according to an embodiment of the present invention; [Figure 4] FIG. 10 is a diagram showing an occupancy grid map according to the present embodiment; [Figure 5] FIG. 10 is a diagram showing a method for generating an occupancy grid map according to the present embodiment. [Figure 6] FIG. 1 shows global routes and local routes according to the present embodiment. [Figure 7] FIG. 10 is a diagram showing a method for generating a global route according to the present embodiment. [Figure 8] FIG. 1 illustrates a method for optimizing a global route according to an embodiment of the present invention. [Figure 9] A flowchart showing a processing procedure for controlling the travel of a moving body according to the present embodiment. [Figure 10] A flowchart showing a detailed processing procedure for generating a route according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention as claimed, and not all combinations of features described in the embodiments are necessarily essential to the invention. Two or more of the features described in the embodiments may be arbitrarily combined. Furthermore, the same reference numerals are used for the same or similar components, and redundant explanations will be omitted.

[0012] <Configuration of moving body> The configuration of a moving body 100 according to this embodiment will be described with reference to Fig. 1. Fig. 1(A) shows a side view of the moving body 100 according to this embodiment, and Fig. 1(B) shows the internal configuration of the moving body 100. In the figure, arrow X indicates the front-to-rear direction of the moving body 100, with F indicating the front and R indicating the rear. Arrows Y and Z indicate the width direction (left-to-right direction) and up-down direction of the moving body 100.

[0013] The moving body 100 is equipped with a battery 113 and is, for example, an ultra-compact mobility vehicle that moves mainly by motor power. An ultra-compact mobility vehicle is a vehicle that is more compact than a typical automobile and has a passenger capacity of approximately one or two people. In this embodiment, a three-wheeled ultra-compact mobility vehicle is described as an example of the moving body 100, but this is not intended to limit the present invention, and the moving body 100 may also be, for example, a four-wheeled vehicle or a saddle-ride vehicle. Furthermore, the vehicle of the present invention is not limited to a vehicle, but may also be a vehicle that carries luggage and runs alongside a person walking, or a vehicle that leads a person. Furthermore, the present invention is not limited to four-wheeled or two-wheeled vehicles, and may also be applicable to walking robots that are capable of autonomous movement.

[0014] The mobile object 100 is an electric autonomous vehicle equipped with a propulsion unit 112 and using a battery 113 as its main power source. The battery 113 is a secondary battery such as a lithium-ion battery, and the mobile object 100 is propelled by the propulsion unit 112 using power supplied from the battery 113. The propulsion unit 112 is a tricycle equipped with a pair of front wheels 120 on the left and right and a tail wheel (driven wheel) 121. The propulsion unit 112 may be in another form, such as a four-wheeled vehicle. The mobile object 100 is equipped with a seat 111 for one or two people.

[0015] The traveling unit 112 includes a steering mechanism 123. The steering mechanism 123 is a mechanism that uses motors 122a and 122b as drive sources to change the steering angle of the pair of front wheels 120. By changing the steering angle of the pair of front wheels 120, the traveling direction of the mobile body 100 can be changed. The tail wheel 121 does not have its own drive source, but is a driven wheel that operates in response to the drive of the pair of front wheels 120. The tail wheel 121 is also connected to the body of the mobile body 100 via a swivel unit. The swivel unit rotates so that the direction of the tail wheel 121 changes separately from the rotation of the tail wheel 121. In this way, the mobile body 100 according to this embodiment employs a differential two-wheel mobility with a tail wheel, but is not limited to this.

[0016] The moving body 100 is equipped with a detection unit 114 that recognizes a plane in front of the moving body 100. The detection unit 114 is an external sensor that monitors the front of the moving body 100, and in this embodiment, is an imaging device that captures an image in front of the moving body 100. In this embodiment, the detection unit 114 will be described as an example of a stereo camera having an optical system such as two lenses and respective image sensors. However, it is also possible to adopt a radar or a lidar (Light Detection and Ranging) instead of or in addition to the imaging device. Furthermore, in this embodiment, an example in which the detection unit is provided only in front of the moving body 100 will be described, but this is not intended to limit the present invention, and the detection unit 114 may be provided behind, or on the left or right of the moving body 100.

[0017] The moving body 100 according to this embodiment captures an image of the area ahead of the moving body 100 using the detection unit 114, and detects obstacles from the captured image. Furthermore, the moving body 100 divides the area around the moving body 100 into a grid, and controls its travel while generating an occupancy grid map in which obstacle information is accumulated in each grid (hereinafter also referred to as a grid). The occupancy grid map will be described in detail later.

[0018] <Control structure of moving object> FIG. 2 is a block diagram of a control system of the mobile object 100 according to this embodiment. The following description focuses on the components necessary for implementing the present invention. Therefore, other components may be included in addition to the components described below. Furthermore, in this embodiment, the mobile object 100 is described as including each of the components described below. However, this is not intended to limit the present invention, and the mobile object 100 may be realized as a mobile object control system including multiple devices. For example, some functions of the control unit 130 may be implemented by a communicatively connected server device, or the detection unit 114 and the GNSS sensor 134 may be provided as external devices. The mobile object 100 includes a control unit (ECU) 130. The control unit 130 includes a processor, such as a CPU, a storage device, such as a semiconductor memory, an interface with an external device, and the like. The storage device stores programs executed by the processor, data used by the processor for processing, and the like. Multiple sets of processors, storage devices, and interfaces may be provided for different functions of the mobile object 100 and configured to communicate with each other.

[0019] The control unit 130 acquires the detection results of the detection unit 114, input information from the operation panel 131, audio information input from the audio input device 133, position information from the GNSS sensor 134, and information received via the communication unit 136, and executes corresponding processing. The control unit 130 controls the motors 122a and 122b (travel control of the traveling unit 112), controls the display on the operation panel 131, and notifies and outputs information to the occupants of the moving object 100 by audio from the speaker 132.

[0020] The voice input device 133 collects the voices of the occupants of the moving body 100. The control unit 130 can recognize the input voices and execute corresponding processing. The GNSS (Global Navigation Satellite system) sensor 134 receives GNSS signals to detect the current position of the moving body 100. The storage device 135 is a storage device that stores images captured by the detection unit 114, obstacle information, previously generated routes, occupancy grid maps, etc. The storage device 135 may also store programs executed by the processor, data used by the processor for processing, etc. The storage device 135 may store various parameters of machine learning models for voice recognition and image recognition executed by the control unit 130 (for example, trained parameters and hyperparameters of a deep neural network, etc.).

[0021] The communication unit 136 communicates with a communication device 140, which is an external device, via wireless communication such as Wi-Fi or fifth generation mobile communication. The communication device 140 is, for example, a smartphone, but is not limited to this and may also be an earphone-type communication terminal, a personal computer, a tablet terminal, a game console, or the like. The communication device 140 connects to a network via wireless communication such as Wi-Fi or fifth generation mobile communication.

[0022] A user who owns the communication device 140 can give instructions to the mobile body 100 via the communication device 140. The instructions include, for example, an instruction to call the mobile body 100 to a location desired by the user and join the mobile body 100. Upon receiving the instruction, the mobile body 100 sets a target position based on the position information included in the instruction. In addition to such instructions, the mobile body 100 can also set a target position from an image captured by the detection unit 114 or based on an instruction from a user riding in the mobile body 100 via the operation panel 131. When setting a target position from a captured image, for example, a person raising their hand toward the mobile body 100 is detected in the captured image, and the position of the detected person is estimated and set as the target position.

[0023] <Functional configuration of mobile units> Next, the functional configuration of the moving body 100 according to this embodiment will be described with reference to Fig. 3. The functional configuration described here is realized in the control unit 130, for example, by the CPU reading a program stored in a memory such as a ROM into the RAM and executing it. Note that the functional configuration described below describes only the functions necessary for explaining the present invention, and does not describe all of the functional configuration actually included in the moving body 100. In other words, the functional configuration of the moving body 100 according to the present invention is not limited to the functional configuration described below.

[0024] The user instruction acquisition unit 301 has a function of receiving instructions from the user, and can receive user instructions via the operation panel 131, user instructions from an external device such as the communication device 140 via the communication unit 136, and user spoken instructions via the voice input device 133. As described above, user instructions include instructions to set a target position (also referred to as a destination) of the moving body 100 and instructions related to driving control of the moving body 100.

[0025] The image information processing unit 302 processes the captured images acquired by the detection unit 114. Specifically, the image information processing unit 302 creates a depth image from the stereo images acquired by the detection unit 114 and converts it into a 3D point cloud. The 3D point cloud image data is used to detect obstacles that impede the travel of the mobile object 100. The image information processing unit 302 may also include a machine learning model that processes image information and execute learning stage processing and inference stage processing of the machine learning model. The machine learning model of the image information processing unit 302 can perform processing to recognize three-dimensional objects, etc. included in the image information, for example, by performing calculations of a deep learning algorithm using a deep neural network (DNN).

[0026] The grid map generator 303 creates a grid map of a predetermined size (for example, each cell is 10cm x 10cm in a 20m x 20m area) based on the image data of the 3D point cloud. This is done to reduce the size of the grid, since the amount of data in the 3D point cloud is large and real-time processing is difficult. The grid map includes, for example, a grid map showing the difference between the maximum and minimum heights of the point cloud within the grid (indicating whether the cell is a step) and a grid map showing the maximum height of the point cloud within the grid from a reference point (indicating the topographical shape of the cell). Furthermore, the grid map generator 303 removes spike noise and white noise from the generated grid map, detects obstacles above a predetermined height, and generates an occupancy grid map that indicates whether or not there are any three-dimensional obstacles in each grid.

[0027] The route generation unit 304 generates a travel route for the mobile object 100 to the target position set by the user instruction acquisition unit 301. Specifically, the route generation unit 304 generates a route using an occupancy grid map generated by the grid map generation unit 303 from images captured by the detection unit 114, without requiring obstacle information from a high-precision map. Note that the detection unit 114 is a stereo camera that captures images of the area ahead of the mobile object 100, and therefore cannot recognize obstacles in other directions. Therefore, it is desirable for the mobile object 100 to store detected obstacle information for a predetermined period of time in order to avoid colliding with obstacles outside the field of view or getting stuck in a dead end. This allows the mobile object 100 to generate a route taking into account both obstacles detected in the past and obstacles detected in real time.

[0028] The path generation unit 304 periodically generates a global path using the occupancy grid map, and also periodically generates local paths that follow the global path. In other words, the target position of the local path is determined by the global path. In this embodiment, the generation period for each path is 100 ms for the global path and 50 ms for the local path, but this is not a limitation of the present invention. Various algorithms are known for generating global paths, such as Rapid-Exploring Random Tree (RRT), Probabilistic Road Map (PRM), and A*. The path generation unit 304 according to this embodiment is based on the A* algorithm, taking into account compatibility and reproducibility when grid cells are treated as nodes, and uses an improved method of this algorithm to further reduce the amount of calculation. Details of this method will be described later. Furthermore, because the vehicle 100 employs a differential two-wheel mobility system with a tail wheel, the path generation unit 304 generates local paths that take into account the tail wheel 121, which is a driven wheel.

[0029] The travel control unit 305 controls the travel of the mobile object 100 according to the local route. Specifically, the travel control unit 305 controls the travel unit 112 according to the local route to control the speed and angular velocity of the mobile object 100. Furthermore, the travel control unit 305 controls the travel in accordance with various operations by the driver. When a deviation occurs in the driving plan of the local route due to an operation by the driver, the travel control unit 305 may again acquire a new local route generated by the route generation unit 304 and control the travel, or may control the speed and angular velocity of the mobile object 100 so as to eliminate the deviation from the local route currently in use.

[0030] <Occupancy grid map> FIG. 4 shows an occupancy grid map 400 including obstacle information according to this embodiment. Since the mobile body 100 according to this embodiment travels without relying on obstacle information from a high-precision map, all obstacle information is obtained from the recognition results of the detection unit 114. At this time, it is necessary to store the obstacle information to avoid collisions with obstacles outside the field of view or getting stuck in dead ends. Therefore, in this embodiment, an occupancy grid map is used as a method of storing obstacle information from the perspectives of reducing the amount of information in the 3D point cloud of stereo images and making it easier to handle in route planning.

[0031] The grid map generator 303 according to this embodiment divides the area surrounding the mobile object 100 into a grid and generates an occupancy grid map containing information indicating the presence or absence of obstacles for each grid. While an example of dividing a predetermined area into a grid is described here, it is also possible to divide the area into other shapes rather than a grid, and create an occupancy map indicating the presence or absence of obstacles for each divided area. The occupancy grid map 400 defines a surrounding area around the mobile object 100 as a 40m x 40m or 20m x 20m area, for example, and divides this area into 20cm x 20cm or 10cm x 10cm grids, which are dynamically set in response to the movement of the mobile object 100. In other words, the occupancy grid map 400 is a region that shifts in real time as the mobile object 100 moves, always centering the mobile object 100. The size of the regions can be set arbitrarily depending on the hardware resources of the mobile object 100.

[0032] Additionally, in the occupancy grid map 400, information on the presence or absence of obstacles detected from the image captured by the detection unit 114 is defined for each grid. For example, the presence or absence information is defined as "0" for a drivable area and "1" for an undrivable area (i.e., the presence of an obstacle). In FIG. 4, 401 indicates a grid in which an obstacle exists. An area in which an obstacle exists indicates an area that the mobile object 100 cannot pass through, and is formed of, for example, a solid object of 5 cm or more in size. Therefore, the mobile object 100 generates a route so as to avoid these obstacles 401.

[0033] <Storage of obstacle information> FIG. 5 illustrates the accumulation of obstacle information in an occupied grid map according to this embodiment. Reference numeral 500 denotes a local map that moves in accordance with the movement of the moving object 100. The local map 500 is shifted in accordance with the movement of the moving object 100 in the x-axis and y-axis directions on the grid map. The local map 500 shows, for example, how a dotted area 501 is deleted and a solid area 502 is added in accordance with the amount of movement Δx of the moving object 100 in the x-axis direction. The deleted area is an area opposite to the moving direction of the moving object 100, and the added area is an area in the same moving direction. Similarly, areas are deleted and added in the y-axis direction in accordance with the movement of the moving object 100. The local map 500 also accumulates information on obstacles detected in the past. Note that if an obstacle is present in a grid included in the deletion area, the obstacle information is deleted from the local map 500, but it is desirable to retain the information separately from the local map 500 for a certain period of time. Such information is useful, for example, when the moving body 100 changes its course and the deleted area is again included in the local map 500, and can improve the accuracy of obstacle avoidance by the moving body 100. Furthermore, by using the accumulated information, it is not necessary to detect obstacles again, and the processing load can be reduced.

[0034] Reference numeral 510 denotes an obstacle detection map showing detection information of obstacles present ahead of the mobile object 100 from captured images captured by the detection unit 114 of the mobile object 100. The obstacle detection map 510 shows real-time information and is periodically generated according to the captured images acquired by the detection unit 114. Note that, since moving obstacles such as people and vehicles are also expected, it is desirable to update the obstacle detection map 510 periodically generated, rather than storing previously detected obstacles in the field of view 511 of the detection unit 114, which is the area ahead of the mobile object 100. This makes it possible to recognize moving obstacles and prevent the generation of routes that avoid obstacles more than necessary. On the other hand, in the area behind the mobile object 100 (strictly speaking, outside the field of view of the detection unit 114), previously detected obstacles are stored as shown in the local map 500. This makes it possible, for example, when an obstacle is detected in the forward area and a detour is generated, to easily generate a route that avoids collisions with the obstacle that has passed through.

[0035] Reference numeral 520 denotes an occupancy grid map generated by adding together the local map 500 and the obstacle detection map 510. In this way, the occupancy grid map 520 is generated as a grid map that combines the local map and obstacle detection information that change in real time with obstacle information that has been detected and accumulated in the past.

[0036] <Route generation> 6 shows a travel path generated by the moving body 100 according to this embodiment. The path generation unit 304 according to this embodiment periodically generates a global path 602 using an occupancy grid map according to a set target position 601, and further periodically generates a local path 603 so as to follow the global path. The methods for generating the global path and the local path will be described later.

[0037] The target position 601 is set based on various instructions. For example, these include instructions from a passenger aboard the vehicle 100 or instructions from a user outside the vehicle 100. Instructions from the passenger are given via the operation panel 131 or the voice input device 133. Instructions given via the operation panel 131 may be given by specifying a specific grid on a grid map displayed on the operation panel 131. In this case, the size of each grid may be set large so that a wider area of the map can be selected. Instructions given via the voice input device 133 may be given using surrounding landmarks as landmarks. The landmarks may include passersby, signs, road signs, outdoor facilities such as vending machines, building components such as windows and entrances, roads, vehicles, motorcycles, and the like, included in the spoken information. Upon receiving an instruction via the voice input device 133, the path generation unit 304 detects the specified landmark from the captured image acquired by the detection unit 114 and sets it as the target position.

[0038] Machine learning models are used for these speech and image recognition tasks. Machine learning models, for example, use deep learning algorithms using deep neural networks (DNNs) to recognize place names, landmark names such as buildings, store names, and landmark names contained in speech and image information. Speech recognition DNNs become trained through training phase processing, and by inputting new speech information into the trained DNN, recognition processing for the new speech information (inference phase processing) can be performed. Image recognition DNNs can also recognize passersby, signs, road signs, outdoor facilities such as vending machines, building components such as windows and entrances, roads, vehicles, and motorcycles contained in images.

[0039] In addition, instructions from a user outside the vehicle 100 can be sent via the user's own communication device 140 to the vehicle 100 via the communication unit 136, or the user can call the vehicle 100 by raising their hand toward the vehicle 100 as shown in Fig. 6. Instructions from the communication device 140 are given by operation input or voice input, similar to the instructions from the passengers.

[0040] <Global route generation method> A global path generation method according to this embodiment will be described below with reference to Figures 7 and 8. In this embodiment, the global path planning is performed by extending the A* algorithm (improved A*), which has optimality and reproducibility under static conditions. Furthermore, the improved A* is optimized using Theta* to generate a path that is independent of the grid.

[0041] (Improved type A*) FIG. 7 shows an improved A* algorithm according to this embodiment. The A* algorithm treats cells on a grid as nodes and performs a full search every cycle, which has the advantage of being able to handle new obstacles and changes in the target position. However, the drawback of A*'s full search every cycle is that it requires a large amount of calculation and is susceptible to significant path changes due to noise, which can cause wandering. In A*, the expansion of search nodes is determined based on the actual travel cost and a heuristic function. The heuristic function uses the estimated distance to the target position. Here, according to this embodiment, search nodes are determined using previously planned route information (e.g., previously generated route information). As such, according to this embodiment, improved A* takes past routes into account, thereby reducing the search area and suppressing large-scale route changes (wandering).

[0042] Reference numeral 700 denotes a cost map that defines a first cost for each grid in a grid map, with the cost increasing as the distance from the current location of the mobile object 100 increases. In other words, the cost map 700 defines the movement cost of the mobile object 100 for each grid and is generated every cycle. Reference numeral 701 denotes the current location of the mobile object 100 and serves as the start position of the global route. Reference numeral 702 denotes the end position of the global route toward the target location of the route to be generated. The end position may be the final target location or a relay location along the way to the target location. Furthermore, a "large" square in the figure indicates that the cost increases toward that location, and a "small" square indicates that the cost decreases toward that location. "∞" indicates that the cost of a grid containing an obstacle is set to infinity. Therefore, the cost map defines a first cost that takes obstacles into account.

[0043] Reference numeral 710 denotes a heuristic map that defines a second cost for each grid in the grid map, where the cost increases as the distance from the target position increases. In other words, the heuristic map 710 defines the estimated distance from the target position for each grid. In 710, since the target position is set to the upper right, the cost is defined to be lower as the target position approaches the upper right and higher as the target position approaches the lower left.

[0044] Reference numeral 720 denotes a grid map used to determine search nodes, and indicates a past path map that defines a third cost for each grid in which the cost increases the farther the distance from the past path. In other words, the past path map 720 defines the presence or absence of a past path for each grid. Reference numeral 721 indicates a past path. In this embodiment, the past path 721 indicates the global path generated last time. However, this is not intended to limit the present invention, and the cumulative path of the past several past paths may be used. The past path map 720 is generated by assigning "0" to grids that have been passed through by past paths and "1" to grids that have not been passed through, and then applying a Gaussian filter, averaging filter, or the like to the grid map.

[0045] Using these grid maps, the search node i* of the improved A* according to this embodiment can be determined by the evaluation function shown in the following Equation 1.

[0046]

number

[0047] Here, OPEN indicates a set of indexes of grid cells included in the OPENLIST of A*. Ci, Hi, and Pi indicate the values of the first cost, second cost, and third cost, respectively. k indicates a coefficient. The above formula 1 makes it possible to search preferentially in the vicinity of the area passed by the previous path 721 within the OPENLIST. If the route generation unit 304 cannot find a route to the target position even after searching the area passed by the previous path 721, it expands the search range in the same way as A* and continues searching until a route is found.

[0048] Reference numeral 730 denotes a global path 731 generated by applying such an improved A* algorithm so as to avoid obstacles. As described above, the improved A* algorithm applied in this embodiment does not require a full search because it takes past paths into consideration, and the search range can be significantly reduced. Furthermore, as shown in Equation 1, the calculation of the evaluation function involves only matrix addition, which reduces the amount of calculation. Furthermore, by setting the cost of a grid in which an obstacle exists to "∞," it becomes difficult to generate a path that passes just past the obstacle.

[0049] (Theta*) FIG. 8 shows a method for optimizing a global path generated by the improved A*. 800 indicates a global path 731 generated by the improved A*. 801 indicates a node corresponding to each grid. The path generated by the improved A* is node-dependent, as shown in the global path 731, and may not be an optimal path even between grids where there are no obstacles. Therefore, according to this embodiment, the path is optimized by applying the Theta* algorithm to the global path 731 generated by the improved A*.

[0050] Reference numeral 810 denotes a global path 811 obtained by optimizing the global path 731 by applying Theta*. It can be seen that the global path 811, indicated by the dotted line, is a straighter path than the global path 731, indicated by the solid line, generated by the improved A*. For example, the global path 731 before optimization forms a trajectory that follows the obstacle 401 forward and backward around node 812, and multiple left and right steering operations are required for driving control. On the other hand, the global path 811 after optimization forms a trajectory that is straight forward and backward around node 812, and is the shortest path with the fewest steering operations.

[0051] 820 shows the node search algorithms of the improved A* and Theta* in the dotted-line area of 810. In the improved A*, as shown by the solid arrow 821, a trajectory is generated by searching from the parent node to the child node for a node that minimizes the distance, i.e., by searching in the forward direction. On the other hand, in the optimization using Theta*, as shown by the dotted arrow 822, the global path 731 is optimized in the reverse direction so as to minimize the distance from the child node to the parent node. In other words, Theta* searches for the optimal parent node. This makes it possible to generate an optimal path that is independent of the grid (node). In addition, since the optimal parent node is searched for again based on the travel cost, excessive reliance on past paths can be prevented.

[0052] <Local path generation method> Next, a local path generation method will be described. The path generation unit 304 generates a local path so as to follow the generated global path. Various local path planning methods exist, including Dynamic Window Approach (DWA), Model Predictive Control (MPC), clothoid tentacles, and Proportional-Integral-Differential (PID) control. In this embodiment, DWA is used as an example, but this is not intended to limit the present invention, and other methods may be used. DWA is widely used because it can take into account constraints such as kinematics and acceleration. The vehicle 100 according to this embodiment is a differential two-wheel mobility vehicle with a tail wheel. Since it is intended for passengers, it is classified as relatively large even among compact mobility vehicles. Therefore, the angle of the tail wheel 121 significantly affects the motion of the mobility vehicle. In the case of DWA using a conventional differential two-wheel model, the target trajectory differs from the actual trajectory, which can be dangerous. Therefore, in this embodiment, DWA is extended to introduce a constraint on the tail wheel angle.

[0053] The tail wheel 121 is a driven wheel, but when the angle of the tail wheel 121 differs significantly from the traveling direction of the mobile body 100, the reaction force from the ground due to the tail wheel increases, and when the direction of the tail wheel returns to the traveling direction, the reaction force suddenly decreases. At this time, an angular velocity in the yaw direction according to the direction of the tail wheel 121 occurs, causing the movement of the vehicle to become significantly disturbed and deviating significantly from the trajectory predicted by DWA. To prevent a collision caused by this, a constraint related to the tail wheel angle is introduced in addition to the constraints of the conventional DWA. First, the tail wheel angle of the differential two-wheel mobility is estimated using the following equation (2). δ=-arctan(Lω / v) Formula (2) Here, δ indicates the angle of the tail wheel 121 (tail wheel angle), v indicates the speed of the moving body 100, ω indicates the angular velocity, and L indicates the wheelbase.

[0054] DWA is an algorithm that determines the optimal combination of speed and acceleration from a speed and angular velocity range (window) that takes into account speed constraints, acceleration constraints, and collision constraints. In addition, this embodiment introduces a constraint based on the tail wheel angle. When the tail wheel angle differs from the traveling direction, a high speed results in a tail wheel reaction force. Therefore, the speed and angular velocity are limited according to the tail wheel angle. For example, the speed and angular velocity constraint ranges (maximum and minimum speed values and maximum and minimum angular velocity values) are set so that the vehicle travels at a low speed until the tail wheel angle returns to the same direction as the traveling direction, and then travels at maximum speed after the tail wheel 121 follows the traveling direction. This prevents the vehicle from being disturbed by a sudden change in tail wheel angle and enables continuous movement.

[0055] <Basic control of moving objects> 9 is a flowchart showing the basic control of the moving body 100 according to this embodiment. The processing described below is realized in the control unit 130, for example, by the CPU reading a program stored in a memory such as a ROM into the RAM and executing it.

[0056] In S101, the control unit 130 sets a target position for the moving object 100 based on a user instruction received by the user instruction acquisition unit 301. As described above, the user instruction can be received in various ways. Next, in S102, the control unit 130 captures an image of the area in front of the moving object 100 using the detection unit 114 and acquires the captured image. The acquired captured image is processed by the image information processing unit 302, and a depth image is created and converted into a 3D point cloud. In S103, the control unit 130 detects obstacles, for example, three-dimensional objects of 5 cm or more, from the 3D point cloud image. In S104, the control unit 130 generates an occupancy grid map of a predetermined area centered on the moving object 100, based on the detected obstacles and the position information of the moving object 100.

[0057] Next, in S105, the control unit 130 generates a driving route for the mobile object 100 using the route generation unit 304. As described above, the route generation unit 304 generates a global route using the occupancy grid map and the first to third costs, and generates local routes according to the generated global route. Subsequently, in S106, the control unit 130 determines the speed and angular velocity of the mobile object 100 according to the generated local route, and controls the traveling. Thereafter, in S107, the control unit 130 determines whether the mobile object 100 has reached the target position based on position information from the GNSS sensor 134. If the target position has not been reached, the process returns to S102, and the process of generating a route while updating the occupancy grid map and controlling the traveling is repeated. On the other hand, if the target position has been reached, the process of this flowchart ends.

[0058] <Route generation control> 10 is a flowchart showing the detailed processing procedure of the path generation control (S105) according to this embodiment. The processing described below is realized in the control unit 130, for example, by the CPU reading a program stored in a memory such as a ROM into the RAM and executing it.

[0059] In S201, the control unit 130 generates a first cost map that defines, for each grid, a first cost that increases the farther the distance from the current position of the mobile body 100. Next, in S202, the control unit 130 generates a second cost map that defines, for each grid, a second cost that increases the farther the distance from the target position. Furthermore, in S203, the control unit 130 generates a third cost map that defines, for each grid, a third cost that increases the farther the distance from the past route.

[0060] Next, in S204, the control unit 130 generates a global path according to the improved A* algorithm using the first to third cost maps generated in S201 to S203 and the information on the occupancy grid map generated in S104. Next, in S205, the control unit 130 optimizes the generated global path using the Theta* algorithm to generate an optimized global path. Thereafter, in S206, the control unit 130 generates local paths that follow the global path optimized in S205, and ends the processing of this flowchart. Note that although the flow of generating a global path and then a local path has been described, the paths are not necessarily generated in that order. This is because the generation cycle of the global path and the generation cycle of the local path are different. For example, if the generation cycle of the global path is 100 ms and the generation cycle of the local area is 50 ms, local paths will be generated twice according to the generated global path.

[0061] <Summary of the embodiment> 1. The mobile object control system (e.g., 100) of the above embodiment includes: An acquisition means for acquiring a captured image; (114) A detection means for detecting an obstacle included in the captured image (130, 302, 303); a map generating means for dividing an area around the moving object and generating an occupancy map for each divided area showing the occupancy of the obstacles detected by the detecting means; and (303) The method further comprises a route generation means for generating a global route from the current position to the target position that avoids the detected obstacle based on a first cost, the cost of which increases as the distance from the current position increases on the occupied map, a second cost, the cost of which increases as the distance from the target position increases on the occupied map, and a third cost, the cost of which increases as the distance from a past route increases on the occupied map (304, S201 to S204, FIG. 7).

[0062] According to this embodiment, in real-time route planning on an occupied map, grids where route planning has been performed in the past are searched preferentially, so a pseudo-potential map is generated so that locations where paths have been performed are lowered, and this is used in the evaluation function. Therefore, according to the present invention, routes for moving objects can be suitably generated without using a high-precision map.

[0063] 2. In the mobile object control system of the above embodiment, the route generation means further optimizes the generated global route using the Theta* algorithm, which searches for a parent node with the shortest distance (304, S206, FIG. 8).

[0064] According to this embodiment, a linear path can be generated without depending on nodes on the grid map.

[0065] 3. In the mobile object control system of the above embodiment, the third cost is generated using a Gaussian filter or an averaging filter (720, S203).

[0066] According to this embodiment, costs based on past paths can be easily generated.

[0067] 4. In the mobile object control system of the above embodiment, the route generation means obtains the third cost by using a filter along the past route (720, S203).

[0068] According to this embodiment, past paths can be used more efficiently.

[0069] 5. In the mobile object control system of the above embodiment, the past route is the route previously generated by the route generation means.

[0070] According to this embodiment, the search range can be further reduced by following the previous path.

[0071] 6. In the moving body of the above embodiment, the first cost is further determined based on the obstacle detected by the detection means (700, S201).

[0072] According to this embodiment, it is possible to avoid generating a path that passes close to an obstacle.

[0073] 7. In the mobile object control system of the above embodiment, the acquisition means acquires a captured image of a region in front of the mobile object (S102), The map generating means generates the occupancy map using information on obstacles in the area not acquired by the acquiring means that have been detected in the past by the detecting means (S103, S104, FIG. 5).

[0074] According to this embodiment, even when the moving body turns around and turns back, it is possible to use information about obstacles detected in the past, and to avoid collisions with those obstacles or getting stuck in a dead end.

[0075] 8. In the mobile object control system of the above embodiment, the route generation means A local route of the mobile object is further generated based on a dynamic window approach (DWA) so as to follow the global route (S206).

[0076] According to this embodiment, it is possible to perform driving control taking into account the direction of the driven wheels.

[0077] 9. The moving object control system of the above embodiment further includes a travel control means for determining the speed and angular velocity of the moving object based on the local route and controlling the travel (S106).

[0078] According to this embodiment, travel control can be performed taking into account the direction of the driven wheels.

[0079] 10. In the mobile object control system of the above embodiment, the route generation means generates the global route and the local route at different cycles.

[0080] According to this embodiment, it is possible to reduce unnecessary route generation and reduce the processing load.

[0081] 11. In the mobile object control system of the above embodiment, the generation period of the local route is shorter than the generation period of the global route.

[0082] According to this embodiment, it is possible to reduce unnecessary route generation and reduce the processing load.

[0083] 12. In the mobile object control system of the above embodiment, the acquisition means is a stereo camera (114); The detection means converts image data of the stereo images captured by the stereo cameras into a cubic point cloud (S103).

[0084] According to this embodiment, the amount of processing can be reduced, and real-time processing can be suitably realized.

[0085] 13. In the mobile body control system of the above embodiment, the map generation means divides the area around the mobile body into a grid and generates an occupancy grid map as the occupancy map, which shows the occupancy of obstacles detected by the detection means for each grid.

[0086] According to this embodiment, a predetermined planar area can be easily divided in the x and y directions, and the predetermined range can be covered without omission.

[0087] 14. The moving body (e.g., 100) of the above embodiment is An acquisition means for acquiring a captured image; (114) A detection means for detecting an obstacle included in the captured image (130, 302, 303); a map generating means for dividing an area around the moving object and generating an occupancy map for each divided area showing the occupancy of the obstacles detected by the detecting means; and (303) The method further comprises a route generation means for generating a global route from the current position to the target position that avoids the detected obstacle based on a first cost, the cost of which increases as the distance from the current position increases on the occupied map, a second cost, the cost of which increases as the distance from the target position increases on the occupied map, and a third cost, the cost of which increases as the distance from a past route increases on the occupied map (304, S201 to S204, FIG. 7).

[0088] According to this embodiment, in real-time route planning on an occupied map, grids where route planning has been performed in the past are searched preferentially, so a pseudo-potential map is generated so that locations where paths have been performed are lowered, and this is used in the evaluation function. Therefore, according to the present invention, routes for moving objects can be suitably generated without using a high-precision map. [Explanation of symbols]

[0089] 100...mobile body, 111...seat, 112...traveling unit, 113...battery, 114...detection unit, 120...front wheel, 121...tail wheel, 122a, 122b...motor, 123...steering mechanism, 130...control unit, 131...operation panel, 132...speaker, 133...voice input device, 134...GNSS sensor, 135...storage device, 136...communication unit, 140...communication device, 301...user instruction acquisition unit, 302...image information processing unit, 303...grid map generation unit, 304...route generation unit, 305...traveling control unit

Claims

1. A mobile object control system, an acquisition means for acquiring a captured image; a detection means for detecting an obstacle included in the captured image; a map generating means for dividing an area around the moving object and generating an occupancy map for each divided area showing the occupancy of the obstacles detected by the detecting means; a route generating means for generating a global route from the current position to the target position that avoids the detected obstacle, based on a first cost that increases as the distance from the current position increases on the occupied map, a second cost that increases as the distance from the target position increases on the occupied map, and a third cost that increases as the distance from a past route increases on the occupied map; A mobile object control system comprising:

2. 2. The mobile object control system according to claim 1, wherein the route generation means further optimizes the generated global route using a Theta* algorithm that searches for a parent node with a minimum distance.

3. 2. The mobile object control system according to claim 1, wherein the third cost is generated using a Gaussian filter or an averaging filter.

4. 4. The mobile object control system according to claim 3, wherein the route generation means obtains the third cost by using a filter along the past route.

5. 5. The mobile object control system according to claim 4, wherein the past route is a route previously generated by the route generation means.

6. 3. The mobile object control system according to claim 2, wherein the first cost is further determined based on the obstacle detected by the detection means.

7. the acquisition means acquires a captured image of a region in front of the moving object, 3. The mobile object control system according to claim 2, wherein the map generation means generates the occupancy map using information on obstacles in areas not acquired by the acquisition means that have been detected in the past by the detection means.

8. The route generation means 8. The mobile object control system according to claim 7, further comprising: generating a local path for the mobile object based on a dynamic window approach (DWA) so as to follow the global path.

9. 9. The mobile object control system according to claim 8, further comprising a travel control means for determining a speed and an angular velocity of the mobile object based on the local route and controlling the travel of the mobile object.

10. 9. The mobile object control system according to claim 8, wherein said route generating means generates said global route and said local route at different cycles.

11. 11. The mobile object control system according to claim 10, wherein a generation cycle of the local route is shorter than a generation cycle of the global route.

12. the acquisition means is a stereo camera, 2. The mobile object control system according to claim 1, wherein the detecting means converts image data of the stereo images taken by the stereo camera into a cubic point cloud.

13. 3. The mobile body control system according to claim 2, wherein the map generation means divides the area around the mobile body into a grid and generates, as the occupancy map, an occupancy grid map that indicates the occupancy of obstacles detected by the detection means for each grid.

14. A mobile object, an acquisition means for acquiring a captured image; a detection means for detecting an obstacle included in the captured image; a map generating means for dividing an area around the moving object and generating an occupancy map for each divided area showing the occupancy of the obstacles detected by the detecting means; a route generating means for generating a global route from the current position to the target position that avoids the detected obstacle, based on a first cost that increases as the distance from the current position increases on the occupied map, a second cost that increases as the distance from the target position increases on the occupied map, and a third cost that increases as the distance from a past route increases on the occupied map; A moving object comprising:

15. A control method for a mobile object control system, comprising: an acquisition step of acquiring a captured image; a detection step of detecting an obstacle included in the captured image; a map generation step of dividing an area around the moving object and generating an occupancy map for each divided area showing the occupancy of the obstacle detected in the detection step; a route generation step of generating a global route from the current position to the target position that avoids the detected obstacle based on a first cost, the cost of which increases as the distance from the current position increases on the occupation map, a second cost, the cost of which increases as the distance from the target position increases on the occupation map, and a third cost, the cost of which increases as the distance from a past route increases on the occupation map; A control method for a mobile object control system, comprising:

16. A program for causing a computer to function as each of the means of the mobile object control system according to any one of claims 1 to 13.

Images