Mobile object control system, control method thereof, program, and mobile object

Abstract

To suitably perform a speed plan according to a recognition state of a road shape when highly accurate map information is not used.SOLUTION: A mobile object control system acquires a picked-up image of a travel area at a movement destination by a mobile object, recognizes a road shape included in the picked-up image, generates a locus of the mobile object on the basis of the recognized road shape, and generates a speed plan of the mobile object on the basis of the generated locus and a recognition state of the road shape.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to a mobile object control system, a control method therefor, a program, and a mobile object. [Background technology]

[0002] In recent years, small mobile objects, such as electric vehicles with a seating capacity of one or two people, known as ultra-compact mobility (also called micromobility), and mobile robots that provide various services to people, have become known. Some of these mobile objects perform autonomous driving while periodically generating a route to a destination. However, small mobile objects have limited hardware resources, making it difficult to secure an area for storing highly accurate map information for route generation or a communication device for quickly acquiring large amounts of map information. Therefore, there is a demand for such small mobile objects to generate a route and control their speed according to the generated route without using highly accurate map information.

[0003] Patent Document 1 proposes an automated driving system that, when an operation intervention that changes the braking force acting on the vehicle is performed during driving control according to a target route and planned speed, re-plans the driving based on the current actual speed, rather than changing the speed plan. Also, Patent Document 2 proposes a vehicle control device that recognizes the surrounding situation and controls driving according to target trajectory candidates that satisfy predetermined conditions (constraints related to the speed plan). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-93740 [Patent Document 2] Japanese Patent Application Publication No. 2022-6429 Summary of the Invention [Problem to be solved by the invention]

[0005] Although the above-described conventional technologies consider controlling speed according to a target route, etc., they do not generate routes without high-precision map information. On the other hand, when generating routes without high-precision map information, it is necessary to analyze images captured by a camera or the like attached to the mobile body to recognize road structures, etc., and generate a route based on the recognition results. In other words, a route is generated within the range of images captured by the camera, etc., and the route can be generated in stages as the mobile body moves. For example, when a route change is required at an intersection, etc., the road shape beyond the change may not be recognized before entering the intersection, and the route may not be planned. In such cases, speed planning is required based on the recognition status of the road shape to assist route generation and avoid sudden deceleration, etc.

[0006] The present invention has been made in view of the above-mentioned problems, and aims to perform speed planning appropriately in accordance with the recognition status of road shapes when high-precision map information is not used. [Means for solving the problem]

[0007] According to the present invention, a mobile object control system includes an imaging means for acquiring an image of a travel area to which a mobile object is to move, a recognition means for recognizing a road shape included in the image, a trajectory generation means for generating a trajectory of the mobile object based on the road shape recognized by the recognition means, and a speed planning means for generating a speed plan for the mobile object based on the trajectory generated by the trajectory generation means and a recognition status of the road shape. When a predetermined road shape having an approach section and an exit section involving a lane change is recognized, the speed planning means changes the generated speed plan in stages according to the recognition status of the approach section and the exit section of the predetermined road shape and the direction indication information. It is characterized by the following. [Effects of the Invention]

[0008] According to the present invention, speed planning can be performed appropriately in accordance with the recognition status of road shapes when high-precision map information is not used. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing an example of the hardware configuration of a mobile body according to an embodiment of the present invention; [Figure 2] FIG. 1 is a block diagram showing a control configuration of a moving body according to an embodiment of the present invention. [Figure 3] FIG. 1 is a block diagram showing the functional configuration of a moving body according to an embodiment of the present invention; [Figure 4] 1A and 1B are diagrams showing a captured image and a road shape in the captured image according to the present embodiment; [Figure 5] FIG. 10 is a diagram showing an example of a speed planning method according to the present embodiment. [Figure 6] FIG. 10 is a diagram showing a hysteresis function according to the present embodiment; [Figure 7] FIG. 10 is a diagram showing an example of a procedure for generating a trajectory at an intersection according to the present embodiment; [Figure 8] FIG. 10 is a diagram showing an example of a speed planning procedure for an intersection according to the present embodiment; [Figure 9] A flowchart showing a processing procedure for controlling the travel of a moving body according to the present embodiment. [Figure 10] A flowchart showing the processing procedure for speed planning of a moving object according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention as claimed, and not all combinations of features described in the embodiments are necessarily essential to the invention. Two or more of the features described in the embodiments may be arbitrarily combined. Furthermore, the same reference numerals are used for the same or similar components, and redundant explanations will be omitted.

[0011] <Configuration of moving body> The configuration of a moving body 100 according to this embodiment will be described with reference to Fig. 1. Fig. 1(a) shows a side view of the moving body 100 according to this embodiment, and Fig. 1(b) shows the internal configuration of the moving body 100. In the figure, arrow X indicates the front-to-rear direction of the moving body 100, with F indicating the front and R indicating the rear. Arrows Y and Z indicate the width direction (left-to-right direction) and up-down direction of the moving body 100.

[0012] The mobile body 100 is an ultra-compact mobility vehicle equipped with a propulsion unit 12, which moves mainly by motor power using a battery 13 as its main power source. An ultra-compact mobility vehicle is a vehicle that is more compact than a typical automobile and has a passenger capacity of approximately one or two people. In this embodiment, a four-wheeled ultra-compact mobility vehicle is described as an example of the mobile body 100, but this is not intended to limit the present invention, and the mobile body may also be, for example, a three-wheeled vehicle or a saddle-ride vehicle. Furthermore, the mobile body of the present invention is not limited to vehicles, but may also be a mobile body that carries luggage and runs alongside a person walking, or a mobile body that leads a person. Furthermore, the present invention is not limited to four-wheeled or two-wheeled vehicles, and may also be applicable to walking robots that are capable of autonomous movement.

[0013] The battery 13 is a secondary battery such as a lithium-ion battery, and the mobile object 100 is self-propelled by the propulsion unit 12 using power supplied from the battery 13. The propulsion unit 12 is a four-wheeled vehicle equipped with a pair of left and right front wheels 20 and a pair of left and right rear wheels 21. The propulsion unit 12 may be in the form of a tricycle or other form. The mobile object 100 is equipped with a seat 14 for one or two people. An operation unit 25 is provided in front of the seat 14 to allow the passenger to input direction instructions. The operation unit 25 is any device that indicates the direction of travel of the mobile object 100, and may be, for example, a device that allows input in multiple directions, such as a joystick. Before entering a road configuration having an exit requiring a lane change, such as an intersection, the driver can operate the operation unit 25 to indicate which exit to use to exit.

[0014] The traveling unit 12 includes a steering mechanism 22. The steering mechanism 22 is a mechanism that uses a motor 22a as a drive source to change the steering angle of the pair of front wheels 20. By changing the steering angle of the pair of front wheels 20, the traveling direction of the mobile body 100 can be changed. The traveling unit 12 also includes a drive mechanism 23. The drive mechanism 23 is a mechanism that uses a motor 23a as a drive source to rotate the pair of rear wheels 21. By rotating the pair of rear wheels 21, the mobile body 100 can move forward or backward.

[0015] The moving body 100 is equipped with detection units 15 to 17 that detect targets around the moving body 100. The detection units 15 to 17 are a group of external sensors that monitor the periphery of the moving body 100, and in the present embodiment, each is an imaging device that captures an image of the periphery of the moving body 100, and includes, for example, an optical system such as a lens and an image sensor. However, instead of or in addition to the imaging device, it is also possible to employ radar or lidar (Light Detection and Ranging).

[0016] Two detection units 15 are arranged at the front of the moving body 100, spaced apart in the Y direction, and mainly detect targets ahead of the moving body 100. Detection units 16 are arranged on the left and right sides of the moving body 100, respectively, and mainly detect targets to the sides of the moving body 100. Detection unit 17 is arranged at the rear of the moving body 100 and mainly detects targets behind the moving body 100. Furthermore, in this embodiment, an example in which detection units are provided on the front, rear, left and right sides of the moving body 100 will be described, but this is not intended to limit the present invention, and a configuration in which detection units are provided only in a certain direction (for example, the front) of the moving body 100 may also be used.

[0017] The mobile object 100 according to this embodiment captures an image of the area ahead of the mobile object 100 using at least the detection unit 15, extracts the road shape from the captured image, and generates a route based on recognition information indicating the extracted road shape, operation instructions from the driver via the operation unit 25, or information regarding course changes obtained from a route plan to the destination. The recognition information is output by a machine learning model that processes image information (captured image). The machine learning model performs processing to recognize the road shape contained in the image information, for example, by performing calculations using a deep learning algorithm using a deep neural network (DNN). The recognition information includes various road lines and lane information, the lane in which the vehicle is located (Ego lane), various intersections (intersections, etc.), and entrances to various roads (Road entrances).

[0018] <Control structure of moving object> FIG. 2 is a block diagram of a control system of the mobile object 100 according to this embodiment. Here, the configuration necessary for implementing the present invention will be mainly described. Therefore, other configurations may be included in addition to the configurations described below. Furthermore, in this embodiment, the mobile object 100 will be described as including each of the units described below. However, this is not intended to limit the present invention, and the mobile object 100 may be realized as a mobile object control system including multiple devices. For example, some of the functions of the control unit 30 may be implemented by a server device connected to the mobile object 100 in a communicable manner, or the detection units 15 to 17 and the GNSS sensor 34 may be provided as external devices. The mobile object 100 includes a control unit (ECU) 30. The control unit 30 includes a processor, such as a CPU, a storage device, such as a semiconductor memory, an interface with an external device, and the like. The storage device stores programs executed by the processor, data used by the processor for processing, and the like. Multiple sets of processors, storage devices, and interfaces may be provided for different functions of the mobile object 100 and configured to communicate with each other.

[0019] The control unit 30 acquires the detection results of the detection units 15 to 17, input information from the operation panel 31, voice information input from the voice input device 33, position information from the GNSS sensor 34, direction instruction information from the operation unit 25, and information received via the communication unit 36, and executes corresponding processing. The control unit 30 controls the motors 22a and 23a (travel control of the traveling unit 12), controls the display on the operation panel 31, and notifies and outputs information to the occupants of the mobile object 100 by voice from the speaker 32.

[0020] The voice input device 33 collects the voices of the occupants of the moving body 100. The control unit 30 can recognize the input voices and execute corresponding processing. The GNSS (Global Navigation Satellite system) sensor 134 receives GNSS signals to detect the current position of the moving body 100. The storage device 35 is a storage device that stores images captured by the detection units 15 to 17, obstacle information, previously generated routes, and occupancy grid maps. The storage device 35 may also store programs executed by the processor, data used for processing by the processor, and the like. The storage device 35 may also store various parameters (e.g., learned parameters and hyperparameters of a deep neural network) of machine learning models for voice recognition and image recognition executed by the control unit 30.

[0021] The communication unit 36 ​​communicates with a communication device 120, which is an external device, via wireless communication such as Wi-Fi or fifth-generation mobile communication. The communication device 120 is, for example, a smartphone, but is not limited to this and may also be an earphone-type communication terminal, a personal computer, a tablet terminal, a game console, or the like. The communication device 120 connects to a network via wireless communication such as Wi-Fi or fifth-generation mobile communication.

[0022] A user who owns the communication device 120 can give instructions to the moving body 100 via the communication device 120. The instructions include, for example, an instruction to call the moving body 100 to a location desired by the user and join the moving body 100. Upon receiving the instruction, the moving body 100 sets a target position based on the position information included in the instruction. In addition to such instructions, the moving body 100 can also set a target position from images captured by the detection units 15 to 17, or based on instructions from a user riding in the moving body 100 via the operation panel 31. When setting a target position from a captured image, for example, a person raising their hand toward the moving body 100 is detected in the captured image, and the position of the detected person is estimated and set as the target position.

[0023] <Functional configuration of mobile units> Next, the functional configuration of the moving body 100 according to this embodiment will be described with reference to Fig. 3. The functional configuration described here is realized in the control unit 30, for example, by the CPU reading a program stored in a memory such as a ROM into the RAM and executing it. Note that the functional configuration described below describes only the functions necessary for explaining the present invention, and does not describe all of the functional configuration actually included in the moving body 100. In other words, the functional configuration of the moving body 100 according to the present invention is not limited to the functional configuration described below.

[0024] The user instruction acquisition unit 301 has a function of receiving instructions from the user, and can receive user instructions via the operation unit 25 or the operation panel 31, user instructions from an external device such as the communication device 120 via the communication unit 36, and user spoken instructions via the voice input device 33. As described above, user instructions include instructions to set a target position (also referred to as a destination) of the moving body 100 and instructions related to driving control of the moving body 100.

[0025] The image information processing unit 302 processes the captured images acquired by the detection units 15 to 17. Specifically, the image information processing unit 302 extracts a recognized road shape from the captured images acquired by the detection units 15 to 17. The image information processing unit 302 also includes a machine learning model that processes image information, and may execute a learning stage process and an inference stage process of the machine learning model. The machine learning model of the image information processing unit 302 can perform a process of recognizing the road shape and the like included in the image information by performing calculations of a deep learning algorithm using, for example, a deep neural network (DNN). The recognition information indicating the recognized road shape includes, for example, information indicating lines such as white lines, lanes, the shape of intersections, entrances and exits of intersections, and the like.

[0026] The trajectory generation unit 303 generates a travel path (trajectory) for the mobile object 100 relative to the target position set by the user instruction acquisition unit 301. Specifically, the trajectory generation unit 303 generates a trajectory based on road shapes (recognition information) recognized from images captured by the detection units 15 to 17 and direction instruction information received via the operation unit 25, without requiring obstacle information from a high-precision map. Note that the recognition information is information about road shapes within a predetermined range from the mobile object 100, and road shapes further away cannot be recognized. On the other hand, the recognition information is information that is periodically updated as the mobile object 100 moves. Therefore, distant areas are gradually recognized as the mobile object 100 moves. The trajectory generation unit 303 sequentially generates a trajectory according to the updated recognition information. Furthermore, the direction instruction information is not limited to information received via the operation unit 25, but may be based on route change information obtained by route planning to the destination. Therefore, the operation unit 25 is not an essential component of the present invention, and the present invention can be applied to mobile objects that do not have an operation unit 25.

[0027] The speed planning unit 304 also plans the speed according to the curvature of the trajectory generated by the trajectory generation unit 303, and also plans the speed based on the direction instruction from the driver and the accuracy of the recognized road shape. For example, when a right or left turn is instructed at an intersection, the speed plan controls the vehicle to decelerate to 8 km / h if turning right and to 6 km / h if turning left before going straight and starting to curve. By controlling the speed according to the generated trajectory and the driver's instruction in this way, it is possible to avoid sudden deceleration. Furthermore, if the recognition accuracy of the road shape is high, the deceleration is set small, and if the recognition accuracy is low, the deceleration is set high. The recognition accuracy can be determined, for example, according to parameters assigned to the road structure indicating the road shape recognized by the image information processing unit 302. Details will be described later.

[0028] The traveling control unit 305 controls the traveling of the moving body 100 according to the generated trajectory and speed plan. Specifically, the traveling control unit 305 controls the traveling unit 12 according to the trajectory and speed plan to control the speed and angular velocity of the moving body 100. When a deviation occurs in the trajectory driving plan due to the driver's operation, the traveling control unit 305 may again acquire a new trajectory generated by the trajectory generation unit 303 and control the traveling, or may control the speed and angular velocity of the moving body 100 so as to eliminate the deviation from the trajectory currently in use.

[0029] <Captured image> 4A shows an example of a captured image according to this embodiment, and FIG. 4B shows an example of a road shape included in the captured image. The captured image 400 shown in FIG. 4A shows an image captured by the detection unit 15 provided in front of the moving body 100. The shaded area 401 shows the inside of the cockpit of the moving body 100 captured in the captured image 400. The area other than the shaded area 401 is an area in which the surrounding environment spreading out in the area in front of the moving body 100 is captured.

[0030] FIG. 4(b) shows the road shape included in the captured image 400 shown in FIG. 4(a). The dotted area 410 indicates a three-way intersection (T-junction). As shown in FIG. 4(b), a three-way intersection is located in the area ahead of the mobile object 100, and there are two exits for the intersection: one that can be reached by proceeding straight ahead and one that can be reached by turning right. Beyond the exit for proceeding straight ahead, a road continues with a large curve to the right. As shown in the captured image 400, from the viewpoint of the mobile object 100 before entering the intersection 410, it is possible to recognize the existence of multiple exits indicated by the arrows, but it is not possible to recognize the road shape beyond the exit. Therefore, as the mobile object 100 according to this embodiment moves, it generates a trajectory and performs speed planning sequentially or stepwise using the road shape that becomes clear. In other words, as the mobile object 100 approaches a predetermined road shape, the accuracy of its recognition increases, and it generates a trajectory and speed plan according to the recognition level. Furthermore, when the road shape recognized from the captured image 400 includes multiple exit sections, or when at least one exit section is located outside a predetermined range from the current traveling direction, the moving body 100 according to this embodiment determines that the road shape includes at least one exit section that will result in a lane change. In this case, according to this embodiment, it is determined that there is a possibility of a lane change, and trajectory generation, as described below, is executed. Regarding the speed plan, deceleration may be initiated when direction indication information that will result in a lane change is acquired, or when at least one exit section that will result in a lane change is recognized. Thus, according to this embodiment, when there is a possibility of a lane change, deceleration is initiated in preparation for the lane change.

[0031] <Speed ​​planning method> 5 shows an example of a speed planning method according to this embodiment. Here, the functional configuration of the speed planning by the speed planning unit 304 will be described. The speed plan for the moving body 100 according to this embodiment is planned according to the recognition status of the road shape, such as an intersection, by the image information processing unit 302. The speed planning unit 304 has, as its functional configuration, an intersection distance speed plan 501, a curvature-based speed plan 502, and a detection accuracy speed plan 503.

[0032] In the intersection distance speed plan 501, a speed plan is generated based on the distance to an approach section of an intersection or the like recognized by the image information processing unit 302. Specifically, a speed plan is generated here for decelerating the mobile object 100 to a target speed based on the destination direction of the mobile object 100 after entering the intersection. For example, if the moving direction is a left turn that requires a lane change, the speed of the mobile object 100 is planned to be 6 km / h at the recognized approach section of the intersection or the like. If the moving direction is a right turn that requires a lane change, the speed of the mobile object 100 is planned to be 8 km / h at the recognized approach section of the intersection or the like. Furthermore, if the moving direction is straight, the speed of the mobile object 100 is planned to be 10 km / h at the recognized approach section of the intersection or the like. Note that these numerical values ​​are merely examples and do not limit the present invention. Right and left turns are determined based on right and left turn operation instructions from the operation unit 25 or information regarding the lane change obtained from the route plan to the destination. The deceleration value in the speed plan may be selected from multiple discontinuous candidate values. These candidate values ​​are stored in advance in the storage device 35.

[0033] In the curvature-based speed plan 502, a speed plan is generated according to the trajectory generated by the trajectory generation unit 303, particularly the curvature of the trajectory. Here, the speed plan is generated according to the generated trajectory so as to prevent sharp turns or sudden deceleration within the speed limit. In addition, in the curvature-based speed plan 502, a speed plan is generated so as to change the speed of the moving body according to the generation status of the generated trajectory. For example, the longer the generated trajectory, the higher the speed of the moving body 100 is adjusted.

[0034] Furthermore, the detection accuracy speed plan 503 generates a speed plan according to the recognition accuracy of the road shape recognized by the image information processing unit 302. For example, the speed plan is generated such that the higher the recognition accuracy of the road shape, the higher the speed, and the lower the recognition accuracy, the lower the speed. In other words, the speed plan is generated such that the higher the recognition accuracy of the road shape, the lower the deceleration value, and the lower the recognition accuracy, the higher the deceleration value. As a result, when the road shape cannot be clearly recognized, the speed of the mobile body 100 can be reduced and recognition of the road shape can be prioritized. On the other hand, when the road shape is clearly recognized, the generated trajectory becomes longer, and the speed of the mobile body 100 can be increased. Note that the recognition status of the road shape can be determined by referring to the parameters of the recognition information output from the image information processing unit 302. As described above, the parameters include various road lines and lane information, the lane in which the vehicle is located (Ego lane), various intersections (intersections, etc.), entrances to various roads (Road entrances), etc., and each parameter increases as recognition is achieved. Therefore, if the road information (parameters) of the entrance and all exits at an intersection and beyond can be recognized, it can be determined that the intersection has been clearly recognized.

[0035] As described above, in the moving body 100 according to this embodiment, multiple speed plans are generated as needed based on various conditions. The multiple generated speed plans (speed values) are input to the minimization block 504, and the speed plan with the smallest speed value is selected. Note that speed plans are not always output from the functional components 501 to 503, but are generated and input as needed at timing when a speed plan can be generated in each functional component. Therefore, if multiple inputs are received simultaneously, the speed plan with the smallest speed value among them is selected and input to the subsequent functional component of the deceleration plan 505. In the deceleration plan 505, a deceleration value for the moving body 100 is determined based on the current speed of the moving body 100 and the input speed plan. Note that, although the deceleration value is used as an example here, acceleration also occurs after a lane change, and in that case, the acceleration value is used. That is, here, the acceleration value or deceleration value is generated according to the speed plan.

[0036] Furthermore, when changing the deceleration value, control is performed using a hysteresis function as shown in FIG. 6. In FIG. 6, the horizontal axis represents the deceleration value input to the deceleration plan 505, and the vertical axis represents the deceleration value output by the deceleration plan 505. The solid line represents the increase in the deceleration value, and the dotted line represents the decrease in the deceleration value. Normally, it is preferable to decelerate at 0.1 G; however, depending on the timing of a direction instruction, etc., there may be cases where the target speed cannot be achieved unless deceleration of 0.1 G or more is performed. On the other hand, if the deceleration value that makes the deceleration G variable is calculated sequentially, the vibration of the moving body 100 becomes severe, adversely affecting the ride comfort. Therefore, according to this embodiment, the deceleration value is changed with hysteresis as shown in FIG. 6.

[0037] <Trajectory generation procedure for intersections, etc.> Fig. 7 shows a trajectory generation procedure according to the recognition situation of an intersection according to this embodiment. Here, a trajectory generation procedure when approaching the intersection (T-junction) shown in Fig. 4(b) will be described. According to this embodiment, a trajectory when passing through a road shape such as an intersection having an entrance section and an exit section requiring a lane change is generated sequentially (step by step) according to recognition information of the road shape obtained from a captured image.

[0038] Here, an example will be described in which trajectory control is performed in four stages depending on the distance between the moving object 100 and the intersection. As shown in Fig. 7, phase 0 is a state in which the distance from the moving object 100 to the intersection is more than 30 m. In this state, the image information processing unit 302 recognizes the "Ego lane" indicating the driving area in which the moving object 100 is driving from the captured images acquired by the detection units 15 to 17, but does not recognize the shape of the road at the intersection.

[0039] Phase 1 is a state in which the distance from the moving object 100 to the intersection is less than 30 m, and an instruction to turn right has been received from the operation unit 25. In this state, the image information processing unit 302 can recognize the intersection in addition to the above-mentioned "Ego lane." Note that the intersection recognition information here recognizes the shape of the intersection and the entrance ("Road entrance"), but does not clearly recognize the exit (exit) or driving area beyond.

[0040] The road shape recognition information (for example, "Intersection" indicating an intersection, etc.) extracted by the image information processing unit 302 using a machine learning model includes various parameters depending on the recognition situation. The parameters include, for example, "Road entrance" indicating an entrance or exit that requires a lane change, information indicating the boundary of the "Intersection," information indicating the white lines (lines) of the nearby road shape, and information indicating lanes (driving areas, lanes). In other words, depending on the recognition situation, only a parameter indicating the shape (boundary) of the "Intersection" may be included, and it may be that the lane or line at the branch point has not been recognized.

[0041] Therefore, the trajectory generation unit 303 needs to generate and update the trajectory sequentially according to such a recognition situation. In this phase 1, although the instruction to turn right is received, the exit of the intersection and the driving area beyond it have not yet been identified. Therefore, the trajectory generation unit 303 maintains the current trajectory and does not generate a trajectory that changes the course.

[0042] Phase 2 is a state in which the distance from the mobile object 100 to the intersection is less than 20 m, and an instruction to turn right has been received from the operation unit 25. In this state, in addition to the recognition information from Phase 1, the image information processing unit 302 recognizes the driving area "Target lane" ahead and extracts the entrance to the driving area. Note that although the driving area "Target lane" is recognized here, the line indicating the boundary of the driving area cannot be recognized, and the accuracy of the recognition of the driving area is low. On the other hand, the entrance to the intersection "Road entrance" for the instructed right turn can be clearly recognized. For example, in Phase 2, the mobile object is approaching the intersection, and the boundary of the intersection on the right turn side can be clearly recognized, and the upper half can be estimated as the driving lane (driving area "Target lane"), and the lower half can be estimated as the oncoming lane.

[0043] Therefore, in Phase 2, the trajectory generation unit 303 generates a trajectory for the lane change based on information that can be recognized with higher accuracy. Specifically, the trajectory generation unit 303 generates a first trajectory from the current position of the mobile object 100 to the already recognized road entrance of the intersection, and a second trajectory from the road entrance to the exit road of the intersection on the right turn side. The second trajectory is generated to include a trajectory of at least one curve, such as a simple curve, a clothoid curve, or a cubic curve. The second trajectory may include a straight line in addition to such a curve. Furthermore, the second trajectory may be generated as a trajectory from the center of the entrance to the center of the identified exit road.

[0044] Phase 3 is a state in which the moving object 100 has entered an intersection. In this state, the image information processing unit 302 can further recognize a white line indicating the boundary of the driving area "Target lane" ahead, in addition to the recognition information in Phase 2, and can recognize the driving area "Target lane" more accurately than in Phase 2. Therefore, in Phase 3, the trajectory generation unit 303 generates a third trajectory that follows the second trajectory generated in Phase 2 and is within the recognized driving area. This makes it possible to generate a trajectory when a right turn instruction is given at an intersection.

[0045] In the above Phase 3, an example has been described in which the third trajectory is generated when the driving area "Target lane" is more accurately determined by recognizing a white line (e.g., a "Lane instance" line indicating a lane boundary) of the driving area "Target lane" after the moving body 100 enters an intersection, but this does not limit the present invention. For example, instead of determining the driving area "Target lane" as described above, the trajectory generation unit 303 may generate the third trajectory by determining that the driving area "Target lane" has been determined to a certain extent when the moving body 100 proceeds beyond the entrance to the intersection. The timing when the moving body 100 proceeds beyond the entrance to the intersection is the timing when the moving body 100 starts traveling on the second trajectory. Alternatively, the trajectory generation unit 303 may generate the third trajectory by determining that the driving area "Target lane" has been determined to a certain extent when the moving body 100 approaches a predetermined distance from the identified entrance on the right turn side.

[0046] Furthermore, in the present invention, an example has been described in which a route change, such as a right or left turn, is determined based on direction instruction information from the operation unit 25, but this does not limit the present invention. For example, when a route is planned according to a preset destination and it is necessary to turn right at the next intersection, the trajectory generation unit 303 may determine that a right turn will be made and generate a trajectory without receiving the above-mentioned direction instruction information. Also, for example, when the traveling direction reaches a dead end and a left or right turn is required, but direction instruction information from the operation unit 25 is not received, the trajectory may be determined that a left or right turn will be made and generated. In this case, for example, a route change in a direction closer to the destination may be selected for the route plan to the destination.

[0047] Furthermore, according to this embodiment, the speed planning unit 304 plans a target speed for each phase, as shown in the "Speed" row in Fig. 7. For example, if an instruction to turn right or left is received after recognizing an intersection, a lane change will occur, and so it is necessary to decelerate to a predetermined speed (e.g., 8 km / h for a right turn, 6 km / h for a left turn) by the time the vehicle enters the intersection. Therefore, it is desirable for the speed planning unit 304 to decelerate in stages in each phase to avoid sudden deceleration. A specific deceleration method will be described with reference to Fig. 8.

[0048] <Speed ​​planning at intersections, etc.> FIG. 8 shows a speed planning procedure according to the recognition status of a road shape such as an intersection according to this embodiment. In graph 800, the horizontal axis indicates time (s) and the vertical axis indicates the speed (km / h) of the mobile object 100. Here, speed planning when the mobile object 100 approaches the T-junction shown in FIG. 4 will be described. Note that in this embodiment, a T-junction is used as an example of a road shape having an entrance section and an exit section (exit) requiring a lane change. However, the present invention may also use an intersection, a T-junction, an entrance to a facility along the road, an L-shaped driving area, or the like as a road shape having an entrance section and an exit section (exit) requiring a lane change. Examples of entrances to facilities along the road include entrances to shopping malls, gas stations, parking lots, etc.

[0049] First, when the speed planning unit 304 receives direction indication information, for example, via the operation unit 25, regardless of whether an intersection or the like has been detected, it determines that the vehicle will change course in the direction received at the next intersection or the like, and starts deceleration (Phase 0 above). The deceleration value here is determined to be a predetermined value, such as 0.05 G. It may also be determined according to the current speed of the moving body 100. Note that the timing of receiving the direction indication information may be after the intersection or the like has been detected. Although it also depends on the current speed of the moving body 100, if an intersection or the like has been detected first, deceleration may start before the direction indication information is received.

[0050] After the image information processing unit 302 detects an intersection or the like, if it further recognizes the "road entrance" of the intersection or the like, the distance to the intersection or the like is recognized (Phase 1 above). Here, the speed planning unit 304 updates the deceleration value according to the target speed to the entrance of the intersection or the like (for example, 8 km / h for a right turn, 6 km / h for a left turn). For example, it is updated to 0.1 G to 0.2 G.

[0051] After that, when an entrance to a road such as an intersection is recognized (Phase 2 above) and a trajectory within the intersection is generated, a speed plan is generated in the curvature-based speed plan 502, and the deceleration value is adjusted as necessary. Note that before and after Phase 3, the moving body 100 has reached the entrance to the intersection or the like, and therefore has decelerated to the target speed. Furthermore, when the target lane is determined and a trajectory beyond the entrance is generated (Phase 3 above), a new speed plan is generated in the curvature-based speed plan 502, and the deceleration value is adjusted as necessary.

[0052] Furthermore, when passing through an intersection, etc., a speed plan is generated by the detection accuracy speed plan 503 according to the recognition accuracy of the intersection, etc. If multiple speed plans are generated, the smallest speed is selected and the deceleration value is determined, as explained using FIG.

[0053] <Basic flow> 9 is a flowchart showing the basic control of the moving body 100 according to this embodiment. The processing described below is realized in the control unit 30, for example, by the CPU reading a program stored in a memory such as a ROM into the RAM and executing it.

[0054] In S101, the control unit 30 sets a target position of the moving body 100 based on a user instruction received by the user instruction acquisition unit 301. As described above, the user instruction can be received in various ways. Subsequently, in S102, the control unit 30 acquires direction instruction information. This information includes direction instruction information when the driver operates the operation unit 25 and direction instruction information for a course change determined according to the set destination position. Here, for convenience, the process of acquiring direction instruction information is described as being performed in S102, but in reality, it is a process that occurs at any time as an interrupt process when the driver operates the operation unit 25. Therefore, even after the process of S102, direction instruction information is acquired by an operation interrupt and used for trajectory generation.

[0055] Next, in S103, the control unit 30 captures an image of the area ahead (traveling direction) of the moving object 100 using the detection unit 15, and acquires the captured image. Then, in S104, the control unit processes the acquired captured image using the image information processing unit 302, and acquires recognition information indicating the road shape recognized using a machine learning model. Note that the processes of S103 and S104 are performed continuously or periodically, and the captured image and the recognition information acquired from the captured image are information that is updated as needed.

[0056] Next, in S105, the control unit 30 generates a trajectory for the moving object 100 using the trajectory generation unit 303 in accordance with the recognition information acquired in S104. Subsequently, in S106, the control unit 30 generates a speed plan for the moving object 100 based on the generated trajectory, the direction indication information, and the recognition information. A detailed procedure for speed planning by the speed planner 304 will be described later with reference to FIG. 10 . Furthermore, in S107, the control unit 30 determines the speed and angular velocity of the moving object 100 using the travel control unit 305, and controls the traveling. Thereafter, in S108, the control unit 30 determines whether the moving object 100 has reached the target position based on the position information from the GNSS sensor 34. If the moving object 100 has not reached the target position, the process returns to S102, and the process of generating a trajectory and controlling the traveling is repeated while updating the captured image. On the other hand, if the moving object 100 has reached the target position, the process of this flowchart ends.

[0057] <Speed ​​planning procedure> 10 is a flowchart showing the detailed processing procedure of speed planning (S106) according to this embodiment. The processing described below is realized by, for example, the CPU of the control unit 30 reading a program stored in a memory such as a ROM into the RAM and executing it.

[0058] First, in S201, the speed planning unit 304 determines whether the trajectory has been updated in S105. If it has been updated, the process proceeds to S202; if not, the process proceeds to S203. In S202, the speed planning unit 304 generates a speed plan according to the curvature of the updated trajectory. Here, the speed from the current position of the moving body 100 to the end point of the generated trajectory is planned, and the process proceeds to S203. On the other hand, if there is no change in the trajectory generated up to the previous time, it is also possible to generate only a speed plan for the newly generated portion of the trajectory.

[0059] In S203, the speed planning unit 304 determines whether or not direction indication information was received in S102. If it was received, the process proceeds to S204, and if not, the process proceeds to S205. In S204, the speed planning unit 304 determines that a lane change will occur because direction indication information has been received, and starts deceleration, and then proceeds to S205. Here, deceleration starts at a predetermined deceleration value, for example, 0.05G.

[0060] In S205, the speed planning unit 304 determines whether an intersection or the like has been detected in the recognition information acquired in S104. If it has been detected, the process proceeds to S206; if not, the process proceeds to S208. In S206, when the speed planning unit 304 recognizes an approach section of the detected intersection or the like, it generates a speed plan to the approach section in order to decelerate to a predetermined speed depending on the distance from the mobile body 100 to the approach section and the direction of the course change. Subsequently, in S207, the speed planning unit 304 generates a speed plan depending on the recognition accuracy of the intersection or the like, and proceeds to S208.

[0061] In S208, the speed planning unit 304 adjusts the currently executing speed plan using the speed plans generated in S202, S206, and S207, terminates the processing of this flowchart, and returns to S107. As for the adjustment method, for example, as described in Fig. 5, a speed plan that minimizes the speed may be selected. Note that the processing of S206 is omitted if a speed plan to an approach point such as an intersection has already been generated.

[0062] <Summary of the embodiment> 1. The mobile object control system (e.g., 100) of the above embodiment includes: Imaging means for acquiring an image of a travel area to which the moving object is moving (e.g., 15 to 17); A recognition means for recognizing a road shape included in the captured image (e.g., 302); a trajectory generation means (e.g., 303) for generating a trajectory of the moving object based on the road shape recognized by the recognition means; a speed planning means (e.g., 304) for generating a speed plan for the moving body based on the trajectory generated by the trajectory generating means and the recognized state of the road shape; The present invention is characterized by comprising:

[0063] According to this embodiment, when high-precision map information is not used, speed planning can be performed appropriately according to the recognition status of the road shape.

[0064] 2. In the mobile object control system of the above embodiment, the speed planning means generates a speed plan to start decelerating the mobile object when direction indication information involving a course change is acquired.

[0065] According to this embodiment, by starting deceleration before a lane change is made, it is possible to avoid sudden deceleration when actually starting to turn.

[0066] 3. In the mobile body control system of the above embodiment, when the recognition means recognizes the road shape having an entry section, the speed planning means determines the deceleration value of the mobile body based on the distance from the current position of the mobile body to the entry section.

[0067] According to this embodiment, when changing course, the turning can be started at an optimum speed.

[0068] 4. In the mobile object control system of the above embodiment, the speed planning means determines the deceleration value so that the speed of the mobile object is decelerated to a target speed corresponding to each course change direction before reaching the approach section.

[0069] According to this embodiment, it is possible to start turning at an optimum speed depending on the direction of course change.

[0070] 5. In the mobile object control system of the above embodiment, the target velocity is determined further in accordance with the curvature of the trajectory generated by the trajectory generating means.

[0071] According to this embodiment, speed planning can be performed in accordance with the curvature of the generated trajectory to avoid sudden deceleration and sharp turns.

[0072] 6. In the mobile object control system of the above embodiment, the target speed is a predetermined value.

[0073] According to this embodiment, turning can be performed at an optimum speed depending on the turning radius, which differs for right and left turns.

[0074] 7. In the mobile object control system of the above embodiment, the speed planning means selects the deceleration value from a plurality of discontinuous candidate values.

[0075] According to this embodiment, the processing load can be reduced compared to when the calculation is performed each time.

[0076] 8. In the mobile object control system of the above embodiment, the speed planning means varies the deceleration value with hysteresis.

[0077] According to this embodiment, sudden deceleration can be avoided when decelerating according to a speed plan.

[0078] 9. In the mobile object control system of the above embodiment, the speed planning means further changes the deceleration value in accordance with the accuracy of recognition of the road shape by the recognition means.

[0079] According to this embodiment, the recognition of the road shape can be supported by switching the speed depending on the recognition accuracy.

[0080] 10. In the mobile object control system of the above embodiment, the speed planning means The higher the recognition accuracy of the road shape, the smaller the deceleration value is set; The lower the recognition accuracy of the road shape, the larger the deceleration value is set.

[0081] According to this embodiment, it is possible to prioritize the driving speed or support the recognition of the road shape depending on the recognition accuracy.

[0082] 11. In the mobile object control system of the above embodiment, the speed planning means further changes the target speed in accordance with the state of trajectory generation by the trajectory generation means.

[0083] According to this embodiment, it is possible to avoid unnecessary deceleration even though a trajectory has been generated.

[0084] 12. In the mobile object control system of the above embodiment, the speed planning means increases the target speed as the trajectory generated by the trajectory generation means becomes longer.

[0085] According to this embodiment, if the generated trajectory is long, unnecessary deceleration can be avoided.

[0086] 13. The vehicle further comprises a direction indication means for receiving the direction indication information relating to the destination of the moving body.

[0087] According to this embodiment, a speed plan can be generated according to a course change intended by the user.

[0088] 14. In the mobile object control system of the above embodiment, the road shape having the entry section and at least one exit section involving a lane change is any one of an intersection, a T-junction, and a driving area including an entrance to a facility along the road.

[0089] According to this embodiment, speed plans can be generated in accordance with various road configurations.

[0090] Although the embodiments of the invention have been described above, the invention is not limited to the above-described embodiments, and various modifications and variations are possible within the scope of the gist of the invention. [Explanation of symbols]

[0091] 100...mobile body, 12...travel unit, 13...battery, 14...seat, 15-17...detection unit, 20...front wheel, 21...rear wheel, 22...steering mechanism, 22a, 23a...motor, 25...operation unit, 30...control unit, 31...operation panel, 32...speaker, 33...voice input device, 34...GNSS sensor, 35...storage device, 36...communication unit, 120...communication device, 301...user instruction acquisition unit, 302...image information processing unit, 303...trajectory generation unit, 304...speed planning unit, 305...travel control unit

Claims

1. A mobile object control system, an imaging means for capturing an image of a travel area to which the moving object is to move; a recognition means for recognizing a road shape included in the captured image; a trajectory generation means for generating a trajectory of the moving object based on the road shape recognized by the recognition means; a speed planning means for generating a speed plan for the moving body based on the trajectory generated by the trajectory generating means and the recognized state of the road shape; Equipped with A mobile body control system characterized in that, when a predetermined road shape having an entry section and an exit section requiring a lane change is recognized, the speed planning means gradually switches the speed plan to be generated depending on the recognition status of the entry section and the exit section of the predetermined road shape and direction indication information.

2. 2. The mobile object control system according to claim 1, wherein the speed planning means generates a speed plan so as to start decelerating the mobile object when direction indication information involving a course change is acquired.

3. The mobile body control system according to claim 2, characterized in that when the recognition means recognizes a road shape having an entrance section, the speed planning means determines a deceleration value of the mobile body according to the distance from the current position of the mobile body to the entrance section.

4. 4. The mobile body control system according to claim 3, wherein the speed planning means determines the deceleration value so as to decelerate the speed of the mobile body to a target speed corresponding to each of the course change directions before the approach section.

5. 5. The mobile object control system according to claim 4, wherein the target velocity is determined further in accordance with the curvature of the trajectory generated by the trajectory generating means.

6. 5. The mobile object control system according to claim 4, wherein the target speed is a predetermined value.

7. 4. The mobile object control system according to claim 3, wherein said speed planning means selects the deceleration value from a plurality of discontinuous candidate values.

8. 8. The mobile object control system according to claim 7, wherein said speed planning means changes said deceleration value with hysteresis.

9. 4. The mobile object control system according to claim 3, wherein said speed planning means further changes said deceleration value in accordance with the accuracy of recognition of said road shape by said recognition means.

10. The speed planning means The higher the recognition accuracy of the road shape, the smaller the deceleration value is set; 10. The mobile object control system according to claim 9, wherein the deceleration value is increased as the recognition accuracy of the road shape is lower.

11. 5. The mobile object control system according to claim 4, wherein said speed planning means further changes the speed of said mobile object in accordance with the state of trajectory generation by said trajectory generation means.

12. 12. The mobile object control system according to claim 11, wherein the speed planning means increases the speed of the mobile object as the trajectory generated by the trajectory generating means becomes longer.

13. 3. The mobile object control system according to claim 2, further comprising a direction instruction unit that receives the direction instruction information relating to the destination of the mobile object.

14. The mobile object control system of claim 3, characterized in that the specified road shape having the entry section and at least one exit section involving the lane change is any one of an intersection, a T-junction, and a driving area including an entrance to a facility along the road.

15. A control method for a mobile object control system, comprising: an imaging step of acquiring an image of a travel area to which the moving object is to move; a recognition step of recognizing a road shape included in the captured image; a trajectory generation step of generating a trajectory of the moving object based on the road shape recognized in the recognition step; a speed planning step of generating a speed plan for the moving object based on the trajectory generated in the trajectory generating step and the recognized state of the road shape; Including, A control method for a mobile body control system, characterized in that the speed planning process, when a predetermined road shape having an entry section and an exit section requiring a lane change is recognized, gradually switches the generated speed plan depending on the recognition status of the entry section and the exit section of the predetermined road shape and direction indication information.

16. A program for causing a computer to function as each of the means of the mobile object control system according to any one of claims 1 to 14.

17. A mobile object, an imaging means for capturing an image of a travel area to which the moving object is to move; a recognition means for recognizing a road shape included in the captured image; a trajectory generation means for generating a trajectory of the moving object based on the road shape recognized by the recognition means; a speed planning means for generating a speed plan for the moving body based on the trajectory generated by the trajectory generating means and the recognized state of the road shape; Equipped with The speed planning means is characterized in that, when a predetermined road shape having an entry section and an exit section requiring a lane change is recognized, the speed planning means gradually switches the speed plan to be generated depending on the recognition status of the entry section and the exit section of the predetermined road shape and direction indication information.

Images