Mobile vehicle control system, control method thereof, program, and mobile vehicle

Abstract

This moving body control system acquires a captured image of a traveling area in a destination of a moving body, recognizes a road shape which is included in the captured image, and generates a path of the moving body on the basis of the recognized road shape. In the path generation, when a road shape that has an entrance section and at least one advancement section which is associated with a turn is recognized, a first path from the present location of the moving body to the entrance section and a second path from the entrance section to the advancement section that is identified according to indication information pertaining to the destination of the moving body are generated.

Description

【Technical Field】 【0001】 The present invention relates to a movement control system, its control method, program, and a moving body. 【Background Art】 【0002】 In recent years, small moving bodies such as electric vehicles with about 1 to 2 passengers, called ultra-small mobility (also referred to as micromobility), and mobile robots that provide various services to people are known. Some of these moving bodies perform autonomous driving while periodically generating a travel route to a destination. In small moving bodies, hardware resources are scarce, and it is difficult to secure an area for storing highly accurate map information for generating a route or a communication device for quickly acquiring a large amount of map information. Therefore, in such small moving bodies, it is required to generate a route without using highly accurate map information. 【0003】 Patent Document 1 proposes recognizing a road shape by extracting a white line at the road edge from an image acquired by a camera or the like in a curve section without using map information, obtaining the curvature of the white line, and estimating a white line in the extending direction outside the range of the captured image. 【Prior Art Documents】 【Patent Documents】 【0004】 【Patent Document 1】 Japanese Unexamined Patent Application Publication No. 2016 - 43837 【Summary of the Invention】 【Problems to be Solved by the Invention】 【0005】 The conventional technology described above estimates that the curvature of a curved section is constant in the direction of extension, and extracts white lines to recognize the road shape in the direction of extension. Therefore, while it can handle road shapes where the white lines are continuous, such as curves, it cannot handle shapes with multiple exits, such as intersections, because the white lines are not continuous. If the road shape of intersections and the like cannot be understood, it is impossible to generate a route without using high-precision map information. 【0006】 On the other hand, with cameras mounted on moving vehicles, it is not possible to capture images of the intersection exit before entering the intersection. Therefore, even in road structures such as intersections where the white lines are not continuous, a mechanism is needed to generate a suitable driving path from the captured images. 【0007】 The present invention has been made in view of the above problems, and aims to suitably generate a driving route even in road structures that have exits involving changes in direction, such as intersections, without using highly accurate map information. [Means for solving the problem] 【0008】 According to the present invention, a mobile body control system comprising: an imaging means for acquiring an image of the travel area to which the mobile body is traveling; a recognition means for recognizing the road shape included in the image; and the recognition means From the aforementioned captured image A trajectory generation means that generates the trajectory of the moving object based on the recognized road shape, , a travel control means that controls the movement of the moving body according to the trajectory generated by the trajectory generating means, The orbit generation means is equipped with, From the captured image, a trajectory is generated according to the travel area of ​​the moving body, and while the moving body is traveling along the trajectory, The recognition means determines that the road shape having an entry section and at least one exit section involving a change of direction is From the aforementioned captured image Once recognized, the first trajectory from the current position of the moving object to the entry point and the destination of the moving object from the entry point are identified according to instruction information. , recognized from the captured image The invention is characterized by generating a second orbital up to the advance portion. [Effects of the Invention] 【0009】 According to the present invention, a suitable driving route can be generated even in road structures that have exits involving changes in direction, such as intersections, without using highly accurate map information. 【0010】 Other features and advantages of the present invention will become apparent from the following description with reference to the accompanying drawings. In the accompanying drawings, the same or similar components are given the same reference numeral. [Brief explanation of the drawing] 【0011】 The attached drawings are included in the specification and constitute part thereof, illustrating embodiments of the present invention and are used together with the description to explain the principles of the present invention. [Figure 1A] Block diagram showing an example of the hardware configuration of the mobile unit according to this embodiment. [Figure 1B] Block diagram showing an example of the hardware configuration of the mobile unit according to this embodiment. [Figure 2] Block diagram showing the control configuration of the mobile body according to this embodiment. [Figure 3] Block diagram showing the functional configuration of the mobile body according to this embodiment. [Figure 4A] Figure showing captured images according to this embodiment. [Figure 4B] This figure shows the road shape of the captured image according to this embodiment. [Figure 5] This figure shows an example of a method for generating the track of an intersection according to this embodiment. [Figure 6] This figure shows an example of a method for generating the track of an intersection according to this embodiment. [Figure 7] This figure shows an example of the trajectory generation procedure for an intersection according to this embodiment. [Figure 8] Flowchart showing the processing procedure for controlling the movement of the mobile body according to this embodiment. [Figure 9] Flowchart showing the processing procedure for trajectory generation according to this embodiment. [Figure 10] Flowchart showing the processing procedure for generating trajectory within an intersection according to this embodiment. [Modes for carrying out the invention] 【0012】 Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims, and not all combinations of the features described in the embodiments are essential for the invention. Two or more of the features described in the embodiments may be arbitrarily combined. Also, the same or similar configurations are assigned the same reference numerals, and duplicate descriptions are omitted. 【0013】 <Configuration of the Moving Body> Referring to FIG. 1, the configuration of the moving body 100 according to the present embodiment will be described. FIG. 1A shows a side view of the moving body 100 according to the present embodiment, and FIG. 1B shows the internal configuration of the moving body 100. In the figure, the arrow X indicates the front-rear direction of the moving body 100, F indicates the front, and R indicates the rear. The arrows Y and Z indicate the width direction (left-right direction) and the vertical direction of the moving body 100. 【0014】 The moving body 100 includes a traveling unit 12 and is a ultra-compact mobility that moves mainly by the power of a motor with the battery 13 as the main power source. Ultra-compact mobility means a vehicle that is more compact than a general automobile and has a seating capacity of about 1 or 2 persons. In the present embodiment, as an example of the moving body 100, a four-wheel ultra-compact mobility will be described, but the present invention is not intended to be limited, and for example, a three-wheeled vehicle or a straddle-type vehicle may be used. Further, the moving body of the present invention is not limited to a vehicle, and may be a moving body that carries a load and runs parallel to a person's walking, or a moving body that leads a person. Furthermore, the present invention is applicable not only to vehicles such as four-wheeled and two-wheeled vehicles, but also to walking robots capable of autonomous movement. 【0015】 The battery 13 is a secondary battery such as a lithium-ion battery, and the mobile vehicle 100 is propelled by the power supplied from the battery 13 by the driving unit 12. The driving unit 12 is a four-wheeled vehicle equipped with a pair of front wheels 20 and a pair of rear wheels 21. The driving unit 12 may also be in other forms, such as a tricycle. The mobile vehicle 100 is equipped with a seat 14 for one or two people. An operating unit 25 is provided in front of the seat 14 for the occupant to input direction indications. The operating unit 25 is any device that indicates the direction of movement of the mobile vehicle 100, and a device that allows multi-directional input, such as a joystick, can be applied. The driver can, for example, before entering a road shape with an exit that requires a change of direction, such as an intersection, indicate which direction to exit from by operating the operating unit 25. 【0016】 The driving unit 12 is equipped with a steering mechanism 22. The steering mechanism 22 is a mechanism that changes the steering angle of a pair of front wheels 20 using a motor 22a as the drive source. By changing the steering angle of the pair of front wheels 20, the direction of travel of the mobile body 100 can be changed. The driving unit 12 is also equipped with a drive mechanism 23. The drive mechanism 23 is a mechanism that rotates a pair of rear wheels 21 using a motor 23a as the drive source. By rotating the pair of rear wheels 21, the mobile body 100 can be moved forward or backward. 【0017】 The mobile body 100 is equipped with detection units 15 to 17 that detect targets in its vicinity. The detection units 15 to 17 are a group of external sensors that monitor the area around the mobile body 100. In this embodiment, each is an imaging device that captures an image of the area around the mobile body 100, and for example, it comprises an optical system such as a lens and an image sensor. However, radar or lidar (Light Detection and Ranging) can be used instead of or in addition to the imaging device. 【0018】 Two detection units 15 are positioned at the front of the mobile body 100, spaced apart in the Y direction, and primarily detect targets in front of the mobile body 100. Detection units 16 are positioned on the left and right sides of the mobile body 100, respectively, and primarily detect targets to the sides of the mobile body 100. Detection unit 17 is positioned at the rear of the mobile body 100 and primarily detects targets behind the mobile body 100. In this embodiment, an example in which detection units are provided on the front, rear, left, and right sides of the mobile body 100 is described, but there is no intention to limit the present invention, and it is also possible to configure the mobile body 100 to be provided only in some directions (for example, forward). 【0019】 In this embodiment, the mobile body 100 uses at least a detection unit 15 to image the area in front of the mobile body 100, extracts the road shape from the captured image, and generates a route according to recognition information indicating the extracted road shape, operation instructions from the driver's operation unit 25, or information on route changes obtained from the route plan to the destination. The recognition information is output by a machine learning model that processes the image information (captured image). The machine learning model performs calculations using a deep learning algorithm, for example, a deep neural network (DNN), to recognize the road shape included in the image information. The recognition information includes various road lines and lane information, the lane in which the vehicle is located (Ego lane), various intersections (such as crossings), and various road entrances (Road entrance). 【0020】 <Control configuration for mobile devices> Figure 2 is a block diagram of the control system of the mobile body 100 according to this embodiment. Here, the configurations necessary for carrying out the present invention will be mainly described. Therefore, other configurations may be included in addition to the configurations described below. Also, in this embodiment, each part described below is described as being included in the mobile body 100, but there is no intention to limit the present invention, and it may be realized as a mobile body control system including multiple devices. For example, some functions of the control unit 30 may be realized by a server device that is connected to communicate, and detection units 15 to 17 and GNSS sensor 34 may be provided as external devices. The mobile body 100 includes a control unit (ECU) 30. The control unit 30 includes a processor represented by a CPU, a storage device such as a semiconductor memory, and an interface with an external device. The storage device stores programs executed by the processor and data used by the processor for processing. Multiple sets of processors, storage devices, and interfaces may be provided according to the functions of the mobile body 100 and configured to communicate with each other. 【0021】 The control unit 30 acquires detection results from detection units 15-17, input information from the operation panel 31, voice information input from the voice input device 33, position information from the GNSS sensor 34, direction indication information from the operation unit 25, and received information via the communication unit 36, and executes corresponding processing. The control unit 30 controls the motors 22a and 23a (driving control of the driving unit 12), controls the display of the operation panel 31, provides voice notifications to the occupants of the mobile body 100 via the speaker 32, and outputs information. 【0022】 The voice input device 33 captures the voices of the occupants of the mobile unit 100. The control unit 30 recognizes the input voice and can execute corresponding processing. The GNSS (Global Navigation Satellite system) sensor 134 receives GNSS signals and detects the current position of the mobile unit 100. The storage device 35 is a storage device that stores images captured by the detection units 15-17, obstacle information, previously generated routes, and occupied grid maps. The storage device 35 may also store programs executed by the processor and data used by the processor for processing. The storage device 35 may also store various parameters of machine learning models for speech recognition and image recognition executed by the control unit 30 (for example, trained parameters and hyperparameters of a deep neural network). 【0023】 The communication unit 36 ​​communicates with an external device, the communication device 120, via wireless communication such as Wi-Fi or fifth-generation mobile communication. The communication device 120 is, for example, a smartphone, but is not limited to this; it may also be an earphone-type communication terminal, a personal computer, a tablet device, a game console, etc. The communication device 120 connects to a network via wireless communication such as Wi-Fi or fifth-generation mobile communication. 【0024】 A user who owns the communication device 120 can issue instructions to the mobile body 100 via the communication device 120. Such instructions include, for example, instructions to summon the mobile body 100 to a location desired by the user and meet up. Upon receiving such instructions, the mobile body 100 sets a target location based on the location information included in the instructions. In addition to such instructions, the mobile body 100 can also set a target location from images captured by detection units 15-17, or from instructions given by a user riding in the mobile body 100 via the operation panel 31. When setting a target location from captured images, for example, a person raising their hand towards the mobile body 100 is detected in the captured image, and the position of the detected person is estimated and set as the target location. 【0025】 <Functional configuration of the mobile unit> Next, with reference to Figure 3, the functional configuration of the mobile body 100 according to this embodiment will be described. The functional configuration described here is realized in the control unit 30 by, for example, the CPU reading a program stored in memory such as ROM into RAM and executing it. Note that the functional configuration described below will only describe the functions necessary for explaining the present invention and will not describe all of the functional configurations actually included in the mobile body 100. In other words, the functional configuration of the mobile body 100 according to the present invention is not limited to the functional configuration described below. 【0026】 The user instruction acquisition unit 301 has the function of receiving instructions from the user and can receive user instructions via the operation unit 25 or operation panel 31, user instructions from external devices such as the communication device 120 via the communication unit 36, and instructions uttered by the user via the voice input device 33. As described above, user instructions include instructions to set the target position (also called the destination) of the mobile body 100 and instructions related to the driving control of the mobile body 100. 【0027】 The image information processing unit 302 processes the captured images acquired by the detection units 15-17. Specifically, the image information processing unit 302 extracts the recognized road shape from the captured images acquired by the detection units 15-17. The image information processing unit 302 may also include a machine learning model for processing image information and may perform processing during the training and inference phases of the machine learning model. The machine learning model of the image information processing unit 302 can perform processing to recognize road shapes and other information contained in the image information by performing calculations using a deep learning algorithm, for example, a deep neural network (DNN). The recognized information indicating the recognized road shape includes, for example, information indicating lines such as white lines, lanes, intersection shapes, and entrances and exits to intersections. 【0028】 The trajectory generation unit 303 generates a travel path (trajectory) for the mobile body 100 relative to the target position set by the user instruction acquisition unit 301. Specifically, the trajectory generation unit 303 generates a trajectory based on road shape (recognition information) recognized from images captured by detection units 15-17 and direction indication information via the operation unit 25, without requiring obstacle information from a high-precision map. The recognition information is information about the road shape within a predetermined range from the mobile body 100, and cannot recognize road shapes further away. On the other hand, the recognition information is information that is periodically updated as the mobile body 100 moves. Therefore, distant areas are gradually recognized in accordance with the movement of the mobile body 100. The trajectory generation unit 303 sequentially generates a trajectory according to the updated recognition information. Furthermore, the direction indication information is not limited to information received via the operation unit 25, but may also be based on information about changes in course obtained by route planning to the destination. Therefore, the present invention does not require the operation unit 25 as an essential component and can be applied to mobile bodies, etc., that do not have an operation unit 25. 【0029】 Furthermore, the speed planning unit 304 plans the speed according to the curvature of the track generated by the track generation unit 303, and also plans the speed based on the direction indication given by the driver. For example, when a driver is instructed to turn right or left at an intersection, the speed plan controls the vehicle to decelerate to 8 km / h for a right turn and to 6 km / h for a left turn before starting the curve from a straight line. By controlling the speed in accordance with the generated track and the driver's instructions, sudden deceleration can be avoided. 【0030】 The running control unit 305 controls the movement of the mobile body 100 according to the generated track and speed plan. Specifically, the running control unit 305 controls the speed and angular velocity of the mobile body 100 by controlling the running unit 12 according to the track and speed plan. If the driving plan of the track is deviated due to the operator's actions, the running control unit 305 may acquire a new track generated again by the track generation unit 303 and control the movement, or it may control the speed and angular velocity of the mobile body 100 to eliminate the deviation from the track in use. 【0031】 <Image> Figure 4A shows an image captured according to this embodiment. Figure 4B shows an example of a road shape included in the image captured in Figure 4A. The image captured in Figure 4A, 400, is an image captured by a detection unit 15 located in front of the mobile body 100. The shaded area 401 shows the inside of the cockpit of the mobile body 100 as captured in the image 400. The area other than the shaded area 401 is the area where the surrounding environment extending in front of the mobile body 100 has been captured. 【0032】 Figure 4B shows the road shape included in the captured image 400 shown in Figure 4A. The dotted line area 410 indicates a three-way intersection (T-junction). As shown in Figure 4B, in the area in front of the moving body 100, there is a three-way intersection, and there are two exits from the intersection: one by going straight ahead and another by turning right. Beyond the exit when going straight through the intersection, the road continues with a large curve to the right. As shown in the captured image 400, from the viewpoint of the moving body 100 before entering intersection 410, it is possible to recognize that there are multiple exits indicated by the arrows, but the road shape beyond the exits cannot be recognized. Therefore, as the moving body 100 according to this embodiment moves, it generates a trajectory sequentially or in stages using the road shape that has become clear. In other words, as the moving body 100 approaches a predetermined road shape, the accuracy of its recognition increases, and it generates a trajectory according to the degree of recognition. Furthermore, in this embodiment, if the road shape recognized from the captured image 400 includes multiple access points, or if at least one access point is located outside a predetermined range from the current direction of travel, the mobile body 100 determines that the road shape includes at least one access point that involves a change of course. In this case, according to this embodiment, it is determined that there is a possibility of a change of course, and the trajectory generation described later is performed. 【0033】 <Trajectory generation method> Figures 5 and 6 show the trajectory generation method according to this embodiment. Here, we will explain the trajectory generation when a right turn instruction is received via the operation unit 25 when approaching the intersection (T-junction) shown in Figure 4B. In this embodiment, a T-junction is used as an example of a road shape having an entry section and an exit section with a change of course, but the present invention may also describe road shapes having an entry section and an exit section with a change of course, such as intersections, T-junctions, entrances to facilities along the road, or L-shaped driving areas. Entrances to facilities along the road include, for example, entrances to shopping malls, gas stations, parking lots, etc. 【0034】 501 indicates the trajectory at the intersection to be generated. Here, as shown in FIG. 5, let the current position (or the intersection entrance) of the moving object 100 be xs, and the target intersection exit be xg. When receiving a right-turn instruction via the operation unit 25, in order to generate a trajectory for crossing the intersection, the trajectory generation unit 303 acquires the intersection point 504 between the center line 502 of the lane on which the moving object 100 is traveling and the center line 503 of the target lane. Here, the target lane indicates the lane following the exit when the moving object 100 passes through the intersection. 【0035】 Subsequently, the trajectory generation unit 303 acquires the distance d0 between the current position xs of the moving object 100 and the acquired intersection point 504, and the distance dg between the intersection exit xg and the intersection point 504. Further, the trajectory generation unit 303 compares the acquired distance d0 with the distance dg, and when d0 < dg, determines to generate a trajectory that becomes a straight line following a single curve, and when d0 > dg, determines to generate a trajectory that becomes a single curve following a straight line. In the case of d0 = dg, it is determined to generate a trajectory consisting only of a single curve. 【0036】 Next, the trajectory generation unit 303 acquires the start point and the end point of the single curve, and acquires the radius R therefrom. After that, the trajectory generation unit 303 generates a trajectory including the single curve and, if necessary, a straight line. The example shown in FIG. 5 shows the state of generating a trajectory 501 that becomes a single curve following a straight line in the case of d0 > dg. On the other hand, the example shown in FIG. 6 shows the state of generating a trajectory 601 that becomes a straight line following a single curve in the case of d0 < dg. Here, an example of a single curve + straight line or straight line + single curve has been described, but there is no intention to limit the present invention. For example, depending on the current position of the moving object 100 and the shape of the intersection, a trajectory consisting only of a single curve or a straight line + single curve + straight line may be generated. Here, for the sake of easy explanation, a single curve has been described as an example of the curve, but other curves such as a clothoid curve or a cubic curve may be generated instead of or in addition to the single curve. 【0037】 <Trajectory generation procedure according to the recognition situation> Figure 7 shows the trajectory generation procedure according to the recognition status of the intersection according to this embodiment. Here, the trajectory generation procedure when approaching the intersection (T-junction) shown in Figure 4B will be explained. According to this embodiment, the trajectory when passing through a road shape such as an intersection having an entry section and an exit section with a change of course is generated sequentially (in stages) according to the road shape recognition information obtained from the captured image. 【0038】 Here, we will describe an example of trajectory control performed in four stages depending on the distance between the mobile body 100 and the intersection. As shown in Figure 7, Phase 0 is the state where the distance from the mobile body 100 to the intersection is greater than 30m. In this state, the image information processing unit 302 recognizes the "Ego lane" indicating the travel area where the mobile body 100 is traveling from the captured images acquired by the detection units 15-17, but does not recognize the road shape that constitutes the intersection. 【0039】 Phase 1 occurs when the distance from the mobile body 100 to the intersection is less than 30m, and the operating unit 25 has given a command to turn right. In this state, the image information processing unit 302 can recognize the intersection in addition to the "Ego lane" mentioned above. However, in the intersection recognition information at this stage, the shape of the intersection and the entry point ("Road entrance") are recognized, but the exit point and the driving area beyond that are not clearly recognized. 【0040】 The road shape recognition information (for example, "Intersection" indicating an intersection) extracted by the image information processing unit 302 using a machine learning model includes various parameters depending on the recognition status. These parameters include, for example, "Road entrance" indicating an entry point or an exit point involving a change of direction, information indicating the boundary of an "Intersection", information indicating the white lines of the nearby road shape, and information indicating lanes (driving areas). In other words, depending on the recognition status, it may only include parameters indicating the shape (boundary) of an "Intersection", and it may not be possible to recognize the lanes or lines of the branching point. 【0041】 Therefore, the track generation unit 303 needs to sequentially generate and update the track according to such recognition conditions. In this Phase 1, although a right turn instruction has been received, the exit point of the intersection and the driving area beyond it have not yet been identified. Therefore, the track generation unit 303 maintains the current track and does not generate a track that changes the direction. 【0042】 Phase 2 occurs when the distance from the mobile body 100 to the intersection is less than 20m, and the operating unit 25 has given a command to turn right. In this state, the image information processing unit 302 recognizes the target lane ahead, in addition to the recognition information from Phase 1, and extracts the entry point into that lane. However, although the target lane is recognized, the line indicating the boundary of that lane is not recognized, and the accuracy of the recognition of the lane is low. On the other hand, the road entrance to the intersection when making the commanded right turn is clearly recognized. For example, in Phase 2, the vehicle is approaching the intersection, the boundary of the intersection on the right-turn side can be clearly recognized, the upper half can be estimated as the driving lane (target lane), and the lower half can be estimated as the oncoming lane. 【0043】 Therefore, in Phase 2, the trajectory generation unit 303 generates a trajectory for the change of course based on information that can be recognized with higher accuracy. Specifically, the trajectory generation unit 303 generates a first trajectory from the current position of the moving body 100 to the already recognized intersection entrance, and a second trajectory from the entrance to the right-turn exit of the intersection, the "Road entrance". The second trajectory is generated by the trajectory generation method described with reference to Figures 5 and 6, and includes curved trajectories such as simple curves, clothoid curves, and cubic curves. The second trajectory may also include straight lines in addition to such curves. Furthermore, the second trajectory may be generated as a trajectory from the center of the entrance to the center of the specified exit. 【0044】 Phase 3 is the state in which the moving object 100 enters the intersection. In this state, in addition to the recognition information from Phase 2, the image information processing unit 302 can further recognize the white line (line) that indicates the boundary of the driving area "Target lane" ahead, and can recognize the driving area "Target lane" more accurately than in Phase 2. Therefore, in Phase 3, the trajectory generation unit 303 generates a third trajectory that follows the second trajectory generated in Phase 2, and is a third trajectory within the recognized driving area. This makes it possible to generate a trajectory when a right turn instruction is given at the intersection. 【0045】 In Phase 3 described above, an example was explained in which the third trajectory is generated at the timing when the driving area "Target lane" is more accurately determined by recognizing the white lines of the driving area "Target lane" (for example, the lines indicating the boundary of the lane, "Lane instance") after the mobile body 100 has entered the intersection. However, this does not limit the present invention. For example, instead of generating the third trajectory at the timing when the driving area "Target lane" is determined as described above, the trajectory generation unit 303 may generate the third trajectory when the mobile body 100 has moved beyond the intersection entry point, determining that the driving area "Target lane" has been determined to some extent. The timing when the mobile body 100 has moved beyond the intersection entry point is, in other words, the timing when it starts traveling on the second trajectory. Alternatively, the trajectory generation unit 303 may generate the third trajectory when the mobile body 100 has approached a predetermined distance from the specified right-turn entry point, determining that the driving area "Target lane" has been determined to some extent. 【0046】 Furthermore, while the present invention has described an example in which a change of course, such as turning right or left, is determined based on direction information from the operation unit 25, the present invention is not limited to this. For example, when planning a route according to a pre-set destination, if it is necessary to turn right at the next intersection, the trajectory generation unit 303 may determine that a right turn is necessary and generate a trajectory without even receiving the direction information described above. Also, for example, if the direction of travel is a dead end and a left or right turn is necessary, but direction information from the operation unit 25 has not been received, the system may determine that a left or right turn is necessary and generate a trajectory. In this case, for example, a change of course toward the destination may be selected in relation to the route plan to the destination. 【0047】 Furthermore, according to this embodiment, the speed planning unit 304 plans the target speed for each phase, as shown in the "Speed" row of Figure 7. For example, when an instruction to turn right or left is received while an intersection is recognized, a change of course occurs, so it is necessary to decelerate to a predetermined speed (for example, 8 km / h for a right turn, and 6 km / h for a left turn) before entering the intersection. Therefore, it is desirable for the speed planning unit 304 to decelerate gradually in each phase in order to avoid sudden deceleration. 【0048】 <Basic Flow> Figure 8 is a flowchart showing the basic control of the mobile body 100 according to this embodiment. The processes described below are realized in the control unit 30 by, for example, the CPU reading a program stored in memory such as ROM into RAM and executing it. 【0049】 In S101, the control unit 30 sets the target position of the moving body 100 based on the user instruction received by the user instruction acquisition unit 301. As described above, the user instruction can be received in various ways. Next, in S102, the control unit 30 acquires direction instruction information. This includes direction instruction information when the driver operates the operation unit 25, and direction instruction information for course changes determined according to the set target position. Here, for convenience, the process of acquiring direction instruction information is described as taking place in S102, but in reality, it is a process that occurs as an interrupt process at any time when the driver operates the operation unit 25. Therefore, even after the process in S102, direction instruction information is acquired by operation interrupts and used for trajectory generation. 【0050】 Next, in S103, the control unit 30 uses the detection unit 15 to image the area in front of the moving object 100 (direction of travel) and acquires the image. Then, in S104, the control unit processes the acquired image using the image information processing unit 302 and obtains recognition information indicating the road shape recognized using a machine learning model. The processing in S103 and S104 is performed continuously or periodically, and the image and the recognition information acquired from the image are updated as needed. 【0051】 Next, in S105, the control unit 30 generates a trajectory for the mobile body 100 using the trajectory generation unit 303, according to the recognition information acquired in S104. The detailed procedure for trajectory generation by the trajectory generation unit 303 will be described later with reference to Figure 9. Subsequently, in S106, the control unit 30 generates a speed plan for the mobile body 100 based on the generated trajectory and direction information. Furthermore, in S107, the control unit 30 determines the speed and angular velocity of the mobile body 100 using the travel control unit 305 and controls its movement. After that, in S108, the control unit 30 determines whether the mobile body 100 has reached the target position based on the position information from the GNSS sensor 34. If it has not reached the target position, the process returns to S102, and the process of generating a trajectory and controlling its movement is repeated while updating the captured image. On the other hand, if it has reached the target position, the process in this flowchart ends. 【0052】 <Orbital generation process procedure> Figure 9 is a flowchart showing the detailed processing procedure for trajectory generation (S105) according to this embodiment. The processing described below is realized in the control unit 30 by, for example, the CPU reading a program stored in memory such as ROM into RAM and executing it. 【0053】 First, in S201, the trajectory generation unit 303 determines whether the recognition information acquired in S104 includes information indicating an intersection or the like that has an exit section that involves a change of direction. Specifically, if the road shape indicated by the acquired recognition information includes multiple exit sections, or if at least one exit section is located outside a predetermined range from the current direction of travel, the trajectory generation unit 303 determines that the road shape includes at least one exit section that involves a change of direction. The reason for specifying "at least one" exit section here is that the moving body 100 may recognize other exit sections as it approaches an intersection or the like. In other words, the number of recognized exit sections may increase as the moving body 100 moves, and this takes into account the possibility that the number of exit sections that involve a change of direction may increase. If at least one exit section that involves a change of direction is included, the process proceeds to S203; otherwise, it proceeds to S202. In S202, the trajectory generation unit 303 generates a trajectory within a predetermined range along the lane of the moving vehicle, and the processing of this flowchart ends. 【0054】 Meanwhile, in S203, the track generation unit 303 determines whether it has acquired direction indicator information in S102. If it has not, it proceeds to S202; if it has, it proceeds to S204. In S204, the track generation unit 303 determines whether it has recognized the specific entry point identified by the acquired direction indicator information among the multiple entry points included in the intersection, etc. If it has not been recognized, it proceeds to S202; if it has been recognized, it proceeds to S205. 【0055】 In S205, the trajectory generation unit 303 generates a first trajectory from the current location of the moving body 100 to the entrance of an intersection, etc., as a control to generate a trajectory for a change of course. The first trajectory is basically a straight line, connecting the center of the vehicle's lane and the center of the entrance, which has a predetermined width. Subsequently, in S206, the trajectory generation unit 303 generates a second trajectory within the intersection that follows the first trajectory. Details will be described later using Figure 10. 【0056】 Next, in S207, the trajectory generation unit 303 determines whether it has recognized the target travel area located beyond the identified advance section. Whether the target travel area has been recognized is determined according to each parameter of the road shape recognized by the image information processing unit 302, as described above. Here, for example, if a white line for identifying the shape of the target travel area is recognized, it is determined that the travel area has been recognized. Alternatively, instead of such a determination, the travel area may be determined to have been recognized at the timing when the second trajectory starts moving, or when the distance from the moving body 100 to the identified advance section approaches a predetermined distance. If the target travel area has been recognized, the process proceeds to S208, where the trajectory generation unit 303 generates a third trajectory from the identified advance section according to the recognized travel area, and returns the process to S104. 【0057】 <Trajectory generation at intersections, etc.> Figure 10 is a flowchart showing the detailed processing procedure for generating the second trajectory (S206) within the intersection according to this embodiment. The processing described below is realized in the control unit 30 by, for example, the CPU reading a program stored in memory such as ROM into RAM and executing it. 【0058】 In S301, the trajectory generation unit 303 obtains the start and end points of the second trajectory. The start point of the second trajectory is, for example, the center of the entry section. The end point is, for example, the center of the specified exit section. Subsequently, in S302, the trajectory generation unit 303 obtains the intersection point between a straight line along the direction of travel of the moving body 100 (for example, the center line of the vehicle's lane) and a straight line along the direction of travel of the destination (for example, the center line of the target lane). Subsequently, in S303, the trajectory generation unit 303 obtains the first distance d0 from the start point to the intersection point and the second distance dg from the intersection point to the end point. 【0059】 Subsequently, in S304 and S305, the trajectory generation unit 303 compares the first distance d0 with the second distance dg to determine whether to generate a straight line in addition to a simple curve. Specifically, if the first distance d0 and the second distance dg are equal, the process proceeds to S306, where the trajectory generation unit 303 decides to generate a simple curve as the path of the curve and proceeds to S309. If the first distance d0 is shorter than the second distance dg, the process proceeds to S307, where the trajectory generation unit 303 decides to generate a simple curve from the starting point as the path of the curve, and then generate a straight line to the endpoint following the generated simple curve, and proceeds to S309. If the first distance d0 is longer than the second distance dg, the process proceeds to S308, where the trajectory generation unit 303 decides to generate a straight line from the starting point as the path of the curve, and then generates a simple curve to the endpoint following the generated straight line, and proceeds to S309. In S309, the trajectory generation unit 303 obtains the radius R of the simple curve to be generated. Next, in S310, the trajectory generation unit 303 generates a path including curves according to the decisions made in S306 to S308, and returns the process to S306. 【0060】 <Summary of Embodiments> 1. The mobile control system (e.g., 100) of the above embodiment is: An imaging means for acquiring images of the travel area at the destination of a moving object, and (for example, 15-17) Recognition means for recognizing the road shape included in the captured image, and (for example, 302) A trajectory generating means (for example, 303) generates the trajectory of the moving body based on the road shape recognized by the recognition means. Equipped with, The trajectory generation means is characterized in that, when the recognition means recognizes a road shape having an entry section and at least one exit section with a change of course, it generates a first trajectory from the current position of the moving body to the entry section and a second trajectory from the entry section to the exit section specified according to instruction information regarding the destination of the moving body (for example, 303, S205, S206, Figure 7). 【0061】 According to this embodiment, a suitable driving route can be generated even in road structures with multiple exits, such as intersections, without using highly accurate map information. 【0062】 2. In the mobile body control system of the above embodiment, the trajectory generating means generates a trajectory as the second trajectory from the center of the entry section to the center of the specified exit section (for example, Figures 5 and 6). 【0063】 According to this embodiment, a suitable driving route can be generated even in road structures that have exits involving changes in direction, such as intersections, without using highly accurate map information. 【0064】 3. In the mobile body control system of the above embodiment, when the mobile body is traveling along the first or second trajectory, and the recognition means recognizes a travel area following the identified advance portion, the trajectory generation means generates a third trajectory following the second trajectory, which is within the recognized travel area (for example, S208, Figure 7). 【0065】 According to this embodiment, a suitable driving route can be generated even in road structures that have exits involving changes in direction, such as intersections, without using highly accurate map information. 【0066】 4. In the mobile body control system of the above embodiment, when the mobile body starts traveling along the second trajectory, the trajectory generation means generates a third trajectory that follows the second trajectory and is within the travel area following the identified advance portion recognized by the recognition means (for example, S208, Figure 7). 【0067】 According to this embodiment, a suitable driving route can be generated even in road structures that have exits involving changes in direction, such as intersections, without using highly accurate map information. 【0068】 5. In the mobile body control system of the above embodiment, when the mobile body approaches a predetermined distance from the identified advance portion, the trajectory generation means generates a third trajectory following the second trajectory, which is within the travel area following the identified advance portion recognized by the recognition means (for example, S208, Figure 7). 【0069】 According to this embodiment, a suitable driving route can be generated even in road structures that have exits involving changes in direction, such as intersections, without using highly accurate map information. 【0070】 6. In the mobile vehicle control system of the above embodiment, the road shape having the entry section and the plurality of exit sections is any of the following: an intersection, a T-junction, and a driving area including an entrance to a facility along the road. 【0071】 According to this embodiment, if the road shape has an entry section and an exit section involving a change of direction, a suitable driving route can be generated without using highly accurate map information. 【0072】 7. The mobile body control system of the above embodiment further comprises a direction indicating means (for example, 25) for receiving the instruction information regarding the destination of the mobile body. 【0073】 According to this embodiment, it is possible to generate a driving path that corresponds to the driver's intentions. 【0074】 8. In the mobile body control system of the above embodiment, if the trajectory generation means has not received the instruction information from the direction instruction means and a change of course is necessary, it generates the second trajectory up to the advance section specified in response to the change of course. 【0075】 According to this embodiment, when a change of route occurs according to the route plan, or when traveling on a road with a structure that requires a change of route, a suitable travel route can be generated without using highly accurate map information. 【0076】 9. In the above embodiment of the mobile body control system, the trajectory generation means generates a trajectory that includes at least a simple curve as the second trajectory when a change in course occurs (for example, Figures 5, 6, and 10). 【0077】 According to this embodiment, a smooth trajectory can be easily generated even when a lane change occurs at an intersection or the like. 【0078】 10. In the mobile body control system of the above embodiment, a speed planning means generates a speed plan for the mobile body according to the trajectory generated by the trajectory generating means and the instruction information relating to the destination of the mobile body, and (for example, 304) A travel control means (for example, 305) controls the movement of the moving body according to the trajectory generated by the trajectory generating means and the speed plan generated by the speed planning means. To further prepare. 【0079】 According to this embodiment, even when lane changes occur at intersections, a smooth trajectory can be easily generated, and sudden deceleration can be avoided. 【0080】 11. In the mobile control system of the above embodiment, the trajectory generation means determines that the road shape recognized by the recognition means includes at least one advance section that involves a change of course if the road shape recognized by the recognition means includes multiple advance sections, or if at least one advance section is located outside a predetermined range from the current direction of travel (for example, 303, S205, S206, Figure 7). 【0081】 According to this embodiment, intersections and the like involving lane changes can be suitably identified. 【0082】 The invention is not limited to the embodiments described above, and various modifications and changes are possible within the scope of the gist of the invention.

Claims

[Claim 1] A mobile control system, An imaging means for acquiring images of the travel area at the destination of a moving object, Recognition means for recognizing the road shape included in the captured image, A trajectory generation means generates the trajectory of the moving object based on the road shape recognized from the captured image by the recognition means, A travel control means that controls the movement of the moving body according to the trajectory generated by the trajectory generating means. Equipped with, A mobile body control system characterized in that the trajectory generation means generates a trajectory from the captured image according to the travel area of ​​the mobile body, and when the recognition means recognizes from the captured image a road shape having an entry section and at least one exit section with a change of course while the mobile body is traveling along the trajectory, it generates a first trajectory from the current position of the mobile body to the entry section and a second trajectory from the entry section to the exit section recognized from the captured image, which is specified according to instruction information regarding the destination of the mobile body. [Claim 2] The mobile body control system according to claim 1, characterized in that the trajectory generating means generates a trajectory as the second trajectory, from the center of the entry portion to the center of the specified exit portion. [Claim 3] The mobile body control system according to claim 2, characterized in that when the mobile body is traveling along the first or second trajectory, and the recognition means recognizes a travel area following the identified advance portion, the trajectory generation means generates a third trajectory following the second trajectory, which is within the travel area recognized from the captured image. [Claim 4] The mobile body control system according to claim 2, characterized in that when the mobile body starts traveling along the second trajectory, the trajectory generating means generates a third trajectory that follows the second trajectory and is within a travel area following the identified advancing portion recognized by the recognition means from the captured image. [Claim 5] The mobile body control system according to claim 2, characterized in that when the mobile body approaches a predetermined distance from the identified advance portion, the trajectory generating means generates a third trajectory following the second trajectory, which is within the travel area following the identified advance portion recognized by the recognition means from the captured image. [Claim 6] The mobile body control system according to claim 1, characterized in that the road shape having the entry section and at least one exit section with a change of course is any of an intersection, a T-junction, and a driving area including an entrance to a facility along the road. [Claim 7] The mobile body control system according to claim 1, further comprising a direction indicating means for receiving the instruction information regarding the destination of the mobile body. [Claim 8] The mobile body control system according to claim 7, characterized in that the trajectory generating means generates a second trajectory to the advance portion specified in response to the change in direction when the direction indicating means has not received the instruction information and a change in direction is necessary. [Claim 9] The mobile body control system according to claim 2, characterized in that the trajectory generating means generates a trajectory including at least a simple curve as the second trajectory when a change in course occurs. [Claim 10] A speed planning means that generates a speed plan for the moving body according to the trajectory generated by the trajectory generating means and the instruction information relating to the destination of the moving body, A travel control means that controls the movement of the moving body according to the trajectory generated by the trajectory generating means and the speed plan generated by the speed planning means. The mobile body control system according to claim 1, further comprising the features described above. [Claim 11] The mobile body control system according to claim 1, characterized in that the trajectory generation means determines that the road shape recognized from the captured image by the recognition means includes at least one advance section that involves a change of course, if the road shape recognized from the captured image by the recognition means includes a plurality of advance sections, or if at least one advance section is located outside a predetermined range from the current direction of travel. [Claim 12] A control method for a mobile control system, An imaging process to acquire images of the travel area at the destination of the moving object, A recognition step of recognizing the road shape included in the captured image, A trajectory generation step is performed to generate the trajectory of the moving object based on the road shape recognized from the captured image in the recognition step, A travel control step which controls the movement of the moving body according to the trajectory generated in the trajectory generation step, Includes, A control method for a mobile body control system, characterized in that, in the trajectory generation step, a trajectory is generated from the captured image according to the travel area of ​​the mobile body, and when a road shape having an entry section and at least one exit section with a change of course is recognized from the captured image in the recognition step while the mobile body is traveling along the trajectory, a first trajectory from the current position of the mobile body to the entry section and a second trajectory from the entry section to the exit section recognized from the captured image, which is specified according to instruction information regarding the destination of the mobile body. [Claim 13] A program for causing a computer to function as one of the means of a mobile control system according to any one of claims 1 to 11. [Claim 14] It is a mobile object, An imaging means for acquiring images of the driving area at the destination, Recognition means for recognizing the road shape included in the captured image, A trajectory generation means generates the trajectory of the moving object based on the road shape recognized from the captured image by the recognition means, A travel control means that controls the movement of the moving body according to the trajectory generated by the trajectory generating means. Equipped with, The trajectory generation means generates a trajectory from the captured image according to the travel area of ​​the moving body, and when the recognition means recognizes from the captured image a road shape having an entry section and at least one exit section with a change of course while the moving body is traveling along the trajectory, the means generates a first trajectory from the current position of the moving body to the entry section and a second trajectory from the entry section to the exit section recognized from the captured image, which is specified according to instruction information regarding the destination of the moving body.

Images