Moving body control system, control method therefor, program, and moving body

Abstract

This moving body control system acquires a captured image based on a travel region of a moving destination of a moving body, recognizes a road shape included in the captured image, and generates a trajectory of the moving body on the basis of the recognized road shape. In the trajectory generation, when a road shape having an entry portion and at least one forward exit portion that changes with a travel route is recognized, a first trajectory from the current position of the moving body to the entry portion and a second trajectory from the entry portion to the forward exit portion determined on the basis of instruction information relating to the movement destination of the moving body are generated.

Description

Technical Field

[0001] The present invention relates to a mobile body control system, a control method thereof, a program, and a mobile body. Background Art

[0002] In recent years, small mobile bodies such as electric vehicles with a seating capacity of about 1 to 2 persons and mobile robots that provide various services to people, which are called ultra-small mobile bodies (also called micro-mobile bodies), have been known. Among the above mobile bodies, there are mobile bodies that autonomously travel while periodically generating a travel path to a destination. In addition, in small mobile bodies, hardware resources are scarce, and it is difficult to ensure an area for storing high-precision map information used for generating a path and a communication device for quickly acquiring a large amount of map information. Therefore, in the above small mobile bodies, it is required to generate a path without using high-precision map information.

[0003] Patent Document 1 proposes the following: Without using map information, a white line at the road edge is extracted from an image acquired by a camera or the like in a curved section, the curvature of the white line is obtained, and a white line in the extending direction outside the range of the captured image is estimated, thereby identifying the road shape.

[0004] Prior Art Documents

[0005] Patent Documents

[0006] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2016-43837 Summary of the Invention

[0007] Problems to be Solved by the Invention

[0008] In the above prior art, the curvature of a curved section is presumed to be constant in the extending direction, and a white line is extracted to identify the road shape in the extending direction. Therefore, although it is possible to handle road shapes where white lines continue, such as curves, for shapes with multiple exits such as intersections, since the white lines are discontinuous, it is not possible to handle them. If the road shape such as an intersection cannot be grasped, it is not possible to generate a path without using high-precision map information.

[0009] On the other hand, in a captured image based on a camera or the like provided in a mobile body, an image of an intersection exit cannot be captured before entering the intersection. Therefore, a mechanism is required that can appropriately generate a travel path based on a captured image even in a road structure such as an intersection where white lines are discontinuous.

[0010] The present invention has been completed in view of the above problems, and can appropriately generate a travel path without using high-precision map information even in a road structure with an exit accompanied by a change in the travel route such as an intersection.

[0011] Means for Solving the Problems

[0012] According to the present invention, there is provided a mobile body control system, characterized in that the mobile body control system includes: a photographing mechanism that acquires a photographed image of a traveling area based on a moving destination of the mobile body; an identification mechanism that identifies a road shape included in the photographed image; and a trajectory generation mechanism that generates a trajectory of the mobile body based on the road shape identified by the identification mechanism. When the identification mechanism identifies a road shape having at least one entrance portion and at least one forward exit portion accompanied by a change in the traveling route, the trajectory generation mechanism generates a first trajectory from the current position of the mobile body to the entrance portion and a second trajectory from the entrance portion to the forward exit portion determined according to instruction information related to the moving destination of the mobile body.

[0013] Advantages of the Invention

[0014] According to the present invention, it is possible to appropriately generate a traveling path without using high-precision map information even in a road structure having an exit accompanied by a change in the traveling route such as an intersection.

[0015] Other features and advantages of the present invention will become apparent from the following description with reference to the accompanying drawings. In the drawings, the same or similar components are denoted by the same reference numerals. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings are included in the specification and form a part of the specification. The drawings illustrate embodiments of the present invention and, together with the description, are used to explain the principles of the present invention.

[0017] Figure 1A It is a block diagram showing a configuration example of the hardware of the mobile body according to the present embodiment.

[0018] Figure 1B It is a block diagram showing a configuration example of the hardware of the mobile body according to the present embodiment.

[0019] Figure 2 It is a block diagram showing a control configuration of the mobile body according to the present embodiment.

[0020] Figure 3 It is a block diagram showing a functional configuration of the mobile body according to the present embodiment.

[0021] Figure 4A It is a diagram showing a photographed image according to the present embodiment.

[0022] Figure 4B It is a diagram showing a road shape of the photographed image according to the present embodiment.

[0023] Figure 5This is a diagram showing an example of the track generation method for an intersection according to this embodiment.

[0024] Figure 6 This is a diagram showing an example of the track generation method for an intersection according to this embodiment.

[0025] Figure 7 This is a diagram showing an example of the track generation steps for an intersection according to this embodiment.

[0026] Figure 8 This is a flowchart showing the processing steps for controlling the travel of a moving body according to this embodiment.

[0027] Figure 9 This is a flowchart showing the processing steps for track generation according to this embodiment.

[0028] Figure 10 This is a flowchart showing the processing steps for track generation within an intersection according to this embodiment. Detailed Embodiment

[0029] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. In addition, the following embodiments do not limit the invention related to the scope of the technical solution, and furthermore, all combinations of the features described in the embodiments are not necessarily required for the invention. Two or more of the features described in the embodiments can be combined arbitrarily. In addition, the same reference numerals are assigned to the same or similar components, and duplicate descriptions are omitted.

[0030] <Configuration of the Moving Body>

[0031] With reference to FIG. 1, the configuration of the moving body 100 according to this embodiment will be described. Figure 1A This shows the side of the moving body 100 according to this embodiment, Figure 1B This shows the internal configuration of the moving body 100. In the figure, the arrow X indicates the front-rear direction of the moving body 100, F indicates the front, and R indicates the rear. The arrow Y indicates the width direction (left-right direction) of the moving body 100, and the arrow Z indicates the up-down direction of the moving body 100.

[0032] The mobile body 100 is a super-small mobile body that includes a traveling unit 12 and moves mainly by the power of a motor using a battery 13 as the main power source. The super-small mobile body refers to a super-small vehicle that is smaller than a general motor vehicle and has a seating capacity of about one or two persons. In the present embodiment, as an example of the mobile body 100, a four-wheeled super-small mobile body is taken as an example for description, but the present invention is not intended to be limited thereto. For example, it may also be a three-wheeled vehicle or a straddle-type vehicle. In addition, the mobile body of the present invention is not limited to a means of transportation, and may also be a mobile body that carries goods and walks in parallel with people or a mobile body that leads people. Furthermore, the present invention is not limited to vehicles such as four-wheeled or two-wheeled vehicles, and can also be applied to a walking-type robot that can move autonomously, etc.

[0033] The battery 13 is, for example, a rechargeable battery such as a lithium-ion battery, and the mobile body 100 travels autonomously through the traveling unit 12 by the electric power supplied from the battery 13. The traveling unit 12 is a four-wheeled vehicle having a pair of left and right front wheels 20 and a pair of left and right rear wheels 21. The traveling unit 12 may also be in the form of a three-wheeled vehicle or other forms. The mobile body 100 includes a seat 14 for one person or two persons. An operation unit 25 for the occupant to input a direction instruction is provided in front of the seat 14. The operation unit 25 is an arbitrary device for instructing the moving direction of the mobile body 100. For example, a device capable of performing multi-directional input such as a joystick can be applied. For example, before entering a road shape with an exit accompanied by a change in the traveling route such as an intersection, the driver can operate the operation unit 25 to indicate which direction to exit.

[0034] The traveling unit 12 includes a steering mechanism 22. The steering mechanism 22 is a mechanism that changes the steering angle of a pair of front wheels 20 using a motor 22a as a drive source. The traveling direction of the mobile body 100 can be changed by changing the steering angle of a pair of front wheels 20. The traveling unit 12 also includes a drive mechanism 23. The drive mechanism 23 is a mechanism that rotates a pair of rear wheels 21 using a motor 23a as a drive source. The mobile body 100 can move forward or backward by rotating a pair of rear wheels 21.

[0035] The mobile body 100 includes detection units 15 to 17 that detect target objects around the mobile body 100. The detection units 15 to 17 are an external sensor group that monitors the periphery of the mobile body 100. In the case of the present embodiment, they are all imaging devices that capture images around the mobile body 100. For example, they include an optical system such as a lens and an image sensor. However, instead of or in addition to the imaging device, radar or lidar (Light Detection and Ranging) can also be used.

[0036] Two detection units 15 are arranged at intervals in the front part of the moving body 100 in the Y direction, mainly for detecting the target in front of the moving body 100. The detection units 16 are respectively arranged on the left side part and the right side part of the moving body 100, mainly for detecting the target on the side of the moving body 100. The detection unit 17 is arranged at the rear part of the moving body 100, mainly for detecting the target behind the moving body 100. In addition, in the present embodiment, an example in which detection units are arranged around the moving body 100 is described, but the present invention is not intended to be limited thereto, and it may also be a configuration in which detection units are arranged only in a part of the directions (for example, the front) of the moving body 100.

[0037] The moving body 100 according to the present embodiment at least uses the detection unit 15 to photograph the front area of the moving body 100, extracts the road shape from the photographed image, and generates a path based on the recognition information representing the extracted road shape, the operation instruction from the operation unit 25 of the driver, or the information related to the travel route change obtained through the path plan to the destination. The recognition information is output by a machine learning model that processes image information (photographed image). The machine learning model performs operations of a deep learning algorithm using a deep neural network (DNN), for example, and performs processing for recognizing the road shape included in the image information. The recognition information includes various lines of the road, various lane information, the lane in which the own vehicle is located (Ego lane), various interfaces (intersections, etc., Intersection), entrances to various roads (Road entrance), etc.

[0038] <Configuration of the control of the moving body>

[0039] Figure 2 It is a block diagram of the control system of the moving body 100 according to the present embodiment. Here, mainly the configurations necessary for implementing the present invention are described. Therefore, other configurations may be further included on the basis of the configurations described below. In addition, in the present embodiment, the following-described respective parts are described as being included in the moving body 100, but the present invention is not intended to be limited thereto, and it may also be implemented as a moving body control system including a plurality of devices. For example, a part of the functions of the control unit 30 may be implemented by a server device communicably connected, and the detection units 15 to 17 and the GNSS sensor 34 may also be provided as external devices. The moving body 100 includes a control unit (ECU) 30. The control unit 30 includes a processor represented by a CPU, a storage device such as a semiconductor memory, an interface with an external device, etc. Programs executed by the processor, data used by the processor for processing, etc. are stored in the storage device. The processor, the storage device, and the interface may also be configured such that a plurality are provided according to the functions of the moving body 100 and can communicate with each other.

[0040] The control unit 30 acquires the detection results of the detection units 15 to 17, the input information of the operation panel 31, the voice information input from the voice input device 33, the position information from the GNSS sensor 34, the direction indication information from the operation unit 25, and the received information via the communication unit 36, and executes corresponding processing. The control unit 30 controls the motors 22a and 23a (travel control of the travel unit 12), performs display control of the operation panel 31, reports to the occupant of the moving body 100 based on the voice of the speaker 32, and outputs information.

[0041] The voice input device 33 picks up the voice of the occupant of the moving body 100. The control unit 30 can recognize the input voice and execute corresponding processing. The GNSS (Global Navigation Satellite system) sensor 134 receives GNSS signals and detects the current position of the moving body 100. The storage device 35 is a storage device that stores the captured images, obstacle information, past generated paths, occupancy grid maps, etc. based on the detection units 15 to 17. Programs executed by the processor, data used by the processor for processing, etc. may also be stored in the storage device 35. The storage device 35 may also store various parameters of the machine learning models for voice recognition and image recognition executed by the control unit 30 (for example, learning completion parameters, hyperparameters, etc. of the deep neural network).

[0042] The communication unit 36 communicates with an external device, i.e., the communication device 120, via wireless communication such as Wi-Fi, 5G mobile communication, etc. The communication device 120 is, for example, a smartphone, but is not limited thereto, and may be a headset-type communication terminal, a personal computer, a tablet terminal, a game console, etc. The communication device 120 is connected to the network via wireless communication such as Wi-Fi, 5G mobile communication, etc.

[0043] The user having the communication device 120 can give an instruction to the moving body 100 via the communication device 120. This instruction includes, for example, an instruction to call the moving body 100 to a position desired by the user for rendezvous. If this instruction is received, the moving body 100 sets a target position based on the position information included in the instruction. In addition to the above instruction, the moving body 100 can also set a target position according to the captured images of the detection units 15 to 17, or set a target position based on an instruction from the user riding in the moving body 100 via the operation panel 31. When setting a target position according to the captured image, for example, a person who raises a hand toward the moving body 100 in the captured image is detected, and the position of the detected person is estimated and set as the target position.

[0044] <Functional Configuration of the Moving Body>

[0045] Next, with reference to Figure 3 , the functional configuration of the mobile body 100 according to the present embodiment will be described. In the control unit 30, for example, a program stored in a memory such as a ROM is read into the RAM by a CPU and executed to implement the functional configuration described herein. In addition, the functional configuration described below only describes the functions necessary for explaining the present invention, and does not describe all of the functional configurations actually included in the mobile body 100. That is, the functional configuration of the mobile body 100 according to the present invention is not limited to the functional configuration described below.

[0046] The user instruction acquisition unit 301 has a function of receiving an instruction from the user, and can receive a user instruction via the operation unit 25, the operation panel 31, a user instruction from an external device such as the communication device 120 via the communication unit 36, and an instruction based on the user's speech via the voice input device 33. As described above, the user instruction includes an instruction for setting the target position (also referred to as the destination) of the mobile body 100 and an instruction related to the travel control of the mobile body 100.

[0047] The image information processing unit 302 processes the captured images obtained by the detection units 15 to 17. Specifically, the image information processing unit 302 extracts the road shape recognized from the captured images obtained by the detection units 15 to 17. In addition, the image information processing unit 302 includes a machine learning model for processing image information, and can execute the processing in the learning stage and the processing in the inference stage of the machine learning model. The machine learning model of the image information processing unit 302 can perform operations using a deep learning algorithm of a deep neural network (DNN), for example, to perform processing for recognizing road shapes and the like included in the image information. The recognition information indicating the recognized road shape includes, for example, information indicating lines such as white lines, lanes, the shapes of intersections, the entrances and exits of intersections, and the like.

[0048] The trajectory generation unit 303 generates a travel path (trajectory) for the moving body 100 to the target position set by the user instruction acquisition unit 301. Specifically, the trajectory generation unit 303 generates a trajectory based on the road shape (recognition information) recognized from the captured images of the detection units 15 to 17 and the direction instruction information via the operation unit 25 without requiring the obstacle information of the high-precision map. In addition, the recognition information is the information of the road shape within a predetermined range from the moving body 100, and the road shape farther away cannot be recognized. On the other hand, the recognition information is information that is periodically updated as the moving body 100 advances. Therefore, the area in the distance is gradually recognized as the moving body 100 moves. The trajectory generation unit 303 sequentially generates a trajectory based on the updated recognition information. In addition, regarding the direction instruction information, it is not limited to the information received via the operation unit 25, and may also be information based on the information of the travel route change obtained from the path plan to the destination. Therefore, the present invention does not regard the operation unit 25 as an essential component, and can also be applied to a moving body that does not have the operation unit 25, etc.

[0049] In addition, the speed planning unit 304 plans the speed according to the curvature of the trajectory generated by the trajectory generation unit 303, and also plans the speed based on the driver's direction instruction. For example, when an instruction to turn left or right is given at an intersection or the like, in the speed planning, it is controlled that from going straight to starting to turn, the speed is reduced to 8 km in the case of turning right and reduced to 6 km in the case of turning left. By controlling the speed according to the trajectory generated in this way and the instruction from the driver, sharp deceleration and the like can be avoided.

[0050] The travel control unit 305 controls the travel of the moving body 100 according to the generated trajectory and speed plan. Specifically, the travel control unit 305 controls the travel unit 12 according to the trajectory and the speed plan to control the speed and angular velocity of the moving body 100. When there is an offset in the driving plan of the trajectory due to the driver's operation, the travel control unit 305 may obtain a new trajectory generated by the trajectory generation unit 303 again to control the travel, or may control the speed and angular velocity of the moving body 100 in a manner that eliminates the offset from the trajectory in use.

[0051] <Captured Image>

[0052] Figure 4A Shows the captured image related to the present embodiment. Figure 4B Shows an example of the road shape included in Figure 4A the captured image. Figure 4AThe captured image 400 shown represents an image captured by the detection unit 15 provided in front of the moving body 100. In addition, the shaded area 401 represents the cab of the moving body 100 captured in the captured image 400. The area outside the shaded area 401 is the area where the surrounding environment extending in the forward area of the moving body 100 is captured.

[0053] Figure 4B Represents the road shape included in Figure 4A the captured image 400 shown. The dotted area of the symbol 410 represents the intersection of a three-way intersection (T-shaped intersection). As Figure 4B shown, in the forward area of the moving body 100, there is a three-way intersection at the near front, and there is an exit straight ahead in the traveling direction as an exit of the intersection and an exit when turning right. Ahead of the exit when going straight at the intersection, there is a road that makes a large right turn. As shown in the captured image 400, in the perspective of the moving body 100 before entering the intersection 410, although it is possible to recognize the situation where there are multiple exits indicated by arrows, the road shape ahead of the exit cannot be recognized. Therefore, the moving body 100 according to the present embodiment generates a trajectory sequentially or stage by stage using a clear road shape as it moves. That is, as the moving body 100 approaches a predetermined road shape, the accuracy of its recognition increases, and a trajectory is generated according to the recognition situation. In addition, when the road shape recognized from the captured image 400 includes multiple forward projections, or when at least one forward projection is located outside a predetermined range from the current traveling direction, the moving body 100 according to the present embodiment determines that the road shape includes at least one forward projection accompanying a change in the traveling route. In the case of making the above determination, according to the present embodiment, it is determined that there is a possibility of a change in the traveling route, and the trajectory generation described later is executed.

[0054] <Trajectory Generation Method>

[0055] Figure 5 And Figure 6 represents the trajectory generation method according to the present embodiment. Here, the trajectory generation in the case where an instruction to turn right is received via the operation unit 25 when approaching the intersection (T-shaped intersection) shown in Figure 4B will be described. In addition, in the present embodiment, as an example of a road shape having an entrance part and a forward projection (exit) accompanying a change in the traveling route, a T-shaped road is used for description, but the present invention can also be a traveling area such as an intersection, a T-shaped road, an entrance to a facility along the road, an L-shaped shape, etc. as a road shape having an entrance part and a forward projection (exit) accompanying a change in the traveling route. The entrance to a facility along the road includes, for example, an entrance to a shopping center, a gas station, a parking lot, etc.

[0056] Symbol 501 indicates the track at the generated intersection. Here, as Figure 5 shown, the current position of the moving body 100 (or the intersection entrance) is xs, and the target intersection exit is xg. When receiving a right-turn instruction via the operation unit 25, in order to generate a track that crosses the intersection, the track generation unit 303 acquires the intersection point 504 between the center line 502 of the lane in which the moving body 100 is traveling and the center line 503 of the target lane. Here, the target lane line indicates the lane line that continues to the exit when the moving body 100 passes through the intersection.

[0057] Next, the track generation unit 303 acquires the distance d0 between the current position xs of the moving body 100 and the acquired intersection point 504, and the distance dg between the intersection point 504 and the intersection exit xg. Furthermore, the track generation unit 303 compares the acquired distance d0 with the distance dg, and determines to generate a track that continues with a single curve and then becomes a straight line when d0 < dg, and determines to generate a track that continues with a straight line and then becomes a single curve when d0 > dg. In addition, in the case where d0 = dg, it is determined to generate only a track of a single curve.

[0058] Next, the track generation unit 303 acquires the start point and end point of the single curve, and acquires the radius R based on the start point and end point. After that, the track generation unit 303 generates a track in a manner that includes a single curve and, if necessary, a straight line. Figure 5 The example shown indicates the form of generating a track 501 that continues with a straight line and then becomes a single curve in the case where d0 > dg. On the other hand, Figure 6 The example shown indicates the form of generating a track 601 that continues with a single curve and then becomes a straight line in the case where d0 < dg. Here, an example of a single curve + straight line or straight line + single curve has been described, but it is not intended to limit the present invention. For example, depending on the current position of the moving body 100 and the shape of the intersection, there may also be cases where only a single curve is generated, or a track of straight line + single curve + straight line is generated. In addition, here, for the sake of easy explanation, a single curve is taken as an example of the curve, and other curves such as a clothoid or a cubic curve may be generated instead of or in addition to the single curve.

[0059] <Track generation step according to the recognition situation>

[0060] Figure 7 Indicates the track generation step according to the recognition situation of the intersection according to the present embodiment. Here, the track generation step when approaching the Figure 4B shown intersection (T-junction) will be described. According to the present embodiment, the track is generated sequentially (stage by stage) based on the recognition information of the road shape obtained from the captured image when passing through road shapes such as intersections having an entrance part and a front-out part accompanied by a change in the traveling route.

[0061] Here, an example of trajectory control in four stages according to the distance between the moving body 100 and the intersection will be described. As Figure 7 shown, stage 0 is a state where the distance from the moving body 100 to the intersection is more than 30 m. In this state, the image information processing unit 302 is in the following state: identifying the "lane where the vehicle is located" representing the driving area during the driving of the moving body 100 based on the captured images obtained by the detection units 15 to 17, but not identifying the road shape of the intersection.

[0062] Stage 1 is a state where the distance from the moving body 100 to the intersection is less than 30 m and an instruction to turn right has been received from the operation unit 25. In this state, the image information processing unit 302 can identify the intersection based on the above-mentioned "lane where the vehicle is located". In addition, in the identification information of the intersection here, the shape of the intersection and the entry part ("entrance of the road") are identified, but the forward exit part (exit) and the state of the driving area in front of it are not clearly identified.

[0063] In the identification information of the road shape (such as "intersection" representing an intersection, etc.) extracted by the image information processing unit 302 using the machine learning model, various parameters are included according to the identification situation. In this parameter, for example, it includes "road entrance" representing the entry part and the forward exit part accompanying the change of the travel route, information representing the boundary of the "intersection", information representing the white lines (lines) of the nearby road shape, information representing the lane lines (driving area, lane), etc. That is to say, according to the identification situation, sometimes only the parameters representing the shape (boundary) of the "intersection" are included, and there is a possibility that the lanes and lines in front of the branch cannot be identified.

[0064] Therefore, the trajectory generation unit 303 needs to sequentially generate and update the trajectory according to the above-mentioned identification situation. In this stage 1, an instruction to turn right has been received, but the forward exit part (exit) of the intersection and the driving area in front of it have not been determined yet. Therefore, the trajectory generation unit 303 maintains the current trajectory as the trajectory and does not generate a trajectory for changing the travel route.

[0065] Phase 2 is a state where the distance from the moving body 100 to the intersection is less than 20 m and a right turn instruction has been received from the operation unit 25. In this state, based on the recognition information in the above Phase 1, the image information processing unit 302 recognizes the "Target lane" of the driving area ahead of it and extracts the leading part of this driving area. In addition, here, the driving area "Target lane" is recognized, but the line representing the boundary of this driving area is not recognized, and the accuracy of the recognition of this driving area is low. On the other hand, the leading part of the intersection at the time of the indicated right turn, the "entrance of the road", is in a state where it is clearly recognized. For example, this is because in Phase 2, approaching the intersection, the boundary of the right-turn side intersection can be clearly recognized, and its upper half can be presumed to be the driving lane line (driving area "Target lane"), and the lower half can be presumed to be the oncoming lane line.

[0066] Therefore, in Phase 2, the trajectory generation unit 303 generates a trajectory for changing the travel route based on the information that can be recognized with higher accuracy. Specifically, the trajectory generation unit 303 generates a first trajectory from the current position of the moving body 100 to the "entrance of the road", which is the entry part of the intersection that has been recognized, and a second trajectory from this entry part to the "entrance of the road", which is the leading part of the intersection on the right-turn side. Regarding the second trajectory, it is generated by using Figure 5 and Figure 6 the trajectory generation method described above, and includes trajectories with curves such as single curves, spiral curves, and cubic curves. The second trajectory can also be a trajectory including a straight line based on the above curves. In addition, the second trajectory can also be generated as a trajectory from the center of the entry part to the center of the determined leading part.

[0067] Phase 3 is a state where the moving body 100 enters the intersection. In this state, based on the recognition information in the above Phase 2, the image information processing unit 302 can further recognize the white line (line) representing the boundary of the "Target lane" of the driving area ahead of it, and can recognize the driving area "Target lane" more accurately than in Phase 2. Therefore, in Phase 3, the trajectory generation unit 303 generates a third trajectory that continues the second trajectory generated in the above Phase 2 and is within the recognized driving area. Thus, a trajectory for a right turn instruction at the intersection can be generated.

[0068] In addition, in the above-mentioned Stage 3, an example of generating a third trajectory was described in which, after the moving body 100 enters an intersection, when the white line of the driving area "target lane" is recognized (for example, the line "Lane instance" representing the boundary of the lane line), the driving area "target lane" is more accurately determined. However, the present invention is not limited thereto. For example, it may be that instead of determining the timing of the driving area "target lane" as described above, when the moving body 100 advances to a position in front of the entry part to the intersection, the trajectory generation unit 303 determines that the driving area "target lane" is determined to a certain extent and generates a third trajectory. In addition, the timing when the moving body 100 advances to a position in front of the entry part to the intersection means the timing of starting the driving of the second trajectory. Alternatively, it may be that when the moving body 100 approaches a predetermined distance from the predetermined front exit part on the right turn side, the trajectory generation unit 303 determines that the driving area "target lane" is determined to a certain extent and generates a third trajectory.

[0069] In addition, in the present invention, an example of determining a change in the travel route such as a right turn or a left turn based on information from the operation unit 25 has been described. However, the present invention is not limited thereto. For example, it may be that when making a path plan according to a preset destination, if a right turn is required at the next intersection, the trajectory generation unit 303 may determine a right turn and generate a trajectory without receiving the above-mentioned direction indication information. In addition, it may be that, for example, when the travel direction reaches the end and a left turn or a right turn must be made and no direction indication information from the operation unit 25 is received, it is determined to make a left turn or a right turn and a trajectory is generated. In this case, for example, for the path plan to the destination, a travel route change in the direction closer to the destination may be selected.

[0070] In addition, according to the present embodiment, as Figure 7 shown in the "Speed" row, the speed planning unit 304 plans the target speed in each stage. For example, when an instruction to turn right or left is received in a state where an intersection is recognized, a travel route change occurs, so it is necessary to decelerate to a predetermined speed (for example, 8 km / h in the case of a right turn and 6 km / h in the case of a left turn) until the entry part of the intersection. Therefore, in order to avoid sudden deceleration, the speed planning unit 304 preferably decelerates step by step in each stage.

[0071] <Basic Process>

[0072] Figure 8 This is a flowchart showing the basic control of the moving body 100 according to the present embodiment. In the control unit 30, for example, the program stored in a memory such as a ROM is read into the RAM by the CPU and executed to implement the processes described below.

[0073] In S101, the control unit 30 sets the target position of the moving body 100 based on the user instruction received by the user instruction acquisition unit 301. As described above, the user instruction can be received by various methods. Next, in S102, the control unit 30 acquires direction instruction information. Here, it includes the direction instruction information when the driver operates the operation unit 25 and the direction instruction information for changing the travel route determined according to the set target position. Here, as the process of acquiring the direction instruction information, it is simply described as being performed in S102, but actually it is a process that is generated at any time as an interrupt process at the timing of the driver operating the operation unit 25. Therefore, after the process of S102, the direction instruction information is also acquired through operation interruption for use in track generation.

[0074] Next, in S103, the control unit 30 captures an image of the front area (travel direction) of the moving body 100 through the detection unit 15 to obtain a captured image. Then, in S104, the control unit processes the obtained captured image through the image information processing unit 302 to obtain recognition information representing the road shape recognized using a machine learning model. In addition, the processes of S103 and S104 are executed continuously or periodically, and the captured image and the recognition information obtained from the captured image become information that is updated at any time.

[0075] Next, in S105, the control unit 30 generates an orbit of the moving body 100 based on the recognition information obtained in S104 through the orbit generation unit 303. The detailed steps of the orbit generation by the orbit generation unit 303 will be described later using Figure 9 Next, in S106, the control unit 30 generates a speed plan for the moving body 100 based on the generated orbit and the direction instruction information. Further, in S107, the control unit 30 determines the speed and angular velocity of the moving body 100 through the travel control unit 305 to control the travel. Then, in S108, the control unit 30 determines whether the moving body 100 has reached the target position based on the position information from the GNSS sensor 34. If the target position has not been reached, the process returns to S102, and while updating the captured image, an orbit is generated, and the process of controlling the travel is repeated. On the other hand, if the target position has been reached, the process of this flowchart ends.

[0076] <Steps of Orbit Generation>

[0077] Figure 9 It is a flowchart showing the detailed processing steps of the orbit generation (S105) according to this embodiment. In the control unit 30, for example, the process described below is realized by the CPU reading the program stored in a memory such as a ROM into the RAM and executing it.

[0078] First, in S201, the track generation unit 303 determines whether the identification information obtained in S104 includes information indicating an intersection or the like with a protruding portion accompanied by a travel route change. Specifically, when the road shape represented by the obtained identification information includes a plurality of protruding portions, or when at least one protruding portion is located outside a predetermined range from the current travel direction, the track generation unit 303 determines that the road shape includes at least one protruding portion accompanied by a travel route change. Further, here, it is set as "at least one" protruding portion because there is a possibility of newly recognizing other protruding portions as the moving body 100 approaches an intersection or the like. That is, there is a possibility that the number of recognized protruding portions increases according to the movement of the moving body 100, and the possibility of an increase in the protruding portion accompanied by a travel route change is considered. If it includes at least one protruding portion accompanied by a travel route change, it proceeds to S203, and if it does not include, it proceeds to S202. In S202, the track generation unit 303 generates a track within a predetermined range along the own vehicle lane during traveling, and ends the processing of this flowchart.

[0079] On the other hand, in S203, the track generation unit 303 determines whether direction indication information has been obtained in S102. If not obtained, it proceeds to S202, and if it has been obtained, it proceeds to S204. In S204, the track generation unit 303 determines whether it is possible to recognize the protruding portion determined by the obtained direction indication information among the plurality of protruding portions included in an intersection or the like. If not recognized, it proceeds to S202, and if it has been recognized, it proceeds to S205.

[0080] In S205, as control for generating a track for a travel route change, the track generation unit 303 first generates a first track from the current position of the moving body 100 to the entrance portion of an intersection or the like. The first track is basically in a straight line shape and becomes a track connecting the center of the own vehicle lane and the center of the entrance portion having a predetermined width. Next, in S206, the track generation unit 303 generates a second track within the intersection that continues the first track. The detailed content will be described later. Figure 10 as follows.

[0081] Next, in S207, the trajectory generation unit 303 determines whether a target driving area existing in front of the forward exit part determined as the movement destination has been recognized. As described above, whether the target driving area has been recognized is determined based on each parameter in the road shape recognized by the image information processing unit 302. Here, for example, when a white line whose shape is used to determine the target driving area is recognized, it is determined that the driving area has been recognized. In addition, instead of the above determination, it may also be determined that the driving area has been recognized at the timing of starting the driving of the second trajectory, at the timing when the distance from the moving body 100 to the determined forward exit part approaches a predetermined distance, and the like. When the target driving area is recognized, the process proceeds to S208, and the trajectory generation unit 303 generates a third trajectory starting from the determined forward exit part based on the recognized driving area, and returns the process to S104.

[0082] <Trajectory Generation at Intersections and the Like>

[0083] Figure 10 It is a flowchart showing the detailed processing steps of generating the second trajectory (S206) within the intersection according to the present embodiment. In the control unit 30, for example, the CPU reads a program stored in a memory such as a ROM into the RAM and executes it to implement the processing described below.

[0084] In S301, the trajectory generation unit 303 acquires the start point and the end point of the second trajectory. The start point of the second trajectory is, for example, the center of the entry part. In addition, the end point is, for example, the center of the determined forward exit part. Next, in S302, the trajectory generation unit 303 acquires the intersection point of the straight line along the traveling direction of the moving body 100 (for example, the center line of the lane of the own vehicle) and the straight line along the traveling direction of the movement destination (for example, the center line of the target lane). Next, in S303, the trajectory generation unit 303 acquires a first distance d0 from the start point to the intersection point and a second distance dg from the intersection point to the end point.

[0085] After that, in S304 and S305, the orbit generation unit 303 compares the first distance d0 with the second distance dg to determine whether to generate a straight line based on a single curve. Specifically, when the first distance d0 is equal to the second distance dg, it proceeds to S306, and the orbit generation unit 303 determines to generate a single curve as the path of the curve and proceeds to S309. Further, when the first distance d0 is shorter than the second distance dg, it proceeds to S307, and the orbit generation unit 303 determines that as the path of the curve, a single curve is generated from the above-mentioned starting point, and the generated single curve is continued to generate a straight line to the above-mentioned end point and proceeds to S309. Further, when the first distance d0 is longer than the second distance dg, it proceeds to S308, and the orbit generation unit 303 determines that as the path of the curve, a straight line is generated from the above-mentioned starting point, and the generated straight line is continued to generate a single curve to the above-mentioned end point and proceeds to S309. In S309, the orbit generation unit 303 obtains the radius R of the generated single curve. Next, in S310, the orbit generation unit 303 generates a path including a curve according to the determination in S306 to S308, and returns the process to S306.

[0086] <Summary of the Embodiment>

[0087] 1. The mobile body control system (for example, 100) of the above embodiment is characterized by including:

[0088] An imaging mechanism (for example, 15 to 17) that acquires a captured image of a traveling area based on the moving destination of the mobile body;

[0089] An identification mechanism (for example, 302) that identifies the road shape included in the captured image; and

[0090] An orbit generation mechanism (for example, 303) that generates an orbit of the mobile body based on the road shape identified by the identification mechanism.

[0091] When the identification mechanism identifies a road shape having at least one forward exit portion including an entrance portion and a traveling route change, the orbit generation mechanism generates a first orbit from the current position of the mobile body to the entrance portion, and a second orbit from the entrance portion to the forward exit portion determined according to the instruction information related to the moving destination of the mobile body (for example, 303, S205, S206, Figure 7 ).

[0092] According to this embodiment, it is possible to appropriately generate a traveling path even in a road structure having a plurality of exits such as intersections without using high-precision map information.

[0093] 2. In the mobile body control system of the above-described embodiment, the orbit generation mechanism generates an orbit from the center of the entry part to the center of the determined forward exit part as the second orbit (for example, Figure 5 , Figure 6 ).

[0094] According to this embodiment, it is possible to appropriately generate a travel path without using high-precision map information, even in a road structure such as an intersection that has a road structure with a changing travel route.

[0095] 3. In the mobile body control system of the above-described embodiment, while the mobile body is traveling according to the first orbit or the second orbit, when the recognition mechanism recognizes a travel area that continues the determined forward exit part, the orbit generation mechanism generates a third orbit that continues the second orbit, and the third orbit is located within the recognized travel area (for example, S208, Figure 7 ).

[0096] According to this embodiment, it is possible to appropriately generate a travel path without using high-precision map information, even in a road structure such as an intersection that has an exit with a changing travel route.

[0097] 4. In the mobile body control system of the above-described embodiment, when the mobile body starts traveling on the second orbit, the orbit generation mechanism generates a third orbit that continues the second orbit, and the third orbit is located within the travel area that continues the determined forward exit part and is recognized by the recognition mechanism (for example, S208, Figure 7 ).

[0098] According to this embodiment, it is possible to appropriately generate a travel path without using high-precision map information, even in a road structure such as an intersection that has an exit with a changing travel route.

[0099] 5. In the mobile body control system of the above-described embodiment, when the mobile body approaches to a predetermined distance from the determined forward exit part, the orbit generation mechanism generates a third orbit that continues the second orbit, and the third orbit is located within the travel area that continues the determined forward exit part and is recognized by the recognition mechanism (for example, S208, Figure 7 ).

[0100] According to this embodiment, it is possible to appropriately generate a travel path without using high-precision map information, even in a road structure such as an intersection that has an exit with a changing travel route.

[0101] 6. In the mobile body control system of the above-described embodiment, the road shape having the entry portion and the plurality of forward exit portions means any one of a driving area including an intersection, a T-junction, and an entrance to a facility along the road.

[0102] According to this embodiment, in the case of a road shape having an entry portion and a forward exit portion accompanied by a change in the travel route, a travel path can be appropriately generated without using high-precision map information.

[0103] 7. The mobile body control system of the above-described embodiment further includes a direction indication mechanism that receives the indication information (e.g., 25) related to the moving destination of the mobile body.

[0104] According to this embodiment, a travel path that conforms to the driver's intention can be generated.

[0105] 8. In the mobile body control system of the above-described embodiment, in the case where the indication information is not received by the direction indication mechanism and a travel route change is required, the trajectory generation mechanism generates the second trajectory up to the forward exit portion determined corresponding to the travel route change.

[0106] According to this embodiment, in the case of a travel route change according to a path plan or when traveling on a road with a structure that requires a travel route change, a travel path can be appropriately generated without using high-precision map information.

[0107] 9. In the mobile body control system of the above-described embodiment, in the case of a travel route change, the trajectory generation mechanism generates the second trajectory including at least a single curve (e.g., Figure 5 , Figure 6 , Figure 10 ).

[0108] According to this embodiment, even in the case of a travel route change at an intersection or the like, a smooth trajectory can be easily generated.

[0109] 10. In the mobile body control system of the above-described embodiment, the mobile body control system further includes:

[0110] a speed planning mechanism (e.g., 304) that generates a speed plan for the mobile body based on the trajectory generated by the trajectory generation mechanism and the indication information related to the moving destination of the mobile body; and

[0111] a travel control mechanism (e.g., 305) that controls the travel of the mobile body based on the trajectory generated by the trajectory generation mechanism and the speed plan generated by the speed planning mechanism.

[0112] According to this embodiment, even when a travel route change occurs at an intersection or the like, a smooth trajectory can be easily generated, and sharp deceleration or the like can be avoided.

[0113] 11. In the mobile body control system according to the above embodiment, when the road shape recognized by the recognition mechanism includes a plurality of protruding portions, or when at least one protruding portion is located outside a predetermined range from the current travel direction, the trajectory generation mechanism determines that the road shape recognized by the recognition mechanism includes at least one protruding portion accompanying a travel route change (for example, 303, S205, S206, Figure 7 ).

[0114] According to this embodiment, intersections or the like accompanying a travel route change can be appropriately determined.

[0115] The invention is not limited to the above embodiments, and various modifications and changes can be made within the scope of the gist of the invention.

Claims

1. A mobile body control system, characterized in that, the mobile body control system includes: a photographing mechanism that acquires a photographed image of a traveling area based on a moving destination of the mobile body; a recognition mechanism that recognizes a road shape included in the photographed image; and an orbit generation mechanism that generates an orbit of the mobile body based on the road shape recognized by the recognition mechanism, when the recognition mechanism recognizes a road shape having at least one forward exit portion including an entrance portion and accompanied by a change in the traveling route, the orbit generation mechanism generates a first orbit from the current position of the mobile body to the entrance portion, and a second orbit from the entrance portion to a forward exit portion determined according to indication information related to the moving destination of the mobile body.

2. The mobile body control system according to claim 1, characterized in that, the orbit generation mechanism generates an orbit from the center of the entrance portion to the center of the determined forward exit portion as the second orbit.

3. The mobile body control system according to claim 2, characterized in that, during the process in which the mobile body is traveling according to the first orbit or the second orbit, when the recognition mechanism recognizes a traveling area that continues the determined forward exit portion, the orbit generation mechanism generates a third orbit that continues the second orbit, and the third orbit is located within the recognized traveling area.

4. The mobile body control system according to claim 2, characterized in that, when the mobile body starts traveling on the second orbit, the orbit generation mechanism generates a third orbit that continues the second orbit, and the third orbit is located within a traveling area that continues the determined forward exit portion recognized by the recognition mechanism.

5. The mobile body control system according to claim 2, characterized in that, when the mobile body approaches a predetermined distance from the determined forward exit portion, the orbit generation mechanism generates a third orbit that continues the second orbit, and the third orbit is located within a traveling area that continues the determined forward exit portion recognized by the recognition mechanism.

6. The mobile body control system according to claim 1, characterized in that, the road shape having the entrance portion and at least one forward exit portion accompanied by the change in the traveling route means any one of traveling areas including an intersection, a T-shaped road, and an entrance to a facility along the road.

7. The mobile body control system according to claim 1, characterized in that, the mobile body control system further includes a direction indication mechanism that receives the indication information related to the moving destination of the mobile body.

8. The mobile body control system according to claim 7, characterized in that, in a case where the indication information is not received by the direction indication mechanism and a change in the traveling route is required, the orbit generation mechanism generates the second orbit to a forward exit portion determined corresponding to the change in the traveling route.

9. The mobile body control system according to claim 2, characterized in that, In the case where a travel route change occurs, the track generation mechanism generates a track including at least a single curve as the second track.

10. The mobile body control system according to claim 1, wherein, the mobile body control system further includes: a speed planning mechanism that generates a speed plan for the mobile body based on the track generated by the track generation mechanism and the instruction information related to the movement destination of the mobile body; and a travel control mechanism that controls the travel of the mobile body based on the track generated by the track generation mechanism and the speed plan generated by the speed planning mechanism.

11. The mobile body control system according to claim 1, wherein, in the case where the road shape recognized by the recognition mechanism includes a plurality of protruding portions, or at least one protruding portion is located outside a predetermined range from the current travel direction, the track generation mechanism determines that the road shape recognized by the recognition mechanism includes at least one protruding portion accompanying a travel route change.

12. A control method for a mobile body control system, wherein, the control method for the mobile body control system includes: a photographing step in which a photographed image of a travel area based on the movement destination of the mobile body is acquired; a recognition step in which the road shape included in the photographed image is recognized; and a track generation step in which a track for the mobile body is generated based on the road shape recognized in the recognition step, in the track generation step, when a road shape having an entrance portion and at least one protruding portion accompanying a travel route change is recognized in the recognition step, a first track from the current position of the mobile body to the entrance portion and a second track from the entrance portion to the protruding portion determined according to the instruction information related to the movement destination of the mobile body are generated.

13. A program, wherein, the program causes a computer to function as each mechanism of the mobile body control system according to any one of claims 1 to 11.

14. A mobile body, wherein, the mobile body includes: a photographing mechanism that acquires a photographed image of a travel area of a movement destination; a recognition mechanism that recognizes the road shape included in the photographed image; and a track generation mechanism that generates a track for the mobile body based on the road shape recognized by the recognition mechanism, when a road shape having an entrance portion and at least one protruding portion accompanying a travel route change is recognized by the recognition mechanism, the track generation mechanism generates a first track from the current position of the mobile body to the entrance portion and a second track from the entrance portion to the protruding portion determined according to the instruction information related to the movement destination of the mobile body.

Images