Method and apparatus for generating virtual stop lines

Abstract

To provide a method and a device for generating a virtual stop-line according to the present disclosure.SOLUTION: A method or generating a virtual stop-line comprises: a step of determining a target cross-walk from one or more cross-walks on a travel route; a step 2710 of setting a region of interest related to the target cross-walk; and a step 2730 of determining whether or not a virtual stop-line is generated based on a position of a pedestrian within the region of interest.SELECTED DRAWING: Figure 27

Description

[Technical field]

[0001] The present disclosure provides a method and apparatus for generating a virtual stop line. [Background technology]

[0002] The fusion of information and communication technology and the automotive industry is rapidly making vehicles smarter. As vehicles become smarter, they are evolving from simple mechanical devices to smart cars, and autonomous driving in particular is attracting attention as a core technology for smart cars. Autonomous driving is a technology that allows a vehicle to reach its destination by itself without the driver having to operate the steering wheel, accelerator pedal, brakes, etc.

[0003] As various additional functions related to autonomous driving continue to be developed, there is a continuing demand for research into methods for providing a safe autonomous driving experience for both passengers and pedestrians by using various data to recognize and assess the driving environment and control the vehicle, as well as methods for controlling and slowing down the vehicle when an autonomous vehicle recognizes a pedestrian.

[0004] The above-mentioned background art is technical information possessed by the inventor for the purpose of deriving the present invention or acquired in the process of deriving the present invention, and is not necessarily publicly known art that was disclosed to the general public prior to the filing of the present application. Summary of the Invention [Problem to be solved by the invention]

[0005] An object of the present disclosure is to provide a method and device for generating a virtual stop line. The problems to be solved by the present disclosure are not limited to the problems described above, and other problems and advantages of the present disclosure that are not described will be understood from the following description and will be more clearly understood from the embodiments of the present disclosure. It will also be understood that the problems and advantages to be solved by the present disclosure can be realized by the means and combinations thereof described in the claims. [Means for solving the problem]

[0006] A first aspect of the present disclosure may provide a method for generating a virtual stop line, the method including steps of determining a target crosswalk from among one or more crosswalks on a travel route, setting a region of interest for the target crosswalk, and determining whether to generate a virtual stop line based on a position of a pedestrian within the region of interest.

[0007] A second aspect of the present disclosure can provide a virtual stop line generating device that includes a memory having at least one program stored therein and a processor that operates by executing the at least one program, where the processor determines a target crosswalk from among one or more crosswalks on a travel route, sets a region of interest for the target crosswalk, and determines whether to generate a virtual stop line based on a position of a pedestrian within the region of interest.

[0008] A third aspect of the present disclosure can provide a method for determining a driving speed, including: (a) recognizing a pedestrian and acquiring recognition data and state data; (b) updating a pedestrian intention vector by estimating a probability distribution of multiple intentions that the pedestrian may have based on at least one of the recognition data and the state data; (c) estimating the pedestrian's intention at a next time point based on the updated pedestrian intention vector; and (d) determining the driving speed of the vehicle by taking into account the pedestrian's intention at the next time point.

[0009] A fourth aspect of the present disclosure can provide a driving speed determination device that includes a memory storing at least one program and a processor operating by executing the at least one program, wherein the processor recognizes a pedestrian and acquires recognition data and state data, estimates a probability distribution of multiple intentions that the pedestrian may have based on at least one of the recognition data and the state data, updates a pedestrian intention vector, estimates the pedestrian's intention at a next time point based on the updated pedestrian intention vector, and determines the driving speed of the vehicle by taking into account the pedestrian's intention at the next time point.

[0010] A fifth aspect of the present disclosure can provide a method for planning a driving speed, including the steps of acquiring an area of ​​interest ahead of a vehicle's driving path, detecting nearby objects around the vehicle and setting a relevance of the nearby objects to the vehicle based on the area of ​​interest, determining a reference position profile based on the relevance, and determining a final position profile corresponding to the reference position profile.

[0011] A sixth aspect of the present disclosure can provide a device for planning a driving speed, including a memory having at least one program stored therein, and a processor operating by executing the at least one program, wherein the processor acquires an area of ​​interest ahead of a vehicle's driving path, detects nearby objects around the vehicle, sets a relevance of the nearby objects to the vehicle based on the area of ​​interest, determines a reference position profile based on the relevance, and determines a final position profile corresponding to the reference position profile.

[0012] A seventh aspect of the present disclosure may provide a method for optimizing a vehicle trajectory, the method including: (a) acquiring a reference trajectory; (b) calculating a plurality of predicted trajectories based on one or more control inputs; (c) calculating a cost function based on the reference trajectory, the plurality of predicted trajectories, and at least one of the one or more control inputs; and (d) selecting an optimal control input from the one or more control inputs that minimizes the cost function.

[0013] An eighth aspect of the present disclosure can provide an apparatus for optimizing a vehicle trajectory, including a memory having at least one program stored therein, and a processor operating by executing the at least one program, wherein the processor acquires a reference trajectory, calculates a plurality of predicted trajectories based on one or more control inputs, calculates a cost function based on the reference trajectory, the plurality of predicted trajectories, and at least one of the one or more control inputs, and selects an optimal control input from the one or more control inputs that minimizes the cost function.

[0014] A ninth aspect of the present disclosure may provide a method for optimizing a vehicle trajectory, the method including: (a) obtaining a reference trajectory; (b) determining a plurality of model prediction time points including a current time point using a prediction reference horizon and a control period; (c) generating one or more control inputs at each of the plurality of model prediction time points and calculating a plurality of predicted trajectories based on the one or more control inputs; and (d) selecting an optimal control input corresponding to an optimal predicted trajectory from the one or more control inputs.

[0015] A tenth aspect of the present disclosure can provide an apparatus for optimizing a vehicle trajectory, the apparatus including: a memory having at least one program stored therein; and a processor operating by executing the at least one program, wherein the processor acquires a reference trajectory; determines a plurality of model prediction time points including a current time point using a prediction reference horizon and a control period; generates one or more control inputs at each of the plurality of model prediction time points; calculates a plurality of predicted trajectories based on the one or more control inputs; and selects an optimal control input corresponding to an optimal predicted trajectory from the one or more control inputs.

[0016] It is possible to provide a computer-readable recording medium having recorded thereon a program for executing the method of the first, third, fifth, seventh or ninth aspect on a computer.

[0017] In addition, other methods and devices for implementing the present invention, and a computer-readable recording medium having a program for executing the method recorded thereon can also be provided.

[0018] Further aspects, features, and advantages will become apparent from the accompanying drawings, the claims, and the following detailed description of the invention. Effect of the Invention

[0019] According to the problem-solving means of the present disclosure described above, even if a pedestrian suddenly enters the crosswalk, the vehicle can stop without significantly slowing down, providing a driving experience that takes ride comfort into consideration.

[0020] In addition, by determining whether or not to generate a virtual stop line depending on the absolute position of a pedestrian and the relative position between the pedestrian and the vehicle, it is possible to deal with various environments on the road. [Brief description of the drawings]

[0021] [Figure 1] FIG. 2 is a diagram for explaining an autonomous driving method according to an embodiment. [Diagram 2] FIG. 2 is a diagram for explaining an autonomous driving method according to an embodiment. [Diagram 3] FIG. 2 is a diagram for explaining an autonomous driving method according to an embodiment. [Figure 4] 1 is a schematic diagram illustrating an example of an environment around a travel path of a vehicle according to an embodiment; [Diagram 5] 1 is a schematic diagram illustrating an example of an environment around a travel path of a vehicle according to an embodiment; [Figure 6] 11 is a diagram illustrating a process of determining a target crosswalk according to an embodiment. FIG. [Figure 7] 11A and 11B are diagrams illustrating a process of setting a region of interest according to an embodiment; [Figure 8] FIG. 1 is a diagram showing a Cartesian frame. [Figure 9] FIG. 1 is a diagram showing a Frenet frame. [Figure 10] 1 is a flowchart of a method for determining whether to generate a virtual stop line according to one embodiment. [Figure 11] FIG. 13 is an illustrative diagram showing a realization when the vehicle is at a long distance from the target crosswalk. [Figure 12] FIG. 13 is an illustrative diagram showing a realization when the vehicle is in close range to the target crosswalk. [Figure 13] FIG. 13 is an illustrative diagram showing a realization when the vehicle is in close range to the target crosswalk. [Figure 14] FIG. 1 is a conceptual diagram of a method for determining driving speed utilizing pedestrian intention estimation according to one embodiment. [Figure 15] FIG. 2 is a diagram for explaining first to third intentions of a pedestrian according to one embodiment. [Figure 16] 11A and 11B are diagrams for explaining a method for planning a vehicle travel speed based on behavior prediction information of a nearest object according to an embodiment. [Figure 17] 11A and 11B are diagrams illustrating a process of deriving a corrected position profile based on the relevance of a nearby object according to an embodiment. [Figure 18] 11A and 11B are diagrams illustrating a process of deriving a corrected position profile based on the relevance of a nearby object according to an embodiment. [Figure 19] 1 illustrates a process for determining a reference position profile in an environment with multiple nearby objects according to one embodiment. [Figure 20] 1 illustrates a process for determining a reference position profile in an environment with multiple nearby objects according to one embodiment. [Figure 21] FIG. 13 is an example diagram illustrating a final position profile according to one embodiment. [Figure 22] 1 is a schematic diagram illustrating an example of a process for optimizing a vehicle trajectory according to an embodiment; [Diagram 23] 5 is a diagram for explaining a process in which a driving planning device according to an embodiment calculates a predicted trajectory. FIG. [Figure 24] FIG. 2 is a diagram for explaining a prediction model for selecting an optimal control input by the driving planning device according to one embodiment. [Diagram 25] FIG. 2 is a diagram for explaining a prediction model for selecting an optimal control input by the driving planning device according to one embodiment. [Figure 26] 11A to 11C are diagrams illustrating a process in which a driving planning device according to an embodiment sets control periods non-uniformly. [Figure 27] 1 is a flowchart of a method for generating a virtual stop line according to one embodiment. [Figure 28] FIG. 2 is a block diagram of an apparatus for generating a virtual stop line according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] The advantages and features of the present invention, as well as the methods for achieving them, will become apparent from the embodiments described in detail with the accompanying drawings. However, the present invention is not limited to the embodiments presented below, and can be realized in various different forms, and should be understood to include all modifications, equivalents, and alternatives within the spirit and technical scope of the present invention. The embodiments presented below are provided to complete the disclosure of the present invention and to allow those skilled in the art to fully understand the scope of the invention. In describing the present invention, if a detailed description of related publicly known technology is deemed to obscure the gist of the present invention, the detailed description will be omitted.

[0023] The terms used in this application are merely used to describe certain embodiments and are not intended to limit the present invention. A singular expression includes a plural expression unless the context clearly indicates otherwise. In this application, the terms "include" and "have" are used to specify the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, and are not to be understood as precluding the presence or additional possibility of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0024] Some embodiments of the present disclosure may be illustrated in terms of functional block configurations and various processing steps. Some or all of such functional blocks may be implemented in any number of hardware and / or software configurations for performing specific functions. For example, the functional blocks of the present disclosure may be implemented by one or more microprocessors or by circuit configurations for a given function. Also, for example, the functional blocks of the present disclosure may be implemented in various programming or scripting languages. The functional blocks may be implemented by algorithms executed by one or more processors. Furthermore, the present disclosure may employ conventional techniques for electronic configuration, signal processing, and / or data processing, etc. Terms such as "mechanism," "element," "means," and "configuration" may be used broadly and are not limited to mechanical and physical configurations.

[0025] It should be noted that the connecting lines or connecting members between components shown in the drawings are merely exemplary of functional and / or physical connections or circuit connections, and that in an actual device, connections between components may be represented by various alternative or additional functional connections, physical connections or circuit connections.

[0026] Hereinafter, the term "vehicle" may refer to any type of transportation means having a locomotive and used to move people or goods, such as a car, a bus, a motorcycle, a scooter, or a truck.

[0027] The present disclosure will now be described in detail with reference to the accompanying drawings.

[0028] Referring to FIG. 1, an autonomous driving device according to an embodiment of the present invention can be mounted on a vehicle to realize an autonomous driving vehicle 10. The autonomous driving device mounted on the autonomous driving vehicle 10 may include various sensors (including a camera) for collecting surrounding situation information. As an example, the autonomous driving device may detect the movement of a leading vehicle 20 traveling ahead using an image sensor and / or an event sensor mounted on the front of the autonomous driving vehicle 10. The autonomous driving device may further include sensors for detecting not only the vehicle ahead of the autonomous driving vehicle 10, but also other vehicles 30 traveling on adjacent roads, pedestrians around the autonomous driving vehicle 10, and the like.

[0029] At least one of the sensors for collecting situational information around the autonomous vehicle 10 can have a predetermined angle of view (FoV) as shown in Fig. 1. As an example, when a sensor mounted on the front of the autonomous vehicle 10 has an angle of view (FoV) as shown in Fig. 1, information detected at the center of the sensor can have a relatively high importance. This is because most of the information corresponding to the movement of the leading vehicle 20 is included in the information detected at the center of the sensor.

[0030] The autonomous driving device processes information collected by sensors of the autonomous vehicle 10 in real time to control the movement of the autonomous vehicle 10, while at least a portion of the information collected by the sensors can be stored in a memory device.

[0031] 2, the autonomous driving device 40 may include a sensor unit 41, a processor 46, a memory system 47, a vehicle control module 48, etc. The sensor unit 41 includes a plurality of sensors (including a camera) 42-45, and the plurality of sensors 42-45 may include an image sensor, an event sensor, an illuminance sensor, a GPS device, an acceleration sensor, etc.

[0032] The data collected by the sensors 42 to 45 may be transmitted to the processor 46. The processor 46 can store the data collected by the sensors 42 to 45 in the memory system 47 and control the body control module 48 to determine the movement of the vehicle based on the data collected by the sensors 42 to 45. The memory system 47 may include two or more memory devices and a system controller for controlling the memory devices. Each of the memory devices may be provided as a single semiconductor chip.

[0033] In addition to the system controller of memory system 47, each of the memory devices included in memory system 47 may include a memory controller, and the memory controller may include an artificial intelligence (AI) calculation circuit such as a neural network. The memory controller can generate calculation data by applying a predetermined weight to data received from sensors 42 to 45 or processor 46, and store the calculation data in a memory chip.

[0034] Fig. 3 is a diagram showing an example of video data acquired by a sensor (including a camera) of an autonomous vehicle equipped with an autonomous driving device. Referring to Fig. 3, video data 50 may be data acquired by a sensor attached to the front of the autonomous vehicle. Thus, video data 50 may include a front portion 51 of the autonomous vehicle, a preceding vehicle 52 on the same road as the autonomous vehicle, vehicles 53 traveling around the autonomous vehicle, and a background 54.

[0035] 3, data of the area in which the front part 51 and background 54 of the autonomous vehicle are displayed may be data that is unlikely to affect the operation of the autonomous vehicle. In other words, the front part 51 and background 54 of the autonomous vehicle may be considered to be data having a relatively low importance.

[0036] In contrast, the distance to the preceding vehicle 52, lane-changing movements of the moving vehicle 53, and the like can be very important factors in the safe operation of the autonomous vehicle. Therefore, in the video data 50, data of an area including the preceding vehicle 52 and the moving vehicle 53 can have a relatively high degree of importance in the operation of the autonomous vehicle.

[0037] The memory device of the autonomous driving device can store the image data 50 received from the sensor by assigning different weights to each area. For example, a high weight can be assigned to data of an area including a leading vehicle 52 and a moving vehicle 53, and a low weight can be assigned to data of an area including a front part 51 of the autonomous driving vehicle and a background 54.

[0038] Hereinafter, operations according to various embodiments may be understood to be performed by an autonomous driving device or a processor included in the autonomous driving device.

[0039] The roadway along which the autonomous vehicle travels may include various markings. For example, the roadway may include markings such as lanes, crosswalks, stop lines, etc. Among the various markings, a crosswalk is a path provided on the roadway for pedestrians to cross the roadway.

[0040] An autonomous vehicle is basically controlled to stop at a stop line when the road signal in the vehicle's travel direction is red, and to run when the road signal is green. If a pedestrian is recognized near a pedestrian crossing while traveling, the autonomous vehicle is controlled to decelerate or stop regardless of the road signal. However, if a predetermined area corresponding to the vicinity of the pedestrian crossing is not appropriately set, such a control method may cause a sudden deceleration in a situation where deceleration is not necessary, causing a dangerous situation behind the autonomous vehicle, or may not decelerate sufficiently in a situation where deceleration is necessary, causing a dangerous situation in front and to the sides of the autonomous vehicle. Furthermore, if the pedestrian recognition information is inaccurate, deceleration and acceleration may not occur consistently, which may deteriorate the ride comfort. In view of this, the present disclosure can provide a method for generating a virtual stop line that takes into account both the safety and ride comfort of the vehicle and pedestrians when the autonomous vehicle passes through a pedestrian crossing, and a method for controlling the vehicle thereby.

[0041] Hereinafter, a virtual stop line generating device (hereinafter referred to as a "driving planning device") according to one embodiment of the present invention may refer to an autonomous vehicle, and may be a device that is incorporated inside the vehicle or provided separately, as described below.

[0042] 4 and 5 are schematic diagrams illustrating an example of an environment around a travel route of a vehicle according to an embodiment.

[0043] 4, a crosswalk and one or more pedestrians 410, 420, 430, and 440 may exist around the driving path of a vehicle 400. In order for the vehicle 400 to drive safely, the behavior of the pedestrians 410, 420, 430, and 440 must be taken into consideration.

[0044] 5, various objects may exist ahead of a driving path 510 of a vehicle 500, including a crosswalk and a pedestrian 550 as well as vehicles 520, 530, and 540. In order for the vehicle 500 to travel safely, it is necessary to consider not only the behavior of the pedestrian 550 but also the driving conditions of the other vehicles 520, 530, and 540.

[0045] Therefore, the driving planner according to the present invention can set the range to be considered in the driving of the vehicle 500 as the region of interest 560, and determine the driving speed of the vehicle 500 based on the region of interest 560. However, the driving planner can determine the driving speed by considering even the object 550 not included in the region of interest 560, and the range that the driving planner considers to determine the driving speed or control the vehicle 500 is not limited thereto.

[0046] In one embodiment, the driving planner can obtain an area of ​​interest 560, which is a predetermined area ahead of the driving route 510. The driving planner can determine the driving speed of the vehicle 500 by considering only the objects 520, 530, and 540 included in the area of ​​interest 560. Although FIG. 5 shows that the area of ​​interest 560 includes only vehicles, the area of ​​interest 560 may also include pedestrians. In other words, the objects 520, 530, and 540 included in the area of ​​interest 560 may include not only vehicles but also pedestrians. A method for setting the area of ​​interest 560 will be described later with reference to FIG. 7.

[0047] FIG. 6 is a diagram illustrating a process of determining a target crosswalk according to an embodiment.

[0048] 6, the trip planner can recognize one or more crosswalks 610, 620 on a trip route. For example, the trip planner can recognize one or more crosswalks 610, 620 based on an image and / or a vision sensor. Alternatively, the trip planner can receive traffic information and recognize one or more crosswalks 610, 620 based on the traffic information and the position of the vehicle.

[0049] The trip planner can determine a target crosswalk from among one or more of the crosswalks 610, 620.

[0050] The target crosswalk may refer to a crosswalk that the trip planning device should consider with the highest priority in generating a virtual stop line, for example, the target crosswalk may refer to the closest crosswalk on the travel path ahead of the vehicle.

[0051] In one embodiment, the trip planning device can determine which crosswalk the vehicle is passing based on the current position of the vehicle and determine the target crosswalk. Thus, the trip planning device can determine the crosswalk 420 that the vehicle will pass first or that is closest ahead on the vehicle's travel path as the target crosswalk.

[0052] FIG. 7 is a diagram illustrating a process of setting a region of interest according to an embodiment.

[0053] 7, the environment in which the vehicle travels may include a travel route 700, a target crosswalk 720, etc. For example, the travel route 700 may be a route that connects the midpoints of the width of the road on which the vehicle is currently traveling. Alternatively, the travel route 700 may be a route that connects the midpoints of the start node and the end node of the travel route node acquired when the traffic information is generated.

[0054] In one embodiment, the driving planner can acquire a distance 710 to the target crosswalk 720. As one example, the distance 710 to the target crosswalk 720 may correspond to a distance on the driving route 700 from a current position of the vehicle to the target crosswalk 720 or a stop line (not shown) corresponding to the target crosswalk 720. If the driving route 700 is a curve, the distance on the driving route 700 may differ from an absolute distance (straight-line distance). As another example, the distance 710 to the target crosswalk 720 may mean the absolute distance (straight-line distance).

[0055] As yet another example, the distance 710 to the target crosswalk 720 may correspond to the distance from a position where a camera or sensor is mounted in the vehicle to the target crosswalk 720. Alternatively, the distance 710 to the target crosswalk 720 may correspond to the distance from the front end of the vehicle or the rear end of the vehicle to the target crosswalk 720.

[0056] In one embodiment, the drive planner can obtain a width 721 of the target crosswalk 720 and a crossing length 722 of the target crosswalk 720. The width 721 may correspond to the width of the target crosswalk 720, and the crossing length 722 may correspond to the length of a pedestrian passing through the target crosswalk 720 to cross a roadway.

[0057] 8 and 9 are diagrams showing a Cartesian frame and a Frenet frame.

[0058] 8, the Cartesian frame refers to a coordinate system that indicates a coordinate plane or coordinate space using mutually perpendicular axes. When the road driving environment is configured in the Cartesian frame, the x-axis and y-axis can be formed regardless of the shape of the road.

[0059] Referring to FIG. 9, the Frenet frame means a coordinate system defined by a unit tangent vector, a unit normal vector, and a binormal vector, and can depict the movement of a curve having a curvature. When the road driving environment is configured with the Frenet frame, an s-axis can be formed in the direction of the road shape, and a d-axis can be formed perpendicular to the road. That is, the s-coordinate can indicate the driving length, and the d-coordinate can indicate how far the vehicle is laterally away from the driving path. In FIG. 9, the start point of the driving path is set to s=0, d=0, but this can be changed depending on the frame setting.

[0060] The Frenet frame is intuitive and convenient because it allows the travel path to be mathematically represented more simply, since the travel length is described in s-axis coordinates, and is useful when it is complex or difficult to assign a Cartesian coordinate system to the trajectory.

[0061] 7, the driving planner can obtain the width 721 and crossing length 722 of the target crosswalk 720 using Frenet frame-based parameters. For example, the driving planner can obtain the target crosswalk 720 expressed based on the Frenet frame, and obtain the width 721 and crossing length 722 using the parameters.

[0062] The driving planner can set a region of interest 730 for the target crosswalk 720. The region of interest 730 can mean a predetermined region near the target crosswalk 720 that should be considered when the vehicle generates a virtual stop line. For example, the driving planner may generate a virtual stop line by considering only pedestrians within the region of interest 730, and not consider pedestrians outside the region of interest 730.

[0063] In one embodiment, the trip planner can use the target crosswalk 720 to set margins 741 , 742 , 751 , 752 of the area of ​​interest 730 and use them to set the area of ​​interest 730 .

[0064] The margins 741, 742, 751, 752 of the area of ​​interest 730 are elements for determining the size and position of the area of ​​interest 730, and the larger the margin is set, the wider the area of ​​interest 730, thereby improving safety, and the smaller the margin is set, the narrower the area of ​​interest 730, thereby reducing the amount of calculations performed by the device and enabling efficient driving. In this disclosure, the driving planning device presents a method for achieving safety, ride comfort, and efficiency by setting the area of ​​interest 730 based on the distance 710 to the target crosswalk 720.

[0065] In one embodiment, the drive planning device can set the margins 741, 742, 751, 752 of the area of ​​interest 730 larger the farther the distance 710 to the target crosswalk 720, and set the margins 741, 742, 751, 752 of the area of ​​interest 730 smaller the closer the distance 710 to the target crosswalk 720.

[0066] The recognition of pedestrians and their positions by the driving planner is relatively less accurate the farther the vehicle's current position is from the target crosswalk 720, and relatively more accurate the closer the vehicle is to the target crosswalk 720. Therefore, by setting margins 741, 742, 751, and 752 of the area of ​​interest 730 to be different depending on the distance 710 from the target crosswalk 720, it is possible to prevent a decrease in ride comfort due to sudden deceleration by generating virtual stop lines in advance at long distances and to ensure safety by taking pedestrian position errors into consideration, and by reducing the margins 741, 742, 751, and 752 of the area of ​​interest 730 at short distances to prevent deadlocks that may occur when a pedestrian is standing at the road boundary, it is possible to ensure safety and driving efficiency.

[0067] In another embodiment, the trip planning device can set the margins 741, 742, 751, 752 of the area of ​​interest 730 larger as the vehicle speed is faster, and set the margins 741, 742, 751, 752 of the area of ​​interest 730 smaller as the vehicle speed is slower. The faster the vehicle speed is, the more inaccurate the trip planning device is in recognizing pedestrians in the area of ​​interest 730, and if the margins 741, 742, 751, 752 of the area of ​​interest 730 are not large, it is difficult to decelerate or stop without suddenly decelerating. Therefore, the faster the vehicle speed is, the larger the margins 741, 742, 751, 752 of the area of ​​interest 730 are set, thereby improving safety and ride comfort.

[0068] In one embodiment, the margins 741, 742, 751, 752 of the region of interest 730 may include vertical margins 741, 742 and horizontal margins 751, 752. In this embodiment, the vertical direction may mean a direction parallel to the travel path 700, and the horizontal direction may mean a direction perpendicular to the travel path 700. Thus, the vertical margins 741, 742 mean margins from both ends of the target crosswalk 720 in the vertical direction to both ends of the region of interest 730 in the vertical direction, and the horizontal margins 751, 752 mean margins from both ends of the target crosswalk 720 in the horizontal direction to both ends of the region of interest 730 in the horizontal direction.

[0069] In one embodiment, the drive planner can set the vertical margins 741, 742 based on the width 721 of the target crosswalk 720 and the distance 710 from the target crosswalk 720. For example, the drive planner can set the vertical margins 741, 742 in proportion to the distance 710 from the target crosswalk 720. Alternatively, the drive planner can set the vertical margins 741, 742 in proportion to the width 721 of the target crosswalk 720.

[0070] At this time, the driving planning device can set an approach vertical direction margin 741 to the target crosswalk 720 and an exit vertical direction margin 742 from the target crosswalk 720 so that they are different.

[0071] Similarly, in one embodiment, the trip planner can set the lateral margins 751, 752 based on the crossing length 722 of the target crosswalk 720 and the distance 710 from the target crosswalk 720. For example, the trip planner can set the lateral margins 751, 752 in proportion to the distance 710 from the target crosswalk 720. Alternatively, the trip planner can set the lateral margins 751, 752 in proportion to the crossing length 722 of the target crosswalk 720.

[0072] At this time, the driving planning device can set an approach lateral margin 751 to the target crosswalk 720 and an exit lateral margin 752 from the target crosswalk 720 so that they are different.

[0073] In one embodiment, the trip planning device can set the area of ​​interest 730 to an arbitrary shape. For example, the trip planning device can set the area of ​​interest 730 to an arbitrary shape determined using at least one of the road topography and the trip route 700. Alternatively, the trip planning device can set the area of ​​interest 730 to a specific shape that has already been determined.

[0074] Meanwhile, the driving planning device can set the region of interest 730 and margins 741, 742, 751, and 752 of the region of interest 730 based on the Frenet frame. Specifically, since the region of interest 730 has a lateral margin and a vertical margin based on the travel path of the vehicle, the region of interest 730 can be acquired using Frenet frame-based parameters, and the margins 741, 742, 751, and 752 of the region of interest 730 can also be set based on the Frenet frame.

[0075] FIG. 10 is a flow chart of a method for determining whether to generate a virtual stop line according to one embodiment.

[0076] In one embodiment, the driving planner can detect nearby objects present around the vehicle. The nearby objects may be objects within the region of interest or may be objects outside the region of interest. For example, the driving planner may include or be equipped with various imaging devices or sensors, such as a camera, an image sensor, a radar (RADAR) sensor, a lidar (LIDAR) sensor, etc., which can be used alone or in a fused form to detect nearby objects around the vehicle and collect information about the vehicle's surroundings. Thus, the detected object may correspond to not only the shape of the captured image but also a plurality of measurement points acquired by an image sensor or the like. In addition, the information about the vehicle's surroundings may include recognition data and status data. The nearby objects may include not only pedestrians but also other vehicles.

[0077] In one embodiment, the driving planner can determine whether to generate a virtual stop line based on the position of a pedestrian in the area of ​​interest. If the driving planner controls the vehicle to slow down or stop only based on whether a pedestrian is in the area of ​​interest, efficiency is reduced, and safety may be reduced when there are many vehicles on the road. In the present disclosure described later, the driving planner can improve control stability by calculating a cost function related to pedestrians in the area of ​​interest to determine whether to generate a virtual stop line, or by determining whether to generate a virtual stop line based on the distance from the target pedestrian crossing.

[0078] Referring to FIG. 10, in step 1010, the trip planner can recognize pedestrians within the area of ​​interest.

[0079] In step 1020, the trip planner can determine whether the pedestrian is located in the first area or the second area. For example, the trip planner can determine whether the pedestrian is located in a pedestrian crossing.

[0080] In one embodiment, the trip planner may determine to generate a virtual stop line in response to determining that the pedestrian is located in the first area (step 1030). The first area may refer to an inner area of ​​the target crosswalk. In this case, the trip planner may generate the virtual stop line and control the vehicle to decelerate.

[0081] In another embodiment, the driving planner may again determine whether to generate a virtual stop line based on the distance from the target crosswalk in response to determining that the pedestrian is located in the second area. The second area may refer to an area corresponding to an intersection of an external area of ​​the target crosswalk and an internal area of ​​the area of ​​interest. In determining whether to generate a virtual stop line, the driving planner may determine to generate a virtual stop line if the distance from the target crosswalk is equal to or greater than a predetermined distance, and may determine not to generate a virtual stop line if the distance from the target crosswalk is less than the predetermined distance. That is, even if a pedestrian is located in the area of ​​interest, if the pedestrian is located close to the vehicle, it may determine not to generate a virtual stop line.

[0082] In step 1040, the trip planner can calculate a cost function for the pedestrian, compare it with a threshold, and determine whether to generate a virtual stop line based on the cost function. The cost function can be calculated based on one or more of the vertical distance and vertical speed of the pedestrian with respect to the travel route. The vertical distance and vertical speed of the pedestrian with respect to the travel route can mean the lateral distance and lateral speed of the pedestrian when the travel route is assumed to be vertical. In other words, the trip planner can calculate the cost function by considering how far the pedestrian is from the travel route in the lateral direction and how fast the pedestrian moves in the lateral direction.

[0083] In one embodiment, the trip planning device can calculate the cost function by considering a cost according to the vertical distance of the pedestrian with respect to the trip route and a cost according to the vertical speed of the pedestrian with respect to the trip route. The trip planning device can consider factors other than the above costs, and is not limited thereto.

[0084] In one embodiment, the trip planning device can obtain the vertical distance of the pedestrian with respect to the trip route by the Fresnet frame-based parameters described above when taking the cost into consideration.

[0085] In one embodiment, the driving planner can consider at least one of the vertical distance of the pedestrian with respect to the driving route and the variance of the vertical distance in calculating the cost according to the vertical distance of the pedestrian with respect to the driving route. Also, the driving planner can consider at least one of the general walking speed of the pedestrian (slow walking speed and fast walking speed), the vertical speed of the pedestrian with respect to the driving route and the variance of the vertical speed in calculating the cost according to the vertical speed of the pedestrian with respect to the driving route.

[0086] In one embodiment, the driving planner can calculate the cost function in the form of a Gaussian distribution. Specifically, the driving planner can calculate the cost function in the form of a Gaussian distribution based on the vertical distance of the pedestrian to the driving route, the vertical speed of the pedestrian to the driving route, and other factors.

[0087] In one embodiment, the driving planner can generate a virtual stop line if the cost function is greater than or equal to a threshold (step 1050), and can not generate a virtual stop line if the cost function is less than the threshold (step 1060).

[0088] In this way, by the driving planning device determining whether or not to generate a virtual stop line based on the cost function, it has the effect of adding a reasonable limiting factor to the method of generating a virtual stop line and slowing down.

[0089] 11 to 13 are exemplary diagrams illustrating implementation of a method for setting an area of ​​interest and a method for determining whether to generate a virtual stop line according to an embodiment.

[0090] 11, as described above in step 1020 and step 1030 of FIG. 10, the travel planning device can generate a virtual stop line 1113 in response to determining that a pedestrian 1111 is located in the first area 1112. According to the method for generating a virtual stop line of the present invention, the travel planning device generates the virtual stop line 1113 in advance when the vehicle is far away from the pedestrian 1111 and approaches while decelerating, so that even if the travel planning device determines that the pedestrian 1111 is located in the first area 1112 at a close distance and generates the virtual stop line 1113, the vehicle can stop without suddenly decelerating.

[0091] FIG. 12 is an exemplary diagram showing the implementation when the vehicle is far away from the target crosswalk, and FIG. 13 is an exemplary diagram showing the implementation when the vehicle is close to the target crosswalk.

[0092] 12 and 13, it can be seen that the driving planning device increases the margin of the region of interest as the distance from the target crosswalk increases, resulting in a wider region of interest. Also, the driving planning device can confirm that for a pedestrian located at a second position within the region of interest, i.e., an area outside the target crosswalk, if the distance from the target crosswalk is equal to or greater than a predetermined distance (far distance), a virtual stop line is generated in advance and the pedestrian approaches the target crosswalk, and if the distance from the target crosswalk is less than the predetermined distance (near distance), a virtual stop line is not generated and the pedestrian passes through the target crosswalk.

[0093] On the other hand, even if the vehicle is close to the target crosswalk, the travel planning device will have generated a virtual stop line in advance at a long distance and slowed down the vehicle, so that the vehicle will not pass near the target crosswalk at high speed, thereby ensuring safety.

[0094] 5 again, in one embodiment, the driving planner can detect a nearest object 540. The nearest object 540 means an object that is nearest to the vehicle 500 among the nearest objects traveling on the driving route 510 and exists within the area of ​​interest 560.

[0095] Meanwhile, the driving planning device can obtain behavior prediction information of the nearby objects when detecting the nearby objects 520, 530, 540, and 550. The behavior prediction information means information that predicts the near future behavior of the object by taking into account both the position information and the speed information of the object.

[0096] Since the position of the vehicle 500 cannot be beyond the position of the nearest object 540, the driving planner must take into consideration the position and behavior of the nearest object 540 when planning the driving speed of the vehicle 500. Therefore, when the nearest object 540 is detected, the driving planner can plan the driving speed of the vehicle 500 based on the behavior prediction information of the nearest object 540.

[0097] In one embodiment, the driving planner can set the relevance of the nearby objects 520, 530, 540, 550 based on the region of interest 560. The relevance is a value reflecting the interrelationship between the vehicle 500 and the nearby objects 520, 530, 540, 550, and can be an index indicating how much the position and behavior of the nearby objects 520, 530, 540, 550 should be considered when the driving planner plans the driving speed. For example, when the distance 521 of a certain nearby object 520 to the driving route is close, it is necessary to weight and reflect the position and behavior of the nearby object 520 in planning the driving speed, so the driving planner can set the relevance of the nearby object 520 to be high.

[0098] Similarly, when a certain nearby object 550 is far from the driving route (not shown), or when a certain nearby object 550 does not exist in the region of interest 560, the driving planner can reflect the position and behavior of the nearby object 550 with very little weighting, or not at all, in planning the driving speed. Therefore, when the nearby object 550 does not exist in the region of interest 560, the driving planner can set the relevance of the nearby object 550 to 0.

[0099] Meanwhile, the driving planner can acquire the distance 522 from the driving route by reflecting the width of the vehicle 500 and the nearby objects 520, 530, 540, and 550. For example, the driving planner can acquire the distance 522 from the driving route of the nearby object 520 excluding the space occupied by the volume of each object itself.

[0100] When the driving planning device plans the driving speed of the vehicle, it does not simply make a binary judgment as to whether or not other objects are included in the area of ​​interest and perform a sense of foreignness in the sense of speed / acceleration, but rather sets the relevance with other objects and determines how much the vehicle should speed / accelerate according to that continuous relevance, thereby improving safety and ride comfort.

[0101] In one embodiment, when the nearby objects 520, 530, 540 are present within the region of interest 560, the driving planner can set the relevance to be inversely proportional to the distance to the driving route of the nearby objects 520, 530, 540. For example, in FIG. 5, since the distance (not shown) to the driving route of a specific nearby object 530 is closer than the distance 521 to the driving route of another nearby object 520, the driving planner can set the relevance of the nearby object 530 higher than the relevance of the other nearby objects 520 so as to be inversely proportional to the distance to each driving route.

[0102] In one embodiment, when the closest object 540 is detected, the driving planner can set the relevance of the closest object 540 to 1. By doing so, as described above, the driving speed of the vehicle 500 can be planned so as to maintain a safe distance from the closest object 540 while maximally reflecting the position and behavior of the closest object 540.

[0103] Returning again to FIG. 4, in one embodiment, pedestrians 410, 420, 430, and 440 may each have a corresponding pedestrian intent vector and pedestrian intent.

[0104] The pedestrian intention vector may be a normalized vector that estimates a probability distribution regarding multiple intentions that a pedestrian may have, and is a vector having dimensions corresponding to the multiple intentions. In addition, the pedestrian intention is an estimate of the intention of the pedestrians 410, 420, 430, and 440, and can be defined in a discrete space that is unobservable and has a finite number of actions.

[0105] In one embodiment, the multiple intentions may include a first intention to cross the driving path of the vehicle 400. For example, when the driving planner determines that there is a high probability that the pedestrian 430 intends to cross the driving path of the vehicle 400 based on the lateral position, longitudinal position, lateral speed, longitudinal speed, and the like according to the coordinates of any pedestrian 430, the driving planner may calculate a high value for the first intention of the pedestrian intention vector of the pedestrian 430.

[0106] In one embodiment, the multiple intentions may include a second intention to wait without crossing the driving path of the vehicle 400. For example, when the driving planner determines that there is a high probability that the pedestrian 410 intends to wait without crossing the driving path of the vehicle 400 based on the lateral position, longitudinal position, lateral speed, longitudinal speed, and the like according to the coordinates of any pedestrian 410, the driving planner may calculate a high value for the second intention of the pedestrian intention vector of the pedestrian 410.

[0107] In one embodiment, the multiple intentions may include a third intention other than the first intention and the second intention. For example, when the driving planner determines that there is a high probability that the pedestrian 420, 440 has an intention that is not a factor to be considered in the driving of the vehicle 400, other than an intention to cross the driving path of the vehicle 400 or an intention to wait without crossing, based on the lateral position, longitudinal position, lateral speed, longitudinal speed, etc. according to the coordinates of any pedestrian 420, 440, the driving planner may calculate a high value for the third intention of the pedestrian intention vector of the pedestrian 420, 440.

[0108] In one embodiment, the cruise planner can estimate or determine the intention corresponding to the maximum value of the pedestrian intention vector as the pedestrian intention. For example, if the first intention is "CROSSING", the second intention is "WAIT", and the third intention is "NONE", the pedestrian intention vector can be defined as {CROSSING, WAIT, NONE}. Assuming that the cruise planner estimates the pedestrian intention of a pedestrian 430 around the vehicle 400 and determines that the intention is highly likely to be an intention to cross the travel route of the vehicle 400, the pedestrian intention vector of the pedestrian 430 can be calculated as {0.8, 0.15, 0.05}, and the first intention corresponding to the maximum value 0.8 from among the multiple intentions (CROSSING, WAIT, NONE) of the pedestrian intention vector can be estimated or determined as the pedestrian intention of the pedestrian 430.

[0109] FIG. 14 is a conceptual diagram of a method for determining a driving speed utilizing pedestrian intention estimation according to one embodiment.

[0110] 14, the trip planning device may include a recognition module (Observer) 1410, a pedestrian intention estimation module (Pedestrian Intention Estimator) 1420, and a travel speed determination module (Motion Planner) 1430. For convenience of explanation, it is described that each module performs a different function, but each module may perform the same function and may not be a separate module.

[0111] In one embodiment, the perception module 1410 can recognize pedestrians around the vehicle and obtain perception data 1440 and state data 1440 to send to the pedestrian intention estimation module 1420.

[0112] In one embodiment, the perception module 1410 can detect objects including other vehicles and pedestrians around the vehicle, recognize pedestrians by identifying pedestrians from among the detected objects, and send perception data 1440 and state data 1440 to the pedestrian intent estimation module 1420.

[0113] In one embodiment, the perception module 1410 can obtain state data 1440 in a state space. The state data 1440 refers to all information related to the current state of the surrounding environment. That is, the surrounding environment and the state data 1440 can have the same meaning at one point in time. For example, the state space may include coordinate information including a lateral position and a longitudinal position of a pedestrian, and speed information including a lateral speed and a longitudinal speed of a pedestrian.

[0114] The state space may also include the pedestrian's intention at the current time (not shown). Specifically, the pedestrian's intention at the current time (not shown) may refer to the pedestrian's intention estimated by the driving planner at a previous time. The previous time and the current time will be described later.

[0115] In one embodiment, the perception module 1410 can acquire perception data 1440 in a perception (or observation) space. The perception data 1440 can be a subset of the state data 1440 and does not have all the information compared to the state data 1440. For example, the perception space may include coordinate information including a pedestrian's lateral and longitudinal positions, and speed information including a pedestrian's lateral and longitudinal velocities. However, unlike the state space, the perception space does not include pedestrian intentions, which are unobservable and defined in a discrete space.

[0116] In one embodiment, the pedestrian intent estimation module 1420 can update the pedestrian intent vector with at least one of the perception data 1440 and the state data 1440 received from the perception module 1410.

[0117] In one embodiment, the driving planner can determine an update period. The update period can refer to a period for updating the pedestrian's intention and determining the vehicle's driving speed. By determining the update period, a time point at which the recognition module 1410 acquires the recognition data 1440 and the state data 1440 is determined, and a time point at which the pedestrian intention estimation module 1420 updates the pedestrian intention vector or estimates the pedestrian intention 1450 is determined. In other words, the recognition module 1410 acquiring the recognition data 1440 and the state data 1440 can refer to acquiring the recognition data 1440 at the current time point and the state data 1440 at the current time point.

[0118] On the other hand, the time when one update period ends with respect to the current time can be defined as the next time, and the time before one update period with respect to the current time can be defined as the previous time. Therefore, the driving planning device can obtain the next time by determining the update period.

[0119] In one embodiment, the pedestrian intention estimation module 1420 may update the current pedestrian intention vector to the next pedestrian intention vector based on an update period.

[0120] For example, the pedestrian intention estimation module 1420 can apply Bayes' rule to update the pedestrian intention vector at the current time point to the pedestrian intention vector at the next time point. Bayes' rule is a rule that shows the relationship between the prior probability and the posterior probability of two random variables. Using Bayes' rule, it is possible to calculate the posterior probability, which includes uncertainty, from the prior probability, whose value can be obtained by observation. Thus, the pedestrian intention estimation module 1420 can use Bayes' rule to calculate the pedestrian intention vector at the next time point from the recognition data 1440 at the current time point and / or the pedestrian intention vector at the current time point, and update the pedestrian intention vector at the current time point to the pedestrian intention vector at the next time point.

[0121] For example, the pedestrian intention estimation module 1420 can calculate the probability that state data at the next time point will be acquired based on an observation probability function that takes the recognition data 1440 at the current time point as input and a state transition function that takes the state data 1440 at the current time point as input.

[0122] The state transition function means a function that calculates the probability that state data at the next time point will be acquired for the current state data 1440. Specifically, the state transition function may be a function that calculates a conditional probability that state data at the next time point will be acquired with the current state data 1440 as a sample space. Thus, the state transition function may be a function related to the current state data 1440 and the current pedestrian intention vector.

[0123] The observation function means a function that calculates the probability that the recognition data 1440 at the current time point is acquired for the state data at the next time point. Specifically, the observation function may be a function that calculates the conditional probability that the recognition data 1440 at the current time point is acquired with the state data at the next time point as a sample space.

[0124] In one embodiment, the state transition function and the observation function may be empirically designed functions utilizing Gaussian Distribution, or the observation function may be a function designed based on the performance indicators of the recognition module 1410, such as the recognition performance for pedestrians, standard deviation, degree of noise, etc.

[0125] In summary, the pedestrian intention estimation module 1420 can calculate the probability that the next state data will be obtained for the current recognition data 1440 and the current pedestrian intention vector.

[0126] In one embodiment, the pedestrian intention estimation module 1420 can update the pedestrian intention vector at the current time point to a pedestrian intention vector at the next time point based on the calculated probability that the next time point state data will be acquired. The pedestrian intention vector at the next time point is a vector that estimates a probability distribution of multiple intentions that the pedestrian may have at the next time point.

[0127] In one embodiment, the driving planner can update the pedestrian intention vector at the current time point to the pedestrian intention vector at the next time point based on a pedestrian behavior model. The pedestrian behavior model is a model that analyzes the characteristics and behavior patterns of pedestrians and predicts the behavior at the next time point based on the behavior at the current time point. Specifically, by using a pedestrian behavior model corresponding to each pedestrian intention, the behavior at the next time point can be predicted for the pedestrian intention vector, and the probability of multiple intentions can be calculated from the state transition function and the observation function based on the predicted behavior.

[0128] The trip planner can determine the probability of each of a plurality of intentions of the pedestrian at the next time point based on the pedestrian behavior model, and thereby update the pedestrian intention vector. That is, the trip planner can utilize the pedestrian behavior model to design a state transition function and update the pedestrian intention vector.

[0129] In one embodiment, the pedestrian intention estimation module 1420 can estimate the next pedestrian intention 1450 based on the updated pedestrian intention vector and send the estimated next pedestrian intention 1450 to the driving speed determination module 1430.

[0130] For example, the pedestrian intention estimation module 1420 may estimate the intention corresponding to the maximum value of the updated pedestrian intention vector as the next pedestrian intention 1450. Specifically, if the pedestrian intention vector is defined as {CROSSING, WAIT, NONE} and a pedestrian's next pedestrian intention vector is {0.8, 0.15, 0.05}, the pedestrian's next pedestrian intention 1450 may be "CROSSING."

[0131] FIG. 15 is a diagram for explaining the first to third intentions of a pedestrian according to an embodiment.

[0132] The drive planning device can recognize a pedestrian 1530 estimated as a first intention, a pedestrian 1510 estimated as a second intention, and pedestrians 1520, 1540 estimated as a third intention, around a vehicle 1500. Among the descriptions of the vehicle 1500 and the pedestrians 1510, 1520, 1530, 1540 in Fig. 15, descriptions that overlap with the descriptions of the vehicle 400 and the pedestrians 410, 420, 430, 440 in Fig. 4 will be omitted.

[0133] In order to provide a safe trip plan for all passengers and pedestrians 1510, 1520, 1530, 1540, the trip planning device must take into account the intentions of the pedestrians 1510, 1520, 1530, 1540 when determining the travel speed of the vehicle 1500.

[0134] The pedestrian 1530 estimated as having the first intention is likely to be located on the travel path of the vehicle 1500 at the next time point, based on the current lateral and vertical positions of the pedestrian 1530 and the current lateral and vertical speeds of the pedestrian 1530. Therefore, the first intention may have the maximum value among the pedestrian intention vectors calculated based on the pedestrian behavior model.

[0135] As described above, the pedestrian 1510 estimated as having the second intention has a low probability of being located on the driving path of the vehicle 1500 at the next time point based on the current information of the pedestrian 1510. However, there is a certain probability that the pedestrian is close to the driving path or will be located on the driving path at a nearby time point. Therefore, the second intention can have the maximum value among the pedestrian intention vectors calculated based on the pedestrian behavior model.

[0136] Pedestrians 1520, 1540 estimated to have a third intention have no probability of being located on the driving path of vehicle 1500 at the next time based on the current information of pedestrians 1520, 1540, or have a speed in the opposite direction to the driving path, and therefore do not need to be taken into consideration in planning the driving speed of vehicle 1500. Therefore, the third intention can have the maximum value among the pedestrian intention vectors calculated based on the pedestrian behavior model.

[0137] In one embodiment, the driving planner can estimate the intention corresponding to the maximum value of the pedestrian intention vector, i.e., the pedestrian intention of the pedestrian 1530 at the next time point as the first intention, the pedestrian intention of the pedestrian 1510 at the next time point as the second intention, and the pedestrian intentions of the pedestrians 1520 and 1540 at the next time points as the third intention.

[0138] In one embodiment, the driving speed determination module may determine the driving speed of the vehicle by taking into account the pedestrian intention at the next time point received from the pedestrian intention estimation module.

[0139] For example, the driving speed determination module can determine weights for the pedestrians 1510, 1520, 1530, 1540 from the estimated pedestrian intentions and determine the driving speed of the vehicle based on the weights for the pedestrians 1510, 1520, 1530, 1540.

[0140] 15, the driving planner may determine to give a higher weight to pedestrian 1530 estimated as a first intention than pedestrian 1510 estimated as a second intention. In addition, the driving planner may determine to give a higher weight to pedestrian 1510 estimated as a second intention than pedestrians 1520 and 1540 estimated as a third intention.

[0141] The weights given to the pedestrians 1510, 1520, 1530, 1540 may determine how much the behavior of the pedestrians is reflected in determining the traveling speed of the vehicle 1500. In one embodiment, for the pedestrian 1530 estimated as a first intention, acceleration and deceleration may be determined conservatively, for the pedestrian 1510 estimated as a second intention, the traveling speed may be determined with uncertainty, and the behavior of the pedestrians 1520, 1540 estimated as a third intention may be reflected relatively less or the traveling speed may be determined with little consideration.

[0142] In one embodiment, the driving planner can repeat the above steps or the above embodiments for each time point determined based on the update period. For example, the driving planner can recognize the pedestrians 1510, 1520, 1530, 1540 for each time point, obtain recognition data and state data for the current time point, estimate a probability distribution of multiple intentions that the pedestrians 1510, 1520, 1530, 1540 may have at the next time point, update the pedestrian intention vector, estimate the pedestrian intention at the next time point based on the updated pedestrian intention vector, and determine the driving speed of the vehicle 1500 taking the estimate into consideration. The driving planner can determine the driving speed of the vehicle 1500 at the time point when two update periods have elapsed based on the current time point by driving to the next time point at the determined driving speed and then repeating the above steps at the next time point. In this way, the driving speed can be determined so as to reflect the intentions of the pedestrians 1510, 1520, 1530, 1540 that change at each time point.

[0143] In one embodiment, assuming that "CROSSING" means a first intention, "WAIT" means a second intention, and "NONE" means a third intention, the driving planner can determine the driving speed by giving the highest weight to the pedestrian estimated as the first intention. Also, the driving planner can determine the driving speed by giving a higher weight to the pedestrian estimated as the second intention than to the pedestrian estimated as the third intention, and the driving planner can determine the driving speed by giving the lowest weight to the pedestrian estimated as the third intention. In other words, the driving planner can determine the driving speed by giving a conservative weight to the pedestrian estimated as the first intention, and a weight that rapidly decreases as the distance from the vehicle 1500 increases to the pedestrian estimated as the third intention. However, the method in which the driving planner determines the weight of the pedestrian is not limited thereto.

[0144] FIG. 16 is a diagram for explaining a method for planning the travel speed of a vehicle based on behavior prediction information of the nearest object according to an embodiment.

[0145] 16, it can be assumed that a nearest object 1620, which is the closest object among objects traveling on a traveling route of a vehicle 1610, is present within the region of interest. In this case, in one embodiment, the traveling planner can determine a reference position profile 1650 based on at least one of the standard position profile 1630 of the vehicle 1610, the behavior prediction information of the nearest object 1620, and the relevance of the nearest object 1620.

[0146] Specifically, when a nearest object 1620 exists, the driving planning device derives a corrected position profile 1650 by correcting the standard position profile 1630 based on the relevance or behavior prediction information of the nearest object 1620, and can determine the reference position profile 1650 using the corrected position profile 1650. In the present embodiment shown in FIG. 16, since there is one nearest object 1620, the corrected position profile 1650 can be determined as the reference position profile 1650 as it is.

[0147] The position profile refers to a profile that reflects the change in displacement of an arbitrary object over time, with the time axis and displacement axis set to the x-axis and y-axis, respectively.

[0148] Meanwhile, the position profiles of the nearby object, the nearest object, and the Nth peripheral object described later may refer to position profiles in which the behavior prediction information of the object is corrected by reflecting a safe distance between the object and the vehicle with respect to a behavior prediction profile shown in a time-displacement graph. For example, in Fig. 16, the position profile 1640 of the nearest object 1620 may refer to a position profile in which the behavior prediction information of the nearest object 1620 is corrected by reflecting a safe distance between the vehicle 1610 and the nearest object 1620 with respect to a behavior prediction profile 1641 of the nearest object 1620 shown in a time-displacement graph.

[0149] The standard position profile 1630 may refer to a position profile reflecting the current traveling speed of the vehicle 1610. Alternatively, the standard position profile 1630 may refer to a position profile corresponding to a preferred traveling speed for the vehicle 1610 (e.g., a speed corresponding to a predetermined percentage of the maximum speed limit set on the road being traveled) when there is no nearby object around the vehicle 1610 or no nearby object within the area of ​​interest. In other words, the standard position profile 1630 may refer to a position profile when the vehicle 1610 travels at a desired speed.

[0150] The corrected position profile 1650 means a position profile obtained by correcting the standard position profile 1630 by reflecting behavior prediction information of each nearby object 1620 when there are nearby objects 1620 around the vehicle 1610. In other words, since the corrected position profile 1650 corresponds to a position profile corrected by each nearby object, when there are multiple nearby objects excluding the closest object, multiple corrected position profiles corresponding to the number of the nearby objects can be derived. A method of deriving multiple corrected position profiles will be described in detail with reference to Figs. 17 and 18.

[0151] The reference position profile 1650 means a position profile obtained by correcting the standard position profile 1630 in consideration of behavior prediction information of all of the nearby objects 1620 when the nearby objects 1620 are present around the vehicle 1610. In one embodiment, the driving planner can determine one reference position profile 1650 by using all of the derived corrected position profiles 1650.

[0152] In one embodiment, the driving planner determines a standard position profile 1630 of the vehicle 1610, and determines a reference position profile 1650 based on the standard position profile 1630 and the behavior prediction information of the nearest object 1620. For example, the time when the standard position profile 1630 and the position profile 1640 of the nearest object 1620 derived from the behavior prediction information of the nearest object 1620 come into contact is defined as t 1 Then, the driving planner can calculate the distance from the current time to t 1 Up to t matches the standard position profile 1630, 1 Thereafter, a reference position profile 1650 can be determined to match the position profile 1640 of the closest object 1620 .

[0153] By determining a final position profile (not shown), which will be described later, with reference to the reference position profile 1650 determined in this manner, it is possible to enable safe travel without collision with other objects.

[0154] 17 and 18 are diagrams illustrating a process of deriving a corrected position profile based on the relevance of a nearby object according to an embodiment.

[0155] FIG. 17 shows a process of determining a reference position profile based on the relevance of a nearby object, assuming that the travel planning device has set the relevance of the nearby object to 0.4.

[0156] When the driving planner finds a nearby object stopped at a predetermined distance 20 m ahead of the current position of the vehicle in the lateral direction, the driving planner can obtain a position profile 1711 of the nearby object as shown in the graph in Fig. 17. At this time, the driving planner can derive a corrected position profile 1710 by correcting the standard position profile based on the relevance of the nearby object.

[0157] In one embodiment, the driving planner can derive the corrected position profile 1710 so that the standard position profile is 40% closer to the position profile 1711 of the nearby object. For example, the time when the standard position profile and the position profile 1711 of the nearby object meet can be set as t 1 Then, the driving planner can calculate the distance from the current time to t 1 Up to t matches the standard position profile, 1 Thereafter, a corrected position profile 1710 can be derived so as to be 40% closer to the position profile 1711 of the nearby object.

[0158] Since there is one nearby object present within the region of interest, the driving planner can determine the derived corrected position profile 1710 as the reference position profile.

[0159] FIG. 18 shows a process of determining a reference position profile based on the relevance of a nearby object, assuming that the travel planning device has set the relevance of the nearby object to 0.8.

[0160] In one embodiment, if the driving planner finds a nearby object stopped at a predetermined distance laterally 20 m ahead of the vehicle's current position, the driving planner can obtain a position profile 1821 of the nearby object as shown in the graph in FIG. 18.

[0161] At this time, the driving planning device can derive the corrected position profile 1820 so that the standard position profile is 80% closer to the position profile 1821 of the nearby object. For example, the time when the standard position profile and the position profile 1821 of the nearby object contact each other is set as t 2 Then, the driving planner can calculate the distance from the current time to t 2 Up to t matches the standard position profile, 2 Thereafter, a corrected position profile 1820 can be derived so as to be 80% closer to the position profile 1821 of the nearby object.

[0162] Since there is one nearby object present within the region of interest, the driving planner can determine the derived corrected position profile 1820 as the reference position profile.

[0163] 19 and 20 are diagrams illustrating a process of determining a reference position profile in an environment where multiple nearby objects exist, according to an embodiment.

[0164] The driving planner can detect nearby objects 1910, 1920, and 1930 including a closest object 1930 within a region of interest of a vehicle 1900. Hereinafter, for convenience of explanation, the closest object 1930 is defined as a first nearby object, and the nearby objects 1910 and 1920 that are not the closest object 1930 are defined as second nearby objects.

[0165] In one embodiment, the driving planning device can derive one or more corrected position profiles 1911, 1921 by correcting the standard position profile 1901 based on behavior prediction information and relevance of a first nearby object 1930 and one or more second nearby objects 1910, 1920. The corrected position profiles 1911, 1921 correspond to the second nearby objects 1910, 1920, respectively.

[0166] In order to derive a corrected position profile 1911 corresponding to the second nearby object 1910, the driving planning device considers the standard position profile 1901, the position profile of the second nearby object 1910, and the position profile of the nearest object 1930. Therefore, the time when the standard position profile 1901 and the position profile of the second nearby object 1910 come into contact is defined as t 1 The time when the standard position profile 1901 and the position profile of the nearest object 1930 come into contact is defined as t 3 It is defined as:

[0167] The travel planning device calculates the distance from the current time to t 1 Up to t matches standard position profile 1901, and1 From 3 up to t corresponds to a position profile corrected to correspond to the behavior prediction information of the second nearby object 1910 in proportion to the relevance of the second nearby object 1910, 3 Thereafter, the standard position profile 1901 can be corrected to correspond to the behavior prediction information of the closest object 1930, thereby deriving the corrected position profile 1911. In one embodiment, when the relevance of the second closest object 1910 is 0.2, t 1 From 3 The standard position profile 1901 is corrected so as to be 20% closer to the position profile of the second closest object 1910 up to t 3 Thereafter, the standard position profile 1901 can be corrected so that it corresponds to the position profile of the closest object 1930 and the position profile of the second closest object 1910 in a ratio of 8:2.

[0168] Similarly, the driving planner can derive a corrected position profile 1921 corresponding to another second nearby object 1920. The time when the standard position profile 1901 and the position profile of the second nearby object 1920 come into contact with each other is defined as t 2 The time when the standard position profile 1901 and the position profile of the nearest object 1930 come into contact is defined as t 3 It is defined as:

[0169] The travel planning device calculates the distance from the current time to t 2 Up to t matches standard position profile 1901, and 2 From 3 up to t corresponds to a position profile corrected to correspond to the behavior prediction information of the second nearby object 1920 in proportion to the relevance of the second nearby object 1920, 3Thereafter, the standard position profile 1901 can be corrected to correspond to the behavior prediction information of the closest object 1930, thereby deriving the corrected position profile 1921. In one embodiment, when the relevance of the second closest object 1920 is 0.6, t 1 From 3 The standard position profile 1901 is corrected so as to be 60% closer to the position profile of the second closest object 1920 up to t 3 Thereafter, the standard position profile 1901 can be corrected so that it corresponds to the position profile of the closest object 1930 and the position profile of the second closest object 1920 in a ratio of 6:4.

[0170] Referring to FIG. 20, the driving planner can determine a reference position profile 2040 using one or more corrected position profiles 2011, 2021 corresponding to a first nearby object and one or more second nearby objects 2010, 2020, respectively.

[0171] 20, it can be seen that any displacement value of each of the corrected position profiles 2011, 2021 is less than or equal to any displacement value of the standard position profile 2001. In this manner, the driving planning device can determine the reference position profile 2040 such that any displacement value of the reference position profile 2040 is always less than or equal to any displacement value of each of the corrected position profiles 2011, 2021.

[0172] In one embodiment, the time when the corrected position profiles 2011 and 2021 meet is t * Then, the driving planner can calculate t 1 , t 2 , t * and t 3Based on this, time is divided into five intervals, and a reference position profile 2040 can be determined so as to match the position profile having the smallest displacement value among the standard position profile 2001 and one or more corrected position profiles 2011, 2021 in each interval.

[0173] Specifically, the travel planning device 1 Up to t corresponds to the Standard Position Profile 2001, 1 From 2 Up to and t 2 From * The correction position profile 2011 has the smallest displacement value in the section up to t * From 3 Up to and t 3 In the subsequent section, the reference position profile 2040 can be determined so as to match the corrected position profile 2021 having the smallest displacement value.

[0174] With reference to the above-mentioned embodiment, a method for the driving planning device to determine a reference position profile will be described assuming that a nearest object (first proximate object) and one or more second proximate objects are detected within the region of interest. For convenience of explanation, the second proximate object is further defined as a first periphery object, a second periphery object, ..., an Nth periphery object in the order of proximity from the front of the vehicle, and a corrected position profile corresponding to the Nth periphery object is defined as an Nth corrected position profile. Here, N is a natural number corresponding to the number of second proximate objects.

[0175] The driving planning device can derive the Nth corrected position profile by making corrections to correspond to the behavior prediction information of the Nth peripheral object in proportion to the relevance of the Nth peripheral object from the point where the position profile of the Nth peripheral object and the standard position profile intersect, and by making corrections to correspond to the behavior prediction information of the nearest object from the point where the position profile of the nearest object and the standard position profile intersect.

[0176] In addition, the driving planning device can divide the time domain into multiple sections based on one or more time points at which the position profile of the Nth peripheral object and the standard position profile intersect and one or more time points at which the Nth corrected position profiles intersect (if N is 1, there is no time point at which the Nth corrected position profiles intersect), and determine a reference position profile so that in each section it matches the position profile having the smallest displacement value among the standard position profile, one or more Nth corrected position profiles, and the position profile of the nearest object.

[0177] In one embodiment, when the nearest object is not detected within the region of interest, the driving planning device can derive the Nth corrected position profile by correcting the position profile of the Nth peripheral object to correspond to the behavior prediction information of the Nth peripheral object in proportion to the relevance of the Nth peripheral object from the point where the position profile of the Nth peripheral object and the standard position profile intersect.

[0178] In addition, a reference position profile can be determined so as to match the position profile having the smallest displacement value among the standard position profile and one or more Nth corrected position profiles in each of multiple sections divided into a time domain based on one or more time points at which the position profile of the Nth peripheral object and the standard position profile intersect and one or more time points at which the Nth corrected position profiles intersect (if N is 1, there is no time point at which the Nth corrected position profile intersects).

[0179] FIG. 21 is an example diagram illustrating a final position profile according to one embodiment.

[0180] In one embodiment, the trip planner can determine a final position profile 2120 that corresponds to the reference position profile 2110 .

[0181] The final position profile 2120 refers to a position profile that indicates the final speed at which the vehicle 2100 actually travels.

[0182] In one embodiment, the trip planner can calculate a cost function based on the derived reference position profile 2110. The cost function is a function that indicates a relationship with a cost-related variable derived from the position profile traveled by the vehicle 2100. The cost-related variable is known from each term of the cost function.

[0183] In one embodiment, the driving planner can calculate a cost function based on an error cost corresponding to a displacement difference from the reference position profile 2110, an acceleration cost corresponding to the acceleration of the vehicle 2100, and an acceleration change cost corresponding to the jerk of the vehicle 2100.

[0184] The error cost corresponds to a cost due to a difference between the reference position profile 2110 and the position profile in which the vehicle 2100 travels. The reference position profile 2110 is a minimum position profile for traveling without colliding with nearby objects in the vicinity, so the greater the difference, the greater the impact on the safety of traveling. Therefore, the error cost is related to the safety of traveling.

[0185] The acceleration cost corresponds to a cost due to acceleration derived from a position profile of the vehicle 2100 traveling. A large magnitude of acceleration means a sudden acceleration or deceleration, so the acceleration cost is related not only to the safety of traveling but also to the comfort of the ride.

[0186] The acceleration change cost corresponds to a cost due to a jerk of the vehicle 2100. The jerk corresponds to a cost indicating a change in acceleration over time. When the vehicle 2100 travels, an instantaneous shock occurs due to a change in acceleration, and the larger the jerk, the worse the ride comfort becomes due to the instantaneous shock. Therefore, the acceleration change cost is related to the ride comfort.

[0187] In one embodiment, the trip planner can determine the position profile that minimizes the cost function as the final position profile. For example, the trip planner can perform an iterative calculation to obtain the position profile that minimizes the cost function, and can determine the solution derived from the calculation as the final position profile.

[0188] Meanwhile, the driving planner may determine a reference position profile and a final position profile for a predetermined time. The predetermined time means a specific time for planning a driving speed, and may be a predefined time or may be a time that varies based on the vehicle's driving speed, the number of nearby objects in the vicinity, etc. For example, the driving planner may ensure driving stability by setting a long predetermined time since the greater the number of nearby objects, the greater the traffic volume and the greater the complexity of the driving speed plan.

[0189] FIG. 22 is a diagram illustrating an example of a process for optimizing a vehicle trajectory according to an embodiment.

[0190] Referring to FIG. 22 , an environment for optimizing a vehicle trajectory according to the present invention may include a reference trajectory 2210 , driving information 2220 , a Model Predictive Control (MPC) module 2230 , a control input 2240 , and a vehicle 2250 .

[0191] The reference trajectory 2210 may mean a route along which the vehicle 2250 should travel. That is, it may be a travel route determined by the driving planning device according to the above-described embodiment. Alternatively, it may be a travel route acquired based on a travel speed determined by the driving planning device according to the above-described embodiment.

[0192] In one embodiment, the reference trajectory 2210 may include a global path, which corresponds to the path of the vehicle 2250 from a start point to a destination, and a local path, which corresponds to a predetermined onward path to avoid unpredictable, non-fixed obstacles.

[0193] The optimal control input 2240 or optimal trajectory described below may refer to a control input or trajectory for the vehicle 2250 to travel without deviating from the reference trajectory 2210.

[0194] In one embodiment, the reference trajectory 2210 may refer to a local path.

[0195] The driving information 2220 may include all information related to the driving state of the vehicle 2250, such as the current position of the vehicle 2250, the current control state, the current speed, and the received control input 2240. The driving information 2220 may refer to the current state variables of the vehicle 2250.

[0196] The control input 2240 refers to an input that controls the steering angle of the steering wheel. When the control input 2240 is input to the vehicle 2250, a DC motor for rotating the steering wheel and an encoder for reading the steering angle of the steering wheel are connected by a gear.

[0197] The MPC module 2230 refers to a module for determining an optimal control input 2240 for the vehicle 2250. MPC is a method of optimal control that generates an optimized control command by inputting object motion and surrounding environmental conditions into a cost function. The MPC module 2230 can select the optimal control input 2240 by calculating multiple predicted trajectories by one or more control inputs and calculating a cost function. The MPC module 2230 can use an iterative Linear Quadratic Regulator (iLQR) algorithm in selecting the optimal control input 2240.

[0198] The iLQR algorithm is an algorithm that uses Taylor approximation to approximate non-linear functions to linear functions and non-quadratic functions to quadratic functions, and can efficiently optimize a cost function and calculate an optimal trajectory from the initial state to the target state.

[0199] In the present invention, the driving planner can acquire or determine a reference trajectory 2210 and driving information 2220 of a vehicle 2250, and input a control input 2240 determined by an MPC module 2230 to the vehicle 2250. Hereinafter, a method in which the driving planner optimizes a vehicle trajectory will be described in detail with reference to Figs. 21 to 23.

[0200] FIG. 23 is a diagram for explaining a process in which the driving planning device according to an embodiment calculates a predicted trajectory.

[0201] Referring to FIG. 23, the driving planner can calculate a plurality of predicted trajectories 2301, 2302, 2303, 2304 based on one or more control inputs 2310, and select an optimal control input from among the one or more control inputs 2310.

[0202] In one embodiment, the trip planner can use a prediction horizon 2320 and a control period 2330 to determine multiple model prediction time points, including the current time point.

[0203] The forecast reference horizon 2320 represents the future output number to be predicted, which is the future period that the model can predict. The forecast reference horizon 2320 corresponds to the future control input number to be predicted.

[0204] In one embodiment, the driving planner may determine a 0th time point corresponding to the current time point to an Nth time point corresponding to the last time point of the predicted reference horizon 2320 based on a control period 2330. N is a natural number of 2 or more, and the control period 2330 means an interval between each time point. That is, the driving planner may determine a 0th time point corresponding to the current time point to an Nth time point corresponding to the last time point of the predicted reference horizon 2320 at an interval of the control period 2330. Here, N may correspond to a natural number obtained by dividing the predicted reference horizon 2320 by the control period 2330.

[0205] The driving planner can calculate a plurality of forecasted trajectories 2301, 2302, 2303, and 2304 based on a plurality of model prediction time points.

[0206] Since the predicted trajectory differs depending on the control input at multiple model prediction time points, the driving planner can generate one or more control inputs 2310 at each of the multiple model prediction time points and calculate multiple predicted trajectories 2301, 2302, 2303, and 2304 based on the one or more control inputs 2310.

[0207] In one embodiment, the driving planner can input one or more control inputs 2310 to the predictive model to generate multiple predicted trajectories 2301, 2302, 2303, 2304. In one embodiment, the iLQR-based MPC module can select an optimal control input that minimizes a cost function through an iterative optimization calculation, and at this time, the future movement of the vehicle can be understood using the predictive model. Therefore, hereinafter, a predictive model according to one embodiment will be described in detail with reference to Figures 24 and 25.

[0208] 24 and 25 are diagrams for explaining a prediction model for selecting an optimal control input by the driving planning device according to one embodiment.

[0209] Figure 24 shows a kinematic bicycle model with the control point on the rear axle, and Figure 25 shows a kinematic bicycle model with the control point on the front axle. At most vehicle driving speeds, the bicycle model can be used because the turning radius is much larger than the vehicle wheels.

[0210] 24, when the control point is on the rear wheel axle, the driving planner can apply the Instantaneous Center of Rotation (ICR) to calculate the state change rate for the control point. Specifically, the driving planner can derive the steering angle of the handlebars based on the rotation rate, turning radius, length, etc. of the bicycle model, and thereby calculate the state change rate for the control point.

[0211] Similarly, referring to Fig. 25, even if the control point is on the front wheel axle, the driving planner can calculate the state change rate for the control point. In this case, unlike Fig. 24, the state change rate may differ depending on the control point.

[0212] Alternatively, the prediction model may correspond to a kinematic bicycle model when the control point is at the center of gravity, and the prediction model is not limited to the kinematic bicycle model.

[0213] FIG. 26 is a diagram for explaining a process in which the driving planner according to an embodiment sets the control periods 2610 and 2620 non-uniformly.

[0214] In one embodiment, the trip planner can set the control periods 2610 and 2620 non-uniformly and determine multiple model prediction time points. For example, the trip planner can determine a reference time 2650 and set the control periods 2610 and 2620 non-uniformly based on the reference time 2650.

[0215] In general, for the prediction reference horizon of the local route, higher control performance is required for the near ahead for the stability of running. However, the shorter the control period 2610, 2620, the greater the amount of calculation, so it may not be efficient to set the control period 2610, 2620 short for a relatively long distance. Therefore, according to this embodiment, a short control period 2610 is set for the very near ahead, and a long control period 2620 is set for the subsequent prediction reference horizon, thereby achieving rational trajectory optimization.

[0216] The reference time 2650 means a point in time at which the control periods 2610 and 2620 are changed. The reference time 2650 can be set based on the efficiency according to the required control performance and the amount of calculation.

[0217] In one embodiment, since the driving planner needs to reproduce the reference trajectory 2630 using a speed profile in the non-uniform control periods 2610 and 2620, the reference trajectory 2630 may be a trajectory generated by interpolating a local route with a speed profile. For example, the driving planner can calculate time information related to the i-th time point, calculate a distance traveled by the vehicle at the i-th time point according to the calculated time information, and generate the reference trajectory using the distance. Therefore, the reference trajectory may be a trajectory reflecting the speed profile of the vehicle.

[0218] In one embodiment, the trip planner can adjust the reference time 2650 in response to determining that the vehicle has entered an environment where control performance is required above a threshold. As an example, the trip planner can adjust the reference time 2650 to be longer when it determines that the vehicle has entered an unpredictable obstacle-filled or rough terrain, such as an unpaved road. As another example, the trip planner can adjust the reference time 2650 to be longer when it determines that the vehicle has entered a road with very heavy traffic and congestion. In this way, high control performance can be applied to a point further away on the predicted reference horizon, preventing unexpected accidents.

[0219] In another embodiment, the driving planner may adjust the reference time 2650 in response to determining that the driving is to travel a global route that requires control performance equal to or greater than a threshold, by analyzing the global route. For example, when the driving planner determines that the route includes an unpaved road or rough terrain and requires high control performance during analysis of the global route corresponding to the route from the starting point to the destination, the driving planner may adjust the reference time 2650 to be longer before driving.

[0220] The driving planner can set a short control period 2610 from the 0th time point to the reference time 2650, and set a long control period 2620 from the reference time 2650 to the Nth time point. For example, referring to FIG. 26, the reference time 2650h, which is a time point of the forecast reference horizon, short Based on this, the control periods 2610 and 2620 are dt short , dt long Therefore, it can be seen that the time is not uniform from the current time point 0 to the reference time 2650h. short The control period is short, 2610, until dt short Calculate multiple predicted trajectories 2640 based on interval control inputs and set a reference time 2650h short From the Nth time point, the long control period 2620 is dt long A number of predicted trajectories 2640 with interval control inputs can be calculated.

[0221] In one embodiment, the driving planner may adjust the control period 2610, 2620 in response to determining that the vehicle enters an environment requiring control performance above a threshold or travels a global route requiring control performance above a threshold. For example, as described above, when the driving planner determines that the global route corresponds to an obstacle-ridden or rough terrain or a congested road, or that the vehicle has entered such an environment, the driving planner may adjust the control period 2610, 2620 to be shorter.

[0222] Returning again to FIG. 23, the driving planner can calculate a cost function based on at least one of a reference trajectory 2303, a plurality of predicted trajectories 2301, 2302, 2303, 2304, and one or more control inputs 2310.

[0223] Among one or more control inputs 2310, the control input at which the cost function has the minimum value is determined as the optimal control input, so how to set the cost function in the process of optimizing the vehicle trajectory is very important. Therefore, it is necessary to set a cost function that can guarantee both ride comfort and safety.

[0224] In one embodiment, the cost function may include an error cost, an input cost, and a variance cost.

[0225] Specifically, the driving planner can calculate the error cost based on the reference trajectory 2303. The error cost may correspond to a cost due to a difference between the reference trajectory 2303 on which the vehicle should travel and the trajectory on which the vehicle actually travels. If the difference between the reference trajectory 2303 and the trajectory on which the vehicle travels is large, it affects the safety of the driver and pedestrians, so for safety reasons, it is preferable to minimize the difference from the reference trajectory 2303. For example, the driving planner can calculate the error cost based on the difference between the reference trajectory 2303 and the predicted trajectory.

[0226] In addition, the driving planner can calculate the input cost based on the control input. The input cost may correspond to the cost due to the input of the control input. Since controlling the steering angle of the steering wheel while driving affects the ride comfort, it is preferable to minimize the control input for the sake of ride comfort. For example, the driving planner can calculate the input cost based on the control input itself.

[0227] Furthermore, the driving planning device can calculate the change amount cost based on the change amount of the control input. The change amount cost may correspond to the cost due to the steering angle of the steering wheel. Since the larger the steering angle of the steering wheel, the greater the impact on the ride comfort, it is preferable to minimize the change amount of the control input in order to improve the ride comfort.

[0228] In one embodiment, the trip planning device can calculate the cost function by considering the weights of the error cost, the input cost, and the change cost. The weights can be set by considering which of the safety of the trip and the comfort of the trip is more important.

[0229] In one embodiment, the weights of the error cost, the input cost, and the variation cost can be adaptively changed according to the behavior of the vehicle. For example, the predicted trajectories 2301, 2302, 2303, 2304, safety, and ride comfort according to the control input 2310 are different when the vehicle speed is slow and when it is fast, so the weights may not be constants but functions that can be adaptively changed in real time according to the behavior of the vehicle, such as the vehicle speed.

[0230] In one embodiment, the cost function may further include constraints on the control inputs and / or the amount of change in the control inputs.

[0231] In one embodiment, the driving planner can select an optimal control input from one or more control inputs 2310. For example, the driving planner can calculate a plurality of predicted trajectories using a prediction model, calculate a cost function for each of the plurality of predicted trajectories, and select a predicted trajectory that minimizes the cost function as an optimal predicted trajectory 2301. Here, the control input from which the optimal predicted trajectory 2301 is calculated can correspond to the optimal control input.

[0232] On the other hand, as described with reference to FIG. 20, the driving planner can use the iLQR algorithm to select the optimal control input using the prediction model and the cost function.

[0233] In one embodiment, the trip planner can repeat the above steps for each control period 2330 to optimize the vehicle trajectory. For example, the trip planner can delete control inputs at the (M+1)th time point to the Nth time point in response to selecting an optimal control input at the Mth time point (M is a natural number between 1 and N-1). In this way, the trip planner can select a new optimal control input for each time point, thereby achieving excellent control performance.

[0234] FIG. 27 is a flow chart of a method for generating a virtual stop line according to one embodiment.

[0235] Since the method of generating a virtual stop line shown in FIG. 27 is related to the embodiment described above, even if the content is omitted below, the content described above can also be applied to the method of FIG. 27.

[0236] The operation shown in Fig. 27 can be executed by the autonomous driving device described above. Specifically, the operation shown in Fig. 27 can be executed by a processor included in the autonomous driving device described above.

[0237] In step 2710, the device may determine a target crosswalk from among one or more crosswalks on the travel route.

[0238] In one embodiment, the device may determine the closest crosswalk ahead on the travel path as the target crosswalk.

[0239] In step 2720, the device may set a region of interest for the target crosswalk and set the region of interest using a region of interest margin.

[0240] In one embodiment, the device can use the target crosswalk to set a margin of the region of interest. The margin of the region of interest may include a vertical margin and a horizontal margin.

[0241] In one embodiment, the device can obtain a distance to a target crosswalk, obtain a width and a crossing length of the target crosswalk, set a vertical margin based on the width and the distance to the target crosswalk, and set a lateral margin based on the crossing length and the distance to the target crosswalk.

[0242] In one embodiment, the margin of the region of interest can be set larger the farther the distance to the target crosswalk is, and the margin of the region of interest can be set smaller the closer the distance to the target crosswalk is.

[0243] In step 2730, the device may determine whether to generate a virtual stop line based on the position of the pedestrian within the region of interest.

[0244] In one embodiment, if the first region corresponds to an interior region of a target crosswalk and the second region corresponds to an intersection of the exterior region of the target crosswalk and the interior region of the region of interest, the device can determine to generate a virtual stop line in response to determining that the pedestrian is located in the first region, and can determine whether to generate a virtual stop line based on a distance to the target crosswalk in response to determining that the pedestrian is located in the second region.

[0245] In one embodiment, the device may determine to generate a virtual stop line if the distance to the target crosswalk is greater than or equal to a predetermined distance, and may determine not to generate a virtual stop line if the distance to the target crosswalk is less than the predetermined distance.

[0246] In one embodiment, the device may calculate a cost function for the pedestrian, compare the cost function to a threshold, and determine whether to generate a virtual stop line based on the result of the comparison.

[0247] In one embodiment, the cost function may be calculated based on one or more of the vertical distance and vertical speed of the pedestrian relative to the traveled path.

[0248] FIG. 26 is a block diagram of an apparatus for generating a virtual stop line according to one embodiment.

[0249] In one embodiment, the device that generates the virtual stop line may refer to the same device as the trip planning device described above.

[0250] 28, a device (hereinafter, referred to as "device") 2800 for generating a virtual stop line may include a communication unit 2810, a processor 2820, and a DB 2830. Only components related to the embodiment are shown in the device 2800 in Fig. 28. Therefore, a person having ordinary skill in the art would understand that the device 2800 may further include other general-purpose components in addition to the components shown in Fig. 28.

[0251] The communication unit 2810 may include one or more components that enable wired / wireless communication with an external server or device. For example, the communication unit 2810 may include at least one of a short-range communication unit (not shown), a mobile communication unit (not shown), and a broadcast receiving unit (not shown). In one embodiment, the communication unit 2810 may be used to receive traffic information and recognize crosswalks on a driving route.

[0252] The DB 2830 is hardware that stores various data processed within the device 2800, and can store programs for processing and controlling the processor 2820.

[0253] DB2830 may include dynamic random access memory (DRAM), random access memory (RAM), such as static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), CD-ROM, Blu-ray or other optical disk storage, hard disk drive (HDD), solid state drive (SSD), or flash memory.

[0254] The processor 2820 controls the overall operation of the device 2800. For example, the processor 2820 may generally control an input unit (not shown), a display (not shown), a communication unit 2810, a DB 2830, etc. by executing a program stored in a DB 2830. The processor 2820 may generally control the operation of the device 2800 by executing a program stored in a DB 2830.

[0255] The processor 2820 can control at least a part of the operation of the autonomous driving device and / or the driving planner described above in FIGS.

[0256] The processor 2820 may be implemented using at least one of ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, and other electrical units for performing functions.

[0257] In one embodiment, the device 2800 may be a mobile electronic device. For example, the device 2800 may be a smartphone, a tablet PC, a PC, a smart TV, a PDA (personal digital assistant), a laptop, a media player, a navigation device, a camera-equipped device, and other mobile electronic devices. The device 2800 may also be a wearable device, such as a watch, glasses, a headband, or a ring, that has a communication function and a data processing function.

[0258] In another embodiment, the device 2800 may be an electronic device that is integrated into a vehicle. For example, the device 2800 may be an electronic device that is inserted into a vehicle after the manufacturing process for tuning. In that case, the location of the device 2800 and the vehicle (or autonomous vehicle) described above may be the same.

[0259] In yet another embodiment, the device 2800 may be a server located outside the vehicle. The server may be realized as a computer device or multiple computer devices that communicate over a network to provide instructions, code, files, content, services, etc. The server may receive data necessary to generate the virtual stop line from a device installed in the vehicle and generate the virtual stop line based on the received data.

[0260] In yet other embodiments, the processes performed by apparatus 2800 may be performed at least in part by a mobile electronic device, an electronic device integrated into a vehicle, and a server located outside the vehicle.

[0261] The embodiments according to the present invention may be realized in the form of a computer program executable by various components on a computer, and such a computer program may be recorded on a computer readable medium, including magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions, such as ROMs, RAMs, flash memories, and the like.

[0262] On the other hand, the computer program may be one specially designed and constructed for the present invention, or it may be one that is well known and available to those skilled in the art of computer software. Examples of computer programs include not only machine code, such as that produced by a compiler, but also high-level language code that is executed by a computer using an interpreter or the like.

[0263] According to one embodiment, the methods according to various embodiments of the present disclosure may be provided in a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read only memory (CD-ROM)) or may be distributed through an application store (e.g., the Play Store, etc.). TM ) or directly between two user devices. In the case of online distribution, at least a part of the computer program product may be at least temporarily stored or temporarily generated on a machine-readable storage medium, such as a memory of a manufacturer's server, an application store's server, or an intermediary server.

[0264] Unless there is an explicit order description or a contrary description regarding the steps constituting the method according to the present invention, the steps can be performed in any suitable order. The present invention is not necessarily limited to the order of the steps described above. The use of all examples or exemplary terms (such as, for example, etc.) in the present invention is merely for the purpose of explaining the present invention in detail, and the scope of the present invention is not limited by the examples or exemplary terms unless limited by the claims. In addition, those skilled in the art will understand that various modifications, combinations and changes can be made within the scope of the claims or their equivalents according to design conditions and factors.

[0265] Therefore, the concept of the present invention should not be limited to the above-described embodiments, and all scopes equivalent to or modified equivalently to the scope of the attached claims, as well as the scope of the present invention, fall within the scope of the concept of the present invention.

Claims

1. 1. A method for generating a virtual stop line, comprising: determining a target crosswalk from among one or more crosswalks on the travel route; A step of setting an area of ​​interest, which is a partial area in the travel route including the target crosswalk, for the target crosswalk; determining whether to generate a virtual stop line based on a position of a pedestrian within the area of ​​interest; The step of determining whether to generate a virtual stop line comprises: determining, in response to determining that the pedestrian is located in a first area, to generate the virtual stop line; in response to determining that the pedestrian is located in a second area, determining whether to generate the virtual stop line based on a distance to the target pedestrian crossing; The virtual stop line is information for controlling an autonomous vehicle. method.

2. The step of determining the target crosswalk includes: recognizing the one or more pedestrian crossings; and determining a closest crosswalk ahead on the travel path as the target crosswalk.

3. The step of setting the region of interest includes: setting a margin of the region of interest using the target crosswalk; and setting the region of interest using a margin of the region of interest.

4. the margins of the region of interest include a vertical margin and a horizontal margin; The step of setting a margin of the region of interest includes: Obtaining a distance to the target pedestrian crossing; Obtaining a width and a crossing length of the target crosswalk; setting the vertical margin based on the width and a distance to the target crosswalk; and setting the lateral margin based on the crossing length and a distance to the target crosswalk.

5. The step of setting a margin of the region of interest includes:

5. The method according to claim 3, further comprising the step of setting a larger margin of the region of interest the farther the distance to the target crosswalk is, and setting a smaller margin of the region of interest the closer the distance to the target crosswalk is.

6. The method of claim 1, wherein the first region corresponds to an interior region of the target crosswalk, and the second region corresponds to an intersection of an exterior region of the target crosswalk and an interior region of the area of ​​interest.

7. The step of determining whether to generate a virtual stop line comprises: determining that the virtual stop line is to be generated when the distance to the target pedestrian crossing is equal to or greater than a predetermined distance; The method of claim 1 , comprising determining not to generate the virtual stop line if the distance to the target crosswalk is less than the predetermined distance.

8. The step of determining whether to generate a virtual stop line comprises: calculating a cost function for the pedestrian; comparing the cost function to a threshold; and determining whether to generate the virtual stop line based on a result of the comparison.

9. The cost function is: The method of claim 8 , wherein the distance is calculated based on one or more of a vertical distance and a vertical speed of the pedestrian relative to the traveled path.

10. In a device for generating a virtual stop line, a memory having at least one program stored therein; a processor that operates by executing the at least one program; The processor, determining a target crosswalk from among one or more crosswalks on the travel route; A region of interest is set for the target crosswalk, the region being a partial region in the travel route that includes the target crosswalk; determining whether to generate a virtual stop line based on a position of a pedestrian within the region of interest; The processor further comprises: determining, in response to determining that the pedestrian is located in a first area, to generate the virtual stop line; in response to determining that the pedestrian is located in a second area, determining whether to generate the virtual stop line based on a distance to the target pedestrian crossing; The virtual stop line is information for controlling an autonomous vehicle. Device.

11. A computer-readable recording medium having recorded thereon a program for executing the method according to claim 1 on a computer.

Images