Vehicle control device, vehicle control method, and program

Abstract

To provide a vehicle control device capable of controlling a vehicle with a small load based on an image obtained by a camera.SOLUTION: The vehicle control device generates a travel potential field based on the input image, generates a target trajectory based on a gradient of the travel potential, and operates an electric power steering device based on the target trajectory. The vehicle control device generates a travel potential field by executing a process of setting a low potential point at a position determined based on reference subject information in a central region of an input image, a process of setting a first high potential point in a region in which an obstacle appears, a process of setting a value of a travel potential at the low potential point to a first set value, a process of setting a value of a travel potential at the first high potential point to a second set value, and a process of interpolating a value of a travel potential in a region between the low potential point and the first high potential point in the input image.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to a vehicle control device, a vehicle control method, and a program, and more particularly to a vehicle control device, a vehicle control method, and a program that control a vehicle based on an image captured by a camera. [Background technology]

[0002] In recent years, efforts to provide access to sustainable transport systems that take into consideration vulnerable transport participants have become more active. To achieve this, we are focusing on research and development into preventive safety technologies to further improve road safety and convenience.

[0003] For example, Patent Document 1 describes a preventive safety technology that controls the running of a vehicle based on the so-called potential method. Here, the potential method refers to a method of defining a potential function (hereinafter also referred to as a "potential field") according to obstacles and the like present around the vehicle, and generating a target trajectory for the vehicle according to the gradient of this potential function. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2018 / 131090 Summary of the Invention [Problem to be solved by the invention]

[0005] Incidentally, the vehicle control device disclosed in Patent Document 1 recognizes the shape of the road on which the vehicle is traveling and the conditions of surrounding objects by using multiple external sensors such as cameras, radar devices, and lidar devices, as well as map information, and determines the shape of the potential function based on the recognition results. Therefore, the on-board computer that generates the potential function and target trajectory tends to be heavily loaded by integrating the outputs of multiple external sensors and map information.

[0006] The present invention aims to provide a vehicle control device, a vehicle control method, and a program that can control a vehicle with little load based on images obtained by a camera, and ultimately to contribute to the development of a sustainable transportation system. [Means for solving the problem]

[0007] (1) A vehicle control device according to the present invention (for example, a vehicle control device 1 described later) includes input image acquisition means (for example, an input image acquisition unit 2 described later) that acquires, as an input image, an image captured by a camera (for example, an on-board camera C described later) facing forward as viewed from a host vehicle (for example, a vehicle V described later), traveling potential field generation means (for example, a traveling potential field generation unit 3 described later) that generates a traveling potential field indicating a distribution of traveling potential with respect to a future traveling position of the host vehicle based on the input image, target trajectory generation means (for example, a target trajectory generation unit 4 described later) that generates a target trajectory of the host vehicle based on a gradient of the traveling potential in the traveling potential field, and traveling control means (for example, a traveling control unit 5 described later) that operates a steering mechanism (for example, an electric power steering device 9 described later) based on the target trajectory, and the traveling potential field generation means derives a position of a reference subject or The driving potential field is generated by executing the following processes: extracting reference subject information related to the position of the boundary line of the reference subject; setting a low potential point at a position that is within a central region (e.g., a central region CC described below) of the input image and that is determined based on the reference subject information; identifying the position within the input image of an obstacle that hinders safe driving of the vehicle; setting a first high potential point in a region of the input image in which the obstacle is captured; setting the value of the driving potential at the low potential point to a first set value; setting the value of the driving potential at the first high potential point to a second set value that is larger than the first set value; and interpolating the value of the driving potential in a region of the input image between the low potential point and the first high potential point with a value between the first set value and the second set value.

[0008] (2) In this case, the traveling potential field generation means acquires the position of a fitting line (e.g., a left empty boundary line Fa1 described later) of a left empty boundary line (e.g., a left empty boundary line La1 described later) in the input image and the position of a fitting line (e.g., a right empty boundary line Fa2 described later) of a right empty boundary line (e.g., a right empty boundary line La2 described later) as the reference subject information, and if an intersection (e.g., an intersection Pa described later) between the fitting line of the left empty boundary line and the fitting line of the right empty boundary line exists within the central region, it is preferable to set the low potential point at the intersection.

[0009] (3) In this case, the traveling potential field generation means acquires the position of a fitting line for the left road boundary line (e.g., the left road fitting line Fb1 described below) and the position of a fitting line for the right road boundary line (e.g., the right road fitting line Fb2 described below) in the input image as the reference subject information, and if an intersection (e.g., the intersection Pb described below) between the fitting line for the left road boundary line and the fitting line for the right road boundary line exists within the central region, it is preferable to set the low potential point at the intersection.

[0010] (4) In this case, it is preferable that the traveling potential field generation means acquires the position of the road boundary line in the input image as the reference subject information, and if the uppermost end point of the road boundary line in the input image (for example, the uppermost end point Pc described below) is within the central region, set the low potential point at the uppermost end point.

[0011] (5) In this case, it is preferable that the traveling potential field generation means acquires, as the reference subject information, the position of the left empty fitting line of the left empty boundary line in the input image, the position of the right empty fitting line of the right empty boundary line in the input image, the position of the left road fitting line of the left road boundary line in the input image, the position of the right road fitting line of the right road boundary line in the input image, and the position of the road boundary line in the input image, and sets the low potential point at a position determined based on two or more points that exist within the central region among the intersection of the left empty fitting line and the right empty fitting line, the intersection of the left road fitting line and the right road fitting line, and the uppermost end point of the road boundary line in the input image.

[0012] (6) In this case, if a leading vehicle recognized as a target to be followed is present within the central region, the traveling potential field generation means preferably acquires the position of the leading vehicle in the input image as the reference subject information.

[0013] (7) In this case, when the host vehicle is traveling on a road where the sky is blocked by an overhead structure, it is preferable that the traveling potential field generation means acquires, as the reference subject information, the position of a road boundary line or the position of a preceding vehicle that is recognized as a target to be followed.

[0014] (8) In this case, it is preferable that the traveling potential field generating means sets the value of the traveling potential inside a low potential range centered on the low potential point so that the gradient becomes steeper as the low potential point is approached, sets the value of the traveling potential inside a first high potential range centered on the first high potential point so that the gradient becomes steeper as the first high potential point is approached, and sets the value of the traveling potential outside the low potential range and the first high potential range so that the gradient is constant and gentler than inside the low potential range and the first high potential range.

[0015] (9) In this case, it is preferable that the driving potential field generation means generates the driving potential field by further executing the following processes: a process of identifying the position of the lane marking of the vehicle in the input image; a process of setting a second high potential point in an area of ​​the input image where the lane marking is captured; a process of setting the value of the driving potential at the second high potential point to a third set value that is larger than the first set value; and a process of interpolating the value of the driving potential between the low potential point and the second high potential point in the input image with a value between the first set value and the third set value.

[0016] (10) In this case, it is preferable that the traveling potential field generating means does not set the second high potential point when the host vehicle is currently changing lanes or is scheduled to change lanes. [Effects of the Invention]

[0017] (1) In the present invention, the input image acquisition means acquires as an input image an image captured by a camera facing forward as viewed from the host vehicle, the traveling potential field generation means generates a traveling potential field indicating a distribution of traveling potential for a future traveling position of the host vehicle based on the input image, the target trajectory generation means generates a target trajectory for the host vehicle based on a gradient of the traveling potential in the traveling potential field, and the traveling control means operates the steering mechanism based on the target trajectory. Also in the present invention, the traveling potential field generation means extracts reference object information related to a predetermined reference object from the input image, sets a low potential point within a central region of the input image at a position determined based on the reference object information, identifies the position of an obstacle in the input image, and further sets a first high potential point in a region of the input image where the obstacle appears. The traveling potential field generation means generates a traveling potential field for the input image by setting the traveling potential value at the low potential point to a first set value, setting the traveling potential value at the first high potential point to a second set value greater than the first set value, and interpolating the traveling potential value in the region of the input image between the low potential point and the first high potential point using values ​​between the first and second set values. In this way, according to the present invention, a traveling potential field can be generated and a target trajectory can be generated only based on the positions of reference subjects, obstacles, etc. that appear in the input image, without using external sensors other than the camera, map information, etc., thereby enabling the host vehicle to be controlled with little load and ultimately contributing to the development of sustainable transportation systems.

[0018] (2) In the present invention, the traveling potential field generation means acquires, as reference subject information, the position of a fitting line (hereinafter also referred to as the "left sky fitting line") for the left sky boundary line (i.e., the boundary line between the sky on the left side as seen from the host vehicle and ground objects) in the input image and the position of a fitting line (hereinafter also referred to as the "right sky fitting line") for the right sky boundary line (i.e., the boundary line between the sky on the right side as seen from the host vehicle and ground objects). Furthermore, if the intersection of the left sky fitting line and the right sky fitting line is within the central region, the traveling potential field generation means sets a low-potential point, which is the end point of the target route, at this intersection. Therefore, according to the present invention, a traveling potential field can be generated by simple calculations on the input image.

[0019] (3) In the present invention, the traveling potential field generation means acquires, as reference subject information, the position of a fitting line (hereinafter also referred to as the "left road fitting line") for the left road boundary line (i.e., the edge line of the road on which the host vehicle is traveling on the left side as seen from the host vehicle) in the input image and the position of a fitting line (hereinafter also referred to as the "right road fitting line") for the right road boundary line (i.e., the edge line of the road on which the host vehicle is traveling on the right side as seen from the host vehicle). Furthermore, if the intersection of the left road fitting line and the right road fitting line is within the central region, the traveling potential field generation means sets a low-potential point, which is the end point of the target route, at this intersection. Thus, according to the present invention, a traveling potential field can be generated by simple calculations on the input image.

[0020] (4) In the present invention, the traveling potential field generation means acquires the position of the road boundary line in the input image (i.e., the line joining the left road boundary line and the right road boundary line) as reference subject information. Furthermore, if the uppermost end point of the road boundary line in the input image is within the central region, the traveling potential field generation means sets a low potential point, which is the end point of the target route, at this uppermost end point. Therefore, according to the present invention, a traveling potential field can be generated by simple calculations on the input image.

[0021] (5) In the present invention, the traveling potential field generation means acquires the positions of the left empty fitting line, the right empty fitting line, the left road fitting line, the right road fitting line, and the road boundary line in the input image as reference subject information. The traveling potential field generation means also sets a low potential point, which is the end of the target route, at a position determined based on two or more points within the central region among the intersection of the left empty fitting line and the right empty fitting line, the intersection of the left road fitting line and the right road fitting line, and the uppermost point of the road boundary line in the input image. Therefore, according to the present invention, a traveling potential field can be generated by simple calculations on the input image.

[0022] (6) In the present invention, when a preceding vehicle recognized as a target to be followed is present in the central region, the traveling potential field generation means acquires the position of the preceding vehicle in the input image as reference subject information, and sets a low-potential point that is the end of the target route at a position determined based on the position of the preceding vehicle. Thus, according to the present invention, a traveling potential field that automatically follows the preceding vehicle can be generated by simple calculations on the input image.

[0023] (7) In the present invention, when the host vehicle is traveling on a road where the sky is blocked by overhead structures, such as in a tunnel, that is, when the input image barely captures the sky, the traveling potential field generation means acquires the position of the road boundary line or the position of the preceding vehicle recognized as the target to be followed as reference subject information, and sets the position of the low potential point based on the position of the road boundary line or the position of the preceding vehicle. Thus, according to the present invention, even when the input image does not capture enough sky, it is possible to set the low potential point in an appropriate position.

[0024] (8) In the present invention, the traveling potential field setting means sets the value of the traveling potential inside a low potential range centered on the low potential point so that the gradient becomes steeper toward the center, sets the value of the traveling potential inside a first high potential range centered on the first high potential point so that the gradient becomes steeper toward the center, and sets the value of the traveling potential outside the low potential range and the first high potential range so that the gradient is constant and gentler than inside the low potential range and the first high potential range. According to the present invention, by generating a traveling potential field using the above procedure, it is possible to generate, by simple calculations, a traveling potential field that avoids the first high potential point where an obstacle exists and generates a target route that ends at the low potential point.

[0025] (9) In the present invention, the traveling potential field generation means sets a second high potential point in an area of ​​the input image where the lane markings of the vehicle are captured. The traveling potential field generation means also generates a traveling potential field by setting the value of the traveling potential at the second high potential point to a third set value greater than the first set value, and interpolating the value of the traveling potential between the low potential point and the second high potential point in the input image with values ​​between the first set value and the third set value. Therefore, according to the present invention, by generating a traveling potential field using the above-described procedure, it is possible to generate, through simple calculations, a traveling potential field that avoids first high potential points where obstacles exist and second high potential points where lane markings exist and generates a target route that ends at a low potential point.

[0026] (10) As described above, if a second high potential point is set at a position where a lane marking exists for the host vehicle, a target route that avoids the lane marking is generated. Therefore, in the present invention, the traveling potential field generation means does not set a second high potential point as described above when the host vehicle is currently changing lanes or is planning to change lanes. Therefore, according to the present invention, a traveling potential field that generates a target route that crosses the lane marking can be generated by simple calculations. [Brief explanation of the drawings]

[0027] [Figure 1] 1 is a diagram schematically illustrating the configuration of a vehicle equipped with a vehicle control device according to an embodiment of the present invention. [Figure 2] FIG. 2 is a functional block diagram of a vehicle control device. [Figure 3] FIG. 2 is a diagram schematically illustrating a traveling potential field generated by a traveling potential field generating unit. [Figure 4] 10 is a flowchart showing a specific procedure for generating a traveling potential field. [Figure 5] FIG. 10 is a diagram for explaining the procedure of a low-potential point search process. [Figure 6] FIG. 10 is a diagram for explaining the procedure of a high-potential point search process. [Figure 7] FIG. 10 is a diagram showing an example of setting the value of the traveling potential in a portion including a first high potential point and a low potential point. [Figure 8] FIG. 10 is a diagram showing an example of a traveling potential field generated by the traveling potential field generation process. DETAILED DESCRIPTION OF THE INVENTION

[0028] A vehicle control device according to an embodiment of the present invention will be described below with reference to the drawings.

[0029] FIG. 1 is a diagram schematically illustrating the configuration of a vehicle V equipped with a vehicle control device 1 according to this embodiment. The upper part of FIG. 1 shows a plan view of the vehicle V, and the lower part of FIG. 1 shows a side view. Note that the following description will be given assuming that the vehicle V is a so-called right-hand drive four-wheel vehicle in which the driver's seat is located on the right side in the vehicle width direction as viewed along the direction of travel, but the present invention is not limited to this. The vehicle V may also be a so-called left-hand drive four-wheel vehicle in which the driver's seat is located on the left side in the vehicle width direction as viewed along the direction of travel.

[0030] The vehicle V is equipped with an electric power steering device 9 as a steering mechanism that steers the left and right front wheels Wf, a power plant 8 as a driving device that generates a driving force to rotate the front wheels Wf, which are the driving wheels of the vehicle V, a braking device 7 that generates a braking force to stop the rotation of the front wheels Wf and rear wheels Wr, an on-board camera C that captures images of the surroundings of the vehicle V, and a vehicle control device 1 that controls the electric power steering device 9, the power plant 8, and the braking device 7 based on the images captured by the on-board camera C.

[0031] The electric power steering device 9 includes a gearbox 93 that connects a pinion shaft 92 extending from a steering wheel 91 that receives steering operations by the driver to the left and right front wheels Wf, an electric motor 94 provided in the gearbox 93, and a steering sensor 95 that detects the steering amount of the steering wheel 91.

[0032] The gearbox 93 includes a rack shaft extending along the vehicle width direction and meshing with the pinion shaft 92, tie rods connecting both ends of the rack shaft to the left and right front wheels Wf, and converts the rotational movement of the steering wheel 91 caused by the driver's steering operation into movement along the vehicle width direction, thereby steering the left and right front wheels Wf in the direction of travel. The electric motor 94 rotates in response to a control signal output from the vehicle control device 1, and generates a driving force to assist the driver's steering operation or to automatically steer the front wheels Wf without the driver's steering operation. The steering sensor 95 detects the steering amount of the steering wheel 91 and sends a signal corresponding to the detected value to the vehicle control device 1.

[0033] The power plant 8 is a driving force generating source that generates a driving force for rotating the front wheels Wf to move the vehicle V forward or backward along the traveling direction in response to acceleration / deceleration operation of an accelerator pedal (not shown) by the driver and control signals output from the vehicle control device 1. In the following, a case will be described in which the power plant 8 is a drive motor that generates driving force by consuming electric power supplied from a high-voltage battery, a fuel cell stack, or the like (not shown), but the present invention is not limited to this. The power plant 8 may also be an engine that generates driving force by consuming fuel stored in a fuel tank (not shown), or a transmission that changes the speed of the engine output and transmits it to the front wheels Wf.

[0034] The braking device 7 includes a disc brake device that generates braking force to slow down or stop the rotation of each wheel Wf, Wr by tightening discs provided on the axles of each wheel Wf, Wr mainly when driving, based on braking operation of the brake pedal (not shown) by the driver or control signals output from the vehicle control device 1, and a parking brake that generates braking force to keep the rotation of each wheel Wr, Wf stopped mainly when parking.

[0035] The vehicle-mounted camera C is directed forward along the traveling direction as viewed from the vehicle V. In this embodiment, the vehicle-mounted camera C is installed in the center of the body of the vehicle V in the vehicle width direction, but the present invention is not limited to this.

[0036] The vehicle control device 1 controls the electric power steering device 9, the power plant 8, and the braking device 7 based on an image of the front side of the vehicle V captured by an on-board camera C. The vehicle control device 1 is a computer configured with hardware such as a processing means such as a CPU, auxiliary storage means such as an HDD or SSD that stores a program that causes the processing means to execute a traveling potential field generation process (described below), and main storage means such as a RAM for storing data temporarily required for the processing means to execute the program.

[0037] 2 is a functional block diagram of the vehicle control device 1. The vehicle control device 1 has the above-described hardware configuration, which includes an input image acquisition unit 2, a traveling potential field generation unit 3, a target trajectory generation unit 4, and a traveling control unit 5.

[0038] The input image acquisition unit 2 acquires, as an input image, an image of the front side of the vehicle V captured by the on-board camera C. The input image acquisition unit 2 transmits information about the acquired input image to the traveling potential field generation unit 3.

[0039] The traveling potential field generation unit 3 generates a traveling potential field that indicates the distribution of the traveling potential on the input image with respect to the future traveling position of the vehicle V (i.e., the distribution of the traveling potential on the two-dimensional image coordinates in which the input image is defined) based on the input image transmitted from the input image acquisition unit 2. The traveling potential field generation unit 3 transmits information about the generated traveling potential field to the target trajectory generation unit 4.

[0040] FIG. 3 is a diagram schematically illustrating the traveling potential field generated by the traveling potential field generation unit 3. In FIG. 3, the traveling potential values ​​are illustrated by different colors against the input image as the background. More specifically, the larger the traveling potential value, the darker the color. Also in FIG. 3, the point where the traveling potential value is smallest (potential minimum point) is indicated by a white circle. As shown in FIG. 3, the traveling potential field is a scalar function of the traveling potential defined on two-dimensional image coordinates. The specific procedure for generating the traveling potential field by the traveling potential field generation unit 3 will be described later with reference to FIGS. 4 to 8, etc.

[0041] Returning to FIG. 2, the target trajectory generation unit 4 calculates the gradient of the traveling potential field generated by the traveling potential field generation unit 3, and generates a target trajectory on the image coordinates of the vehicle V based on the gradient of this traveling potential field and the current steering angle of the vehicle V, etc. The target trajectory generation unit 4 transmits information related to the generated target trajectory to the traveling control unit 5. Here, the gradient of the traveling potential field is a vector function equivalent to the partial differential of each image coordinate component of the traveling potential field defined on two-dimensional image coordinates. As a result, the target trajectory generation unit 4 generates a target trajectory (see the thick dashed line in FIG. 3) that starts at the front of the vehicle V and travels along the valley of the traveling potential to the potential minimum point.

[0042] The driving control unit 5 operates the electric power steering device 9, the power plant 8, and the braking device 7 based on the target trajectory generated by the target trajectory generating unit 4. More specifically, the driving control unit 5 operates the electric power steering device 9, the power plant 8, and the braking device 7 so that the vehicle V travels along the target trajectory defined on the image coordinates.

[0043] 4 is a flowchart showing the specific steps of the traveling potential field generation process. The traveling potential field generation process shown in FIG. 4 is repeatedly executed by the traveling potential field generator 3 under a predetermined control cycle while the vehicle V is traveling.

[0044] First, in step ST1, the traveling potential field generator 3 acquires an input image transmitted from the input image acquirer 2, and then the process proceeds to step ST2.

[0045] Next, in step ST2, the traveling potential field generator 3 performs a segmentation process on the input image acquired in step ST1 to classify the subjects appearing in the input image into multiple classes, and generates an image in which the boundaries of each class are extracted (hereinafter also referred to as an "edge image"), and then proceeds to step ST3.

[0046] Next, in step ST3, the traveling potential field generator 3 executes a low-potential point search process to search for the position of a low-potential point on the image coordinate system based on the edge image extracted from the input image and information about each class shown in the input image (hereinafter also referred to as "class information"), and then proceeds to step ST4. Here, a low-potential point refers to a point on the image coordinate system where the traveling potential value is the minimum value, and is the end point of the target trajectory as shown in FIG. 3. The specific steps of the low-potential point search process will be described in detail below with reference to FIG. 5.

[0047] FIG. 5 is a diagram showing an example of an input image and is a diagram for explaining the procedure of the low-potential point search process. First, the traveling potential field generation unit 3 extracts, from the edge image, class information, etc., information about the position on the image coordinates of a reference object defined for setting the position of a low-potential point or the position of the boundary line of this reference object as reference object information. Next, the traveling potential field generation unit 3 sets a low-potential point at a position that is within a central area CC defined to include the center of the image coordinates and that is determined based on the extracted reference object information. Several examples of the low-potential point search process are described below.

[0048] Example 1 In the first embodiment, the traveling potential field generation unit 3 sets the position of the low potential point using a subject classified as a "sky" class among multiple subjects captured in the input image as a reference subject. In this case, the traveling potential field generation unit 3 acquires the position of a fitting line Fa1 of the left sky boundary line La1 and the position of a fitting line Fa2 of the right sky boundary line La2 as reference subject information from the edge image, class information, etc. Here, the left sky boundary line La1 refers to the boundary line between the sky and ground objects on the left side as viewed from the vehicle V in the input image, and the right sky boundary line La2 refers to the boundary line between the sky and ground objects on the right side as viewed from the vehicle V in the input image. The fitting line Fa1 of the left sky boundary line (hereinafter also referred to as the "left sky fitting line") refers to a line obtained by fitting the left sky boundary line La1 based on a known algorithm, for example, using a linear function, and the fitting line Fa2 of the right sky boundary line (hereinafter also referred to as the "right sky fitting line") refers to a line obtained by fitting the right sky boundary line La2 based on a known algorithm, for example, using a linear function.

[0049] Furthermore, if the intersection Pa between the left empty fitting line Fa1 and the right empty fitting line Fa2 obtained as described above is within the predetermined central region CC, the traveling potential field generation unit 3 sets a low potential point at this intersection Pa. Furthermore, if the intersection Pa is not within the central region CC, the traveling potential field generation unit 3 sets the position of the low potential point based on other embodiments 2 to 5, etc.

[0050] When the vehicle V is traveling on a road where the sky is blocked by an overhead structure (for example, when the vehicle V is traveling in a tunnel), the left empty fitting line Fa1 and the right empty fitting line Fa2 cannot be acquired. Therefore, even in such a case, the traveling potential field generation unit 3 sets the positions of the low potential points based on the other embodiments 2 to 5.

[0051] <Example 2> In Example 2, the traveling potential field generation unit 3 sets the position of a low potential point using, as a reference object, an object classified into the "road" class from among multiple objects captured in the input image. In this case, the traveling potential field generation unit 3 acquires, as reference object information, the position of a fitting line Fb1 for the left road boundary line and the position of a fitting line Fb2 for the right road boundary line from the edge image, class information, etc. Here, the left road boundary line refers to the edge line on the left side of the road on which the vehicle V is traveling as seen from the vehicle V in the input image, and the right road boundary line refers to the edge line on the right side of the road on which the vehicle V is traveling as seen from the vehicle V in the input image. Furthermore, the fitting line Fb1 for the left road boundary line (hereinafter also referred to as the "left road fitting line") refers to a line obtained by fitting the left road boundary line based on a known algorithm, for example, using a linear function, and the fitting line Fb2 for the right road boundary line (hereinafter also referred to as the "right road fitting line") refers to a line obtained by fitting the right road boundary line based on a known algorithm, for example, using a linear function. In the example of FIG. 4, the left and right road boundary lines and their fitting lines Fb1, Fb2 almost overlap, so only the fitting lines Fb1, Fb2 are shown.

[0052] Furthermore, if the intersection Pb between the left road fitting line Fb1 and the right road fitting line Fb2 obtained as described above is within the central area CC, the traveling potential field generation unit 3 sets a low potential point at this intersection Pb. Furthermore, if the intersection Pb is not within the central area CC, the traveling potential field generation unit 3 sets the position of the low potential point based on other embodiments 1, 3 to 5, etc.

[0053] Example 3 In the third embodiment, the traveling potential field generator 3 sets the position of the low potential point using, as a reference object, an object classified into the "road" class from among multiple objects captured in the input image, as in the second embodiment. In this case, the traveling potential field generator 3 acquires the position of the road boundary line as reference object information from the edge image, class information, etc. Here, the road boundary line refers to the line joining the left road boundary line and the right road boundary line described above.

[0054] Furthermore, if the uppermost endpoint Pc in the input image of the road boundary line obtained as described above is within the central region CC, the traveling potential field generation unit 3 sets a low potential point at this uppermost endpoint Pc. Furthermore, if the uppermost endpoint Pc is not within the central region CC, the traveling potential field generation unit 3 sets the position of the low potential point based on other embodiments 1-2, 4-5, etc.

[0055] Example 4 In the fourth embodiment, the traveling potential field generator 3 generates the traveling potential field of a plurality of subjects appearing in the input image. Of these, a subject classified into the "sky" or "road" class is used as the reference subject to set the positions of low potential points. In this case, the traveling potential field generator 3 acquires the positions of the left sky fitting line Fa1, the right sky fitting line Fa2, the left road fitting line Fb1, the right road fitting line Fb2, and the road boundary line as reference subject information from the edge image, class information, etc.

[0056] The traveling potential field generator 3 also sets a low potential point at a position determined based on two or more of the intersection Pa between the left empty fitting line Fa1 and the right empty fitting line Fa2, the intersection Pb between the left road fitting line Fb1 and the right road fitting line Fb2, and the uppermost endpoint Pc of the road boundary line, all of which are within the central region CC. More specifically, the traveling potential field generator 3 sets a low potential point at the geometric center of gravity of two or more of the three points Pa, Pb, and Pc that are within the central region CC.

[0057] <Example 5> In Example 5, the traveling potential field generation unit 3 sets the position of the low potential point using, as a reference object, an object classified as a "leading vehicle" from among multiple objects captured in the input image. In this case, the traveling potential field generation unit 3 acquires the position of the leading vehicle (more specifically, the position of the center point Pd of the leading vehicle) as reference object information from the edge image, class information, etc. Here, the leading vehicle refers to a vehicle that is traveling in the same lane as vehicle V and ahead of vehicle V, traveling in the same direction as vehicle V.

[0058] Furthermore, if the preceding vehicle extracted as described above is recognized as a target to be followed by vehicle V through processing not shown, and if the center point Pd of the preceding vehicle is present within central region CC, traveling potential field generation unit 3 sets a low potential point at the center point Pd of the preceding vehicle. Furthermore, if there is no preceding vehicle, if the preceding vehicle is not recognized as a target to be followed, or if the center point Pd of the preceding vehicle is not present within central region CC, traveling potential field generation unit 3 sets the position of the low potential point based on other embodiments 1 to 4, etc.

[0059] Returning to FIG. 4, in step ST4, the traveling potential field generator 3 executes a high-potential point search process to search for the position of a high-potential point on the image coordinate system based on the edge image extracted from the input image, class information, etc., and then proceeds to step ST5. Here, a high-potential point refers to a point on the image coordinate system where the traveling potential value is a maximum value. The specific steps of the high-potential point search process will be described in detail below with reference to FIG. 6.

[0060] FIG. 6 is a diagram showing an example of an input image and is a diagram for explaining the procedure of the high-potential point search process. First, the traveling potential field generation unit 3 identifies the position in image coordinates of an obstacle that obstructs the safe traveling of the vehicle V from the edge image, class information, etc. Here, obstacles that obstruct the safe traveling of the vehicle V include, for example, other vehicles excluding the preceding vehicle, curbs, and roadside trees. Next, the traveling potential field generation unit 3 sets a first high-potential point in the area in the image coordinates where the obstacle identified by the above procedure is captured, as indicated by a circle in FIG. 6.

[0061] The traveling potential field generation unit 3 also identifies the position of the lane markings of the vehicle V in the image coordinates from the edge image, class information, etc. Next, the traveling potential field generation unit 3 sets a second high potential point in the area in the image coordinates where the lane markings identified by the above procedure are captured, as indicated by the square marks in FIG.

[0062] Unlike obstacles set as first high potential points, these lane markings do not themselves pose an obstacle to the safe driving of vehicle V. However, if vehicle V crosses a lane marking, it may threaten the safe driving of other vehicles. Therefore, the driving potential field generation unit 3 sets a second high potential point on such lane markings. However, if a second high potential point is set on such a lane marking, vehicle V will not be able to cross the lane marking and will not be able to change lanes. Therefore, it is preferable that the driving potential field generation unit 3 not set a second high potential point when vehicle V is currently changing lanes or is planning to change lanes.

[0063] Returning to Figure 4, in step ST5, the traveling potential field generation unit 3 executes a traveling potential value setting process to set a traveling potential value on the image coordinates based on the positions of the low potential points found in step ST3 and the high potential points found in step ST4, and then ends the process in Figure 4. The specific steps of the traveling potential value setting process will be described below with reference to Figure 7.

[0064] FIG. 7 is a diagram showing an example of setting the value of the traveling potential in the portion including the first high potential point and the low potential point as indicated by the line VI-VI in FIG.

[0065] First, the traveling potential field generator 3 sets the traveling potential value at the low potential point to a predetermined first set value, sets the traveling potential value at the first high potential point to a second set value greater than the first set value, and sets the traveling potential value at the second high potential point to a third set value greater than the first set value. Below, we will describe a case where the second set value and the third set value are set to the same magnitude, but the present invention is not limited to this. The second set value and the third set value may also be set to different values.

[0066] Next, the traveling potential field generation unit 3 interpolates the value of the traveling potential in the region between the low potential point and the first high potential point in the image coordinates using a value between the first set value and the second set value, and interpolates the value of the traveling potential in the region between the low potential point and the second high potential point using a value between the first set value and the third set value.

[0067] More specifically, the traveling potential field generation unit 3 sets the value of the traveling potential inside a low potential range centered on a low potential point so that the gradient becomes steeper as the low potential point is approached, sets the value of the traveling potential inside a first high potential range centered on a first high potential point so that the gradient becomes steeper as the first high potential point is approached, and sets the value of the traveling potential inside a second high potential range centered on a second high potential point so that the gradient becomes steeper as the second high potential point is approached.

[0068] Next, the traveling potential field generation unit 3 sets the value of the traveling potential outside the low potential range, the first high potential range, and the second high potential range so that the gradient is constant and gentler than inside the low potential range, the first high potential range, and the second high potential range. The traveling potential field generation unit 3 generates a traveling potential field by setting the value of the traveling potential across all regions on the image coordinate system using the above procedure.

[0069] FIG. 8 is a diagram showing an example of a traveling potential field generated by the traveling potential field generation process described above. In FIG. 8, the gradient of the traveling potential field, which is a vector function, is indicated by an arrow. In addition, in the example of FIG. 8, for clarity of illustration, a case is shown in which only a second high potential point is set, without a first high potential point being set.

[0070] In the example shown in Fig. 8, mountain ridges of the driving potential are formed on the two driving lanes that extend on both the left and right sides of the vehicle toward the center of the input image. Therefore, in the example shown in Fig. 8, a valley of the driving potential is formed that extends between the two driving lanes and reaches the center of the input image. Therefore, under the driving potential field shown in Fig. 8, the target trajectory generating unit 4 generates a target trajectory that extends along the valley of the driving potential field, as shown by the thick dashed line in Fig. 8.

[0071] The vehicle control device 1 according to this embodiment provides the following effects. (1) The input image acquisition unit 2 acquires as an input image an image captured by an on-board camera C facing forward as viewed from the vehicle V, the traveling potential field generation unit 3 generates a traveling potential field indicating the distribution of traveling potential for the future traveling position of the vehicle V based on the input image, the target trajectory generation unit 4 generates a target trajectory for the vehicle V based on the gradient of the traveling potential in the traveling potential field, and the traveling control unit 5 operates the electric power steering device 9, the power plant 8, the braking device 7, etc. based on the target trajectory. The traveling potential field generation unit 3 also extracts reference object information related to a predetermined reference object from the input image, sets a low potential point within a central region CC of the input image at a position determined based on the reference object information, identifies the position of an obstacle in the input image, and sets a first high potential point in the region of the input image where the obstacle appears. Furthermore, the traveling potential field generator 3 sets the traveling potential value at the low potential point to a first set value, sets the traveling potential value at the first high potential point to a second set value greater than the first set value, and interpolates the traveling potential value in the region of the input image between the low potential point and the first high potential point using values ​​between the first and second set values, thereby generating a traveling potential field for the input image. As described above, according to this embodiment, a traveling potential field can be generated and a target trajectory can be generated only based on the positions of the reference subject, obstacles, etc., shown in the input image, without using any external sensors other than the on-board camera C, map information, etc., and therefore the host vehicle can be controlled with little load, which can ultimately contribute to the development of sustainable transportation systems.

[0072] (2) The traveling potential field generation unit 3 acquires the positions of the left and right empty fitting lines in the input image as reference subject information. Furthermore, if the intersection Pa between the left and right empty fitting lines is within the central region CC, the traveling potential field generation unit 3 sets a low-potential point, which is the end point of the target path, at this intersection Pa. Therefore, according to the present invention, a traveling potential field can be generated by simple calculations on the input image.

[0073] (3) The traveling potential field generation unit 3 acquires the positions of the left road fitting line and the right road fitting line in the input image as reference subject information. Furthermore, if an intersection Pb between the left road fitting line and the right road fitting line exists within the central region CC, the traveling potential field generation unit 3 sets a low-potential point, which is the end point of the target route, at this intersection Pb. Therefore, according to the present invention, a traveling potential field can be generated by simple calculations on the input image.

[0074] (4) The traveling potential field generation unit 3 acquires the position of the road boundary line in the input image as reference subject information. Furthermore, if the uppermost endpoint Pc of the road boundary line in the input image is within the central region CC, the traveling potential field generation unit 3 sets a low-potential point, which is the end point of the target route, at this uppermost endpoint Pc. Therefore, the vehicle control device 1 can generate a traveling potential field by performing simple calculations on the input image.

[0075] (5) The traveling potential field generation unit 3 acquires the positions of the left empty fitting line, the right empty fitting line, the left road fitting line, the right road fitting line, and the road boundary line in the input image as reference subject information. The traveling potential field generation unit 3 also sets a low-potential point, which is the end point of the target route, at a position determined based on two or more points within the central region CC among the intersection Pa of the left empty fitting line and the right empty fitting line, the intersection Pb of the left road fitting line and the right road fitting line, and the uppermost point Pc of the road boundary line in the input image. Therefore, the vehicle control device 1 can generate a traveling potential field by simple calculations on the input image.

[0076] (6) When a preceding vehicle recognized as a target to be followed is present in the central region CC, the traveling potential field generator 3 acquires the position of the preceding vehicle in the input image as reference subject information, and sets a low-potential point, which is the end point of the target route, at a position determined based on the position of the preceding vehicle. Thus, the vehicle control device 1 can generate a traveling potential field that automatically follows the preceding vehicle through simple calculations on the input image.

[0077] (7) When the vehicle is traveling on a road where the sky is blocked by overhead structures, such as in a tunnel, that is, when the input image barely captures the sky, the traveling potential field generator 3 acquires the position of the road boundary line or the position of the preceding vehicle recognized as the target to be followed as reference subject information, and sets the position of the low potential point based on the position of the road boundary line or the position of the preceding vehicle. Therefore, the vehicle control device 1 can set the low potential point at an appropriate position even when the input image does not capture enough sky.

[0078] (8) The traveling potential field generation unit 3 sets the value of the traveling potential inside a low potential range centered on the low potential point so that the gradient becomes steeper toward the center, sets the value of the traveling potential inside a first high potential range centered on the first high potential point so that the gradient becomes steeper toward the center, and sets the value of the traveling potential outside the low potential range and the first high potential range so that the gradient is constant and gentler than inside the low potential range and the first high potential range. By generating a traveling potential field using the above procedure, the vehicle control device 1 can generate, through simple calculations, a traveling potential field that avoids the first high potential point where an obstacle exists and generates a target route that ends at the low potential point.

[0079] (9) The traveling potential field generation unit 3 sets a second high potential point in an area of ​​the input image where the lane markings of the vehicle V are captured. The traveling potential field generation unit 3 also sets the value of the traveling potential at the second high potential point to a third set value greater than the first set value, and generates a traveling potential field by interpolating the value of the traveling potential between the low potential point and the second high potential point in the input image with a value between the first set value and the third set value. Therefore, by generating a traveling potential field using the above procedure, the vehicle control device 1 can generate, through simple calculations, a traveling potential field that avoids first high potential points where obstacles exist and second high potential points where lane markings exist and generates a target route that ends at a low potential point.

[0080] (10) As described above, if a second high potential point is set at a position where a lane marking exists for vehicle V, a target route that avoids the lane marking is generated. Therefore, in the vehicle control device 1, the traveling potential field generator 3 does not set a second high potential point as described above when vehicle V is changing lanes or planning to change lanes. Therefore, according to the present invention, a traveling potential field that generates a target route that crosses over a lane marking can be generated by simple calculations.

[0081] Although one embodiment of the present invention has been described above, the present invention is not limited to this, and the detailed configuration may be modified as appropriate within the scope of the spirit of the present invention. [Explanation of symbols]

[0082] V...Vehicle (own vehicle) C...In-vehicle camera (camera) 1...Vehicle control device (vehicle control device) 2... Input image acquisition unit (input image acquisition means) 3...Traveling potential field generating unit (traveling potential field generating means) 4...Target trajectory generation unit (target trajectory generation means) 5...Travel control unit (travel control means) 7...Brake device 8...Power plant (driving device) 9...Electric power steering device (steering mechanism)

Claims

1. an input image acquisition means for acquiring, as an input image, an image captured by a camera facing forward as viewed from the host vehicle; a traveling potential field generating means for generating a traveling potential field indicating a distribution of traveling potentials with respect to a future traveling position of the host vehicle based on the input image; a target trajectory generating means for generating a target trajectory of the host vehicle based on a gradient of the traveling potential in the traveling potential field; a driving control means for operating a steering mechanism based on the target trajectory, The traveling potential field generating means A process of extracting reference object information relating to a position of a reference object or a boundary line of the reference object from the input image; a process of setting a low potential point within a central region of the input image and at a position determined based on the reference subject information; A process of identifying a position within the input image of an obstacle that may hinder the safe running of the host vehicle; a process of setting a first high potential point in an area of ​​the input image in which the obstacle is captured; a process of setting the value of the traveling potential at the low potential point to a first set value; a process of setting the value of the traveling potential at the first high potential point to a second set value that is greater than the first set value; and interpolating the value of the driving potential in a region of the input image between the low potential point and the first high potential point with a value between the first set value and the second set value, thereby generating the driving potential field.

2. The traveling potential field generating means The position of a fitting line of a left space boundary line and the position of a fitting line of a right space boundary line in the input image are acquired as the reference object information; 2. The vehicle control device according to claim 1, wherein, when an intersection point between the fitting line of the left space boundary line and the fitting line of the right space boundary line exists within the central region, the low potential point is set at the intersection point.

3. The traveling potential field generating means acquiring the position of a fitting line of a left road boundary line and the position of a fitting line of a right road boundary line in the input image as the reference subject information; 2. The vehicle control device according to claim 1, wherein, when an intersection point between the fitting line of the left road boundary line and the fitting line of the right road boundary line exists within the central region, the low potential point is set at the intersection point.

4. The traveling potential field generating means acquiring a position of a road boundary line in the input image as the reference object information; 2. The vehicle control device according to claim 1, wherein, when the uppermost end point of the road boundary line in the input image is within the central region, the low potential point is set to the uppermost end point.

5. The traveling potential field generating means acquiring, as the reference subject information, a position of a left empty fitting line of a left empty boundary line in the input image, a position of a right empty fitting line of a right empty boundary line in the input image, a position of a left road fitting line of a left road boundary line in the input image, a position of a right road fitting line of a right road boundary line in the input image, and a position of a road boundary line in the input image; 2. The vehicle control device according to claim 1, characterized in that the low potential point is set at a position determined based on two or more points that exist within the central region, including the intersection of the left empty fitting line and the right empty fitting line, the intersection of the left road fitting line and the right road fitting line, and the uppermost end point of the road boundary line in the input image.

6. 2. The vehicle control device according to claim 1, wherein, when a preceding vehicle recognized as a follow-up target is present in the central region, the traveling potential field generation means acquires the position of the preceding vehicle in the input image as the reference subject information.

7. The traveling potential field generating means The vehicle control device described in claim 1, characterized in that when the vehicle is traveling on a road where the sky is blocked by overhead structures, the position of the road boundary line or the position of a preceding vehicle recognized as a target to be followed is obtained as the reference subject information.

8. The traveling potential field generating means Within a low potential range centered on the low potential point, the value of the traveling potential is set so that the gradient becomes steeper as the low potential point is approached; inside a first high potential range centered on the first high potential point, the value of the traveling potential is set so that the gradient becomes steeper as the value approaches the first high potential point; 2. The vehicle control device according to claim 1, wherein the value of the driving potential is set so that a gradient is constant outside the low potential range and the first high potential range and is gentler than a gradient inside the low potential range and the first high potential range.

9. The traveling potential field generating means A process of identifying a position of the lane marking of the vehicle in the input image; A process of setting a second high potential point in an area of ​​the input image in which the lane marking is captured; a process of setting the value of the traveling potential at the second high potential point to a third set value that is greater than the first set value; 2. The vehicle control device according to claim 1, further comprising: a process of interpolating values ​​of the driving potential between the low potential point and the second high potential point in the input image with values ​​between the first set value and the third set value, thereby generating the driving potential field.

10. 10. The vehicle control device according to claim 9, wherein the traveling potential field generating means does not set the second high potential point when the host vehicle is currently changing lanes or is scheduled to change lanes.

11. A vehicle control method for controlling a vehicle by a computer based on an image captured by a camera facing forward as viewed from the vehicle, comprising: acquiring an image captured by the camera as an input image; generating a traveling potential field indicating a distribution of traveling potentials with respect to a future traveling position of the host vehicle based on the input image; generating a target trajectory of the host vehicle based on a gradient of the traveling potential in the traveling potential field; and operating a steering mechanism based on the target trajectory, In the step of generating the traveling potential field, A process of extracting reference object information relating to a position of a reference object or a boundary line of the reference object from the input image; a process of setting a low potential point within a central region of the input image and at a position determined based on the reference subject information; A process of identifying a position within the input image of an obstacle that may hinder the safe running of the host vehicle; a process of setting a first high potential point in an area of ​​the input image in which the obstacle is captured; a process of setting the value of the traveling potential at the low potential point to a first set value; a process of setting the value of the traveling potential at the first high potential point to a second set value that is greater than the first set value; and interpolating the value of the driving potential in a region of the input image between the low potential point and the first high potential point with a value between the first set value and the second set value, thereby generating the driving potential field.

12. A program for controlling a vehicle by a computer based on an image captured by a camera facing forward as viewed from the vehicle, The computer, acquiring an image captured by the camera as an input image; generating a traveling potential field indicating a distribution of traveling potentials with respect to a future traveling position of the host vehicle based on the input image; generating a target trajectory of the host vehicle based on a gradient of the traveling potential in the traveling potential field; and operating a steering mechanism based on the target trajectory, In the step of generating the traveling potential field, the computer A process of extracting reference object information relating to a position of a reference object or a boundary line of the reference object from the input image; a process of setting a low potential point within a central region of the input image and at a position determined based on the reference subject information; A process of identifying a position within the input image of an obstacle that may hinder the safe running of the host vehicle; a process of setting a first high potential point in an area of ​​the input image in which the obstacle is captured; a process of setting the value of the traveling potential at the low potential point to a first set value; a process of setting the value of the traveling potential at the first high potential point to a second set value that is greater than the first set value; and interpolating the value of the traveling potential in a region of the input image between the low potential point and the first high potential point with a value between the first set value and the second set value, thereby generating the traveling potential field.

Images