Vehicle control device, flying object control device, vehicle control method, and program

Abstract

To provide a vehicle control device which allows a vehicle to be controlled with little burden and based on an image obtained with a camera.SOLUTION: A vehicle control device comprises: an input image acquisition part for acquiring an image taken by an on-vehicle camera directed to a front part side, as an input image; a target steering amount calculation part for calculating a target steering amount with respect to an electric power steering device based on the input image; and a steering control part for operating the electric power steering to a positive side or a negative side based on the target steering amount. The target steering amount calculation part executes: processing for recognizing a positive side compartment line LP and a negative side compartment line LN of a vehicle based on the input image; processing for calculating at least either a positive side area value in between a neutral reference line L0 and the positive side compartment line LP or a negative side area value in between the neutral reference line L0 and the negative side compartment line LN that are virtually determined in the input image; and processing for calculating a target steering amount based on at least either the positive side area value or the negative side area value.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a vehicle control device, an air vehicle control device, a vehicle control method, and a program. More particularly, the present invention relates to a vehicle control device, an air vehicle control device, a vehicle control method, and a program that control a vehicle or an air vehicle based on an image captured by a camera. [Background technology]

[0002] In recent years, efforts to provide access to sustainable transport systems that take into consideration vulnerable transport participants have become more active. To achieve this, we are focusing on research and development into preventive safety technologies to further improve road safety and convenience.

[0003] For example, Patent Document 1 describes a preventive safety technology that generates a target trajectory indicating the route the vehicle will travel in the future based on an image showing the surrounding conditions of the vehicle obtained by an on-board camera, and then automatically operates the vehicle's steering, accelerator, etc. based on this target trajectory. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2023-82451 Summary of the Invention [Problem to be solved by the invention]

[0005] In the technology described in Patent Document 1, when calculating control variables for the steering, accelerator, etc. from images acquired by an onboard camera, the images acquired by the onboard camera are converted into a bird's-eye view coordinate system, and a target trajectory is then calculated in this bird's-eye view coordinate system. However, such bird's-eye view-based control tends to increase the number of parameters that need tuning and the load on the onboard computer. Furthermore, such bird's-eye view-based control also increases the amount of work required to align output data when integrating output data from multiple sensors.

[0006] The present invention aims to provide a vehicle control device, an aircraft control device, a vehicle control method, and a program that can control vehicles and aircraft with little load based on images obtained by a camera, and ultimately to contribute to the development of sustainable transportation systems. [Means for solving the problem]

[0007] (1) A vehicle control device according to the present invention (for example, a vehicle control device 1 described later) includes: input image acquisition means (for example, an input image acquisition unit 2 described later) that acquires, as an input image, an image captured by a camera (for example, an on-board camera C described later) that is directed forward as viewed from a vehicle (for example, a vehicle V described later); target steering amount calculation means (for example, a target steering amount calculation unit 3 described later) that calculates a target steering amount (for example, a target steering amount Str described later) for a steering mechanism of the vehicle (for example, an electric power steering device 9 described later) based on the input image; and steering control means (for example, a steering control unit 4 described later) that operates the steering mechanism to a positive side or a negative side based on the target steering amount, The target steering amount calculation means is characterized in that it is configured to perform the following processes: a process of recognizing the vehicle's driving path based on the input image; a process of calculating at least one of a positive side area value (e.g., a positive side area value Sp described later) between a neutral reference line (e.g., a neutral reference line L0 described later) virtually defined in the input image and a positive side dividing line (e.g., a positive side dividing line LP described later) which is the positive side boundary of the driving path; and a negative side area value (e.g., a negative side area value Sn described later) between the neutral reference line and a negative side dividing line (e.g., a negative side dividing line LN described later) which is the negative side boundary of the driving path; and a process of calculating the target steering amount based on at least one of the positive side area value and the negative side area value.

[0008] (2) In this case, it is preferable that the target steering amount calculation means calculates the target steering amount so that it becomes larger on the positive side as the positive side area value becomes larger, and calculates the target steering amount so that it becomes larger on the negative side as the negative side area value becomes larger.

[0009] (3) In this case, it is preferable that the target steering amount calculation means virtually sets the neutral reference line extending in the vertical direction in the input image and a plurality of lateral reference lines (for example, lateral reference lines LT1 to LT6 described below) that intersect with the neutral reference line in the input image, calculates the area value of the area surrounded by the positive side demarcation line, the neutral reference line, and the lateral reference line as the positive side area value, and calculates the area value of the area surrounded by the negative side demarcation line, the neutral reference line, and the lateral reference line as the negative side area value.

[0010] (4) In this case, it is preferable that the target steering amount calculation means calculates the target steering amount by subtracting a predetermined negative side reference value from the positive side area value, subtracting the negative side area value from a predetermined positive side reference value, or subtracting the negative side area value from the positive side area value.

[0011] (5) In this case, when the positive side dividing line and the neutral reference line intersect in the input image, it is preferable that the target steering amount calculation means calculates the positive side area value by subtracting the area value between the portion of the neutral reference line above the intersection point with the positive side dividing line and the positive side dividing line from the area value between the portion of the neutral reference line below the intersection point with the positive side dividing line and the positive side dividing line.

[0012] (6) In this case, when the negative side dividing line and the neutral reference line intersect in the input image, it is preferable that the target steering amount calculation means calculates the negative side area value by subtracting the area value between the part of the neutral reference line above the intersection point with the negative side dividing line and the negative side dividing line from the area value between the part of the neutral reference line below the intersection point with the negative side dividing line and the negative side dividing line.

[0013] (7) In this case, it is preferable that the target steering amount calculation means calculates the target steering amount by dividing the difference between the positive side area value and the negative side reference value by the sum of the positive side area value and the negative side reference value, dividing the difference between the positive side reference value and the negative side area value by the sum of the positive side reference value and the negative side area value, or dividing the difference between the positive side area value and the negative side area value by the sum of the positive side area value and the negative side area value.

[0014] (8) In this case, it is preferable that the camera is provided at the center of the width of the vehicle body, and the target steering amount calculation means is configured to perform a process of setting the neutral reference line at the center of the width of the input image when the vehicle is driven along the vehicle driving lane between the positive side marking line and the negative side marking line.

[0015] (9) In this case, it is preferable that the target steering amount calculation means is configured to perform the following processes: storing the position of the neutral reference line that makes the positive side area value and the negative side area value equal as a central position when the vehicle is traveling in the center of the width direction of a straight road; and setting the neutral reference line at the central position when the vehicle is traveling along the vehicle driving lane between the positive side marking line and the negative side marking line.

[0016] (10) In this case, it is preferable that the target steering amount calculation means is configured to perform the following process: when the vehicle is driven from the own vehicle driving lane toward the positive adjacent driving lane adjacent to the positive side dividing line, the neutral reference line is moved to the positive side at a predetermined speed in the input image; and when the vehicle is driven from the own vehicle driving lane toward the negative adjacent driving lane adjacent to the negative side dividing line, the neutral reference line is moved to the negative side at a predetermined speed in the input image.

[0017] (11) In this case, it is preferable that the target steering amount calculation means is configured to perform the following process: when it recognizes the positive side dividing line and a positive side obstacle (for example, the positive side obstacle OBp described below) that hides part of it based on the input image, it moves the neutral reference line to the negative side in the input image; and when it recognizes the negative side dividing line and a negative side obstacle that hides part of it based on the input image, it moves the neutral reference line to the positive side in the input image.

[0018] (12) In this case, it is preferable that the target steering amount calculation means is configured to perform a process of estimating the negative side lane marking based on the positive side lane marking when the positive side lane marking can be recognized based on the input image but the negative side lane marking cannot be recognized, and a process of estimating the positive side lane marking based on the negative side lane marking when the negative side lane marking can be recognized based on the input image but the positive side lane marking cannot be recognized.

[0019] (13) In this case, it is preferable that the target steering amount calculation means is configured to perform the following process: when it recognizes the positive side demarcation line and a positive side obstacle (for example, the positive side obstacle OBp described below) that hides part of it based on the input image, interpolating the part of the positive side demarcation line that is hidden by the positive side obstacle with a line along the negative side of the positive side obstacle; and when it recognizes the negative side demarcation line and a negative side obstacle that hides part of it based on the input image, interpolating the part of the negative side demarcation line that is hidden by the negative side obstacle with a line along the positive side of the negative side obstacle.

[0020] (14) In this case, the vehicle further includes a target vehicle speed calculation means (for example, a target vehicle speed calculation unit 5 described later) that calculates a target vehicle speed of the vehicle based on the input image, and an acceleration / deceleration control means (for example, an acceleration / deceleration control unit 6 described later) that operates an acceleration / deceleration device of the vehicle (for example, a braking device 7 and a power plant 8 described later) based on the target vehicle speed, and the target vehicle speed calculation means performs a process of recognizing the traveling road based on the input image, a process of setting a speed control reference line (for example, a speed control reference line LT3 described later) extending in the width direction of the input image, and a process of determining a speed control reference line on the traveling road above the speed control reference line. It is preferable that the control system is configured to execute the following processes: a process of calculating the area value of the road below the speed control reference line as an upper area value (e.g., an upper area value St described later); a process of calculating the area value of the road below the speed control reference line as a lower area value (e.g., a lower area value Sb described later); a process of calculating the sum of the upper area value and the lower area value as a total area value (e.g., a total area value Stotal described later); a process of calculating the ratio of the upper area value to the total area value as an upper / lower ratio value (e.g., an upper / lower ratio value r described later); and a process of calculating the target vehicle speed based on the upper / lower ratio value.

[0021] (15) In this case, it is preferable that the target vehicle speed calculation means calculates the target vehicle speed so that the target vehicle speed decreases as the upper / lower ratio value increases.

[0022] (16) In this case, it is preferable that the vehicle control device further includes a storage medium that stores a plurality of speed tables that associate the upper / lower ratio value with the target vehicle speed, and that the target vehicle speed calculation means is configured to execute a process of estimating a curvature parameter of the road (for example, the curvature parameter θall described below) based on the input image, and a process of selecting one of the plurality of speed tables based on the curvature parameter, and calculating the target vehicle speed based on the selected speed table and the upper / lower ratio value.

[0023] (17) In this case, it is preferable that the vehicle control device further includes a storage medium that stores a plurality of speed tables that associate the upper / lower ratio value with the target vehicle speed, and that the target vehicle speed calculation means is configured to execute a process of acquiring the driving mode of the vehicle, selecting one of the plurality of speed tables based on the driving mode, and calculating the target vehicle speed based on the selected speed table and the upper / lower ratio value.

[0024] (18) In this case, it is preferable that the target vehicle speed calculation means is configured to perform a process of estimating the negative side lane marking based on the positive side lane marking when the positive side lane marking can be recognized based on the input image but the negative side lane marking cannot be recognized, and a process of estimating the positive side lane marking based on the negative side lane marking when the negative side lane marking can be recognized based on the input image but the positive side lane marking cannot be recognized.

[0025] (19) An aircraft control device according to the present invention (for example, an aircraft control device 1A described later) includes: input image acquisition means (for example, an input image acquisition unit 2A described later) that acquires, as an input image, an image captured by a camera (for example, a camera CA described later) facing forward as viewed from the aircraft (for example, an aircraft F described later); first target control amount calculation means (for example, a yaw axis target control amount calculation unit 31 described later) that calculates a first target control amount (for example, a yaw axis target control amount uy described later) for a first axis attitude control mechanism (for example, a yaw axis attitude control mechanism 9Y described later) that changes the attitude of the aircraft about a first axis (for example, a yaw axis Oy described later) based on the input image; and first axis attitude control means (for example, a yaw axis attitude control unit 41 described later) that operates the first axis attitude control mechanism to a positive side or a negative side based on the first target control amount, is characterized in that it is configured to execute the following processes: a process of recognizing a target (e.g., a target T described later) of the aircraft based on the input image; a process of virtually setting a flight path for the input image based on the position of the target in the input image; a process of calculating a first positive area value (e.g., a positive area value Syp described later) between a first control reference line (e.g., a yaw axis control reference line Ly described later) virtually defined in the input image and a first positive demarcation line (e.g., flight demarcation lines LF1 and LF2 described later) which is the boundary on the positive side of the flight path; and a first negative area value (e.g., a negative area value Syn described later) between the first control reference line and a first negative demarcation line (e.g., flight demarcation lines LF3 and LF4 described later) which is the boundary on the negative side of the flight path; and a process of calculating the first target control variable based on the first positive area value and the first negative area value.

[0026] (20) In this case, the aircraft control device further includes second target control amount calculation means (e.g., pitch axis target control amount calculation unit 32 described later) that calculates a second target control amount (e.g., pitch axis target control amount up described later) for a second axis attitude control mechanism (e.g., pitch axis attitude control mechanism 9P described later) that changes the attitude of the aircraft about a second axis (e.g., pitch axis Op described later) based on the input image, and second axis attitude control means (e.g., pitch axis attitude control unit 42 described later) that operates the second axis attitude control mechanism to the positive side or the negative side based on the second target control amount, and the second target control amount calculation means calculates a second target control amount (e.g., pitch axis target control amount up described later) for a second axis attitude control mechanism (e.g., pitch axis attitude control mechanism 9P described later) that changes the attitude of the aircraft about a second axis (e.g., pitch axis Op described later) based on the input image. It is preferable that the control system is configured to execute the following processes: calculating a second positive area value (e.g., a positive area value Spp described later) between a second control reference line (e.g., a pitch axis control reference line Lp described later) that is virtually defined so as to be perpendicular to the directrix and a second positive demarcation line that is the positive boundary of the flight path (e.g., flight demarcation lines LF2 and LF3 described later), and a second negative area value (e.g., a negative area value Spn described later) between the second control reference line and a second negative demarcation line that is the negative boundary of the flight path (e.g., flight demarcation lines LF1 and LF4 described later), and calculating the second target control variable based on the second positive area value and the second negative area value.

[0027] (21) In this case, the aircraft control device further includes third target control amount calculation means (e.g., roll axis target control amount calculation unit 33 described later) that calculates a third target control amount (e.g., roll axis target control amount ur described later) for a third axis attitude control mechanism (e.g., roll axis attitude control mechanism 9R described later) that changes the attitude of the aircraft around a third axis (e.g., roll axis Or described later) based on the input image, and third axis attitude control means (e.g., roll axis attitude control unit 43 described later) that operates the third axis attitude control mechanism to the positive side or the negative side based on the third target control amount. It is preferable that the third target control variable calculation means is configured to execute the following processes: a process of virtually setting a target posture line (e.g., a target posture line Lt described later) for the input image based on the posture of the target to be tracked in the input image; a process of calculating third area values ​​(e.g., a positive side area value Srp and a negative side area value Srn described later) between a third control reference line (e.g., a first roll axis control reference line Lr1 described later) virtually set in the input image and the target posture line; and a process of calculating the third target control variable based on the third area values. [Effects of the Invention]

[0028] (1) In the present invention, the input image acquisition means acquires as an input image an image captured by a camera facing forward as viewed from the vehicle, the target steering amount calculation means calculates a target steering amount for the vehicle's steering mechanism based on the input image, and the steering control means operates the steering mechanism to the positive or negative side based on the target steering amount. Also in the present invention, the target steering amount calculation means recognizes the vehicle's roadway based on the input image and calculates at least one of a positive-side area value between a neutral reference line and a positive-side lane marking on the roadway in the input image and a negative-side area value between the neutral reference line and a negative-side lane marking on the roadway. Here, the positive-side area value tends to increase in proportion to the distance in the width direction between the neutral line and the positive-side lane marking, and the negative-side area value tends to increase in proportion to the distance in the width direction between the neutral line and the negative-side lane marking. That is, when moving a vehicle along a roadway, it is necessary to operate the steering mechanism toward the positive or negative side so that the distance between the neutral lane marking and the positive or negative lane marking, i.e., the positive or negative area value, remains approximately constant. Furthermore, when the roadway ahead of the vehicle curves toward the positive side, i.e., when the steering mechanism needs to be gradually operated toward the positive side, the distance along the width direction between the neutral lane marking and the positive lane marking increases as the vehicle moves away from the vehicle, so the positive area value increases. Furthermore, when the roadway ahead of the vehicle curves toward the negative side, i.e., when the steering mechanism needs to be gradually operated toward the negative side, the distance along the width direction between the neutral lane marking and the negative lane marking increases as the vehicle moves away from the vehicle, so the negative area value increases. Thus, the steering amount when moving a vehicle along a roadway correlates with the positive or negative area value defined above. Therefore, according to the present invention, the target steering amount can be calculated by utilizing the correlation between the steering amount and the area value in the input image, thereby moving the vehicle along the roadway. As described above, according to the present invention, an appropriate target steering amount can be calculated simply by calculating an area value from an input image, thereby reducing the load on the computer that performs such calculations.

[0029] Furthermore, in an input image captured by a camera mounted on a vehicle, the area of ​​objects farther from the vehicle appears smaller than the area of ​​objects in front of the vehicle. Therefore, the positive side area value and negative side area value calculated as described above can be said to be naturally weighted without setting parameters. Therefore, by calculating the target steering amount based on such positive side area value and negative side area value, the number of parameters that require tuning and the load on the computer can be reduced compared to conventional methods, which can ultimately contribute to the development of sustainable transportation systems.

[0030] (2) In the present invention, the target steering amount calculation means calculates the target steering amount so that it increases toward the positive side as the positive side area value increases, and so that it increases toward the negative side as the negative side area value increases, thereby making it possible to move the vehicle without deviating from the traveling route.

[0031] (3) In the present invention, the target steering amount calculation means calculates the area value of the region surrounded by a neutral reference line extending in the up-down direction in the input image, multiple lateral reference lines intersecting with the neutral reference line, and the positive-side demarcation line as the positive-side area value, and calculates the area value of the region surrounded by the neutral reference line, the multiple lateral reference lines, and the negative-side demarcation line as the negative-side area value. Thus, according to the present invention, the positive-side area value and the negative-side area value can be calculated by simple calculation.

[0032] (4) In the present invention, the target steering amount calculation means calculates the target steering amount by subtracting a predetermined negative reference value from the positive area value, subtracting the negative area value from the predetermined positive reference value, or subtracting the negative area value from the positive area value. Therefore, according to the present invention, the target steering amount can be calculated if at least one of the positive and negative lane markings can be recognized from the input image.

[0033] (5) In the present invention, when the positive-side lane marking line and the neutral reference line intersect in the input image, i.e., when the positive-side lane marking line is bent significantly to the negative side ahead of the vehicle, the target steering amount calculation means calculates the positive-side area value by subtracting the area value between the portion of the neutral reference line above the intersection point with the positive-side lane marking line (i.e., the farther side as seen from the vehicle) and the positive-side lane marking line from the area value between the portion of the neutral reference line below the intersection point with the positive-side lane marking line (i.e., the closer side as seen from the vehicle) and the positive-side lane marking line. This makes it possible to calculate the positive-side area value taking the curvature of the road into consideration.

[0034] (6) In the present invention, when the negative-side lane marking line and the neutral reference line intersect in the input image, i.e., when the negative-side lane marking line is bent significantly to the positive side ahead of the vehicle, the target steering amount calculation means calculates the negative-side area value by subtracting the area value between the portion of the neutral reference line above the intersection point with the negative-side lane marking line (i.e., farther from the vehicle) from the area value between the portion of the neutral reference line below the intersection point with the negative-side lane marking line (i.e., closer to the vehicle) and the negative-side lane marking line. This makes it possible to calculate the negative-side area value taking the curvature of the road into consideration.

[0035] (7) In the present invention, the target steering amount calculation means calculates the target steering amount by dividing the difference between the positive area value and the negative reference value by the sum of the positive area value and the negative reference value, dividing the difference between the positive reference value and the negative area value by the sum of the positive area value and the negative area value, or dividing the difference between the positive area value and the negative area value by the sum of the positive area value and the negative area value. In other words, in the present invention, robustness can be improved by normalizing the positive area value or the negative area value to calculate the target steering amount.

[0036] (8) In the present invention, the camera is provided at the center in the width direction of the vehicle body. Furthermore, when the vehicle is driven along the host vehicle driving lane between the positive and negative lane markings, the target steering amount calculation means sets a neutral reference line at the center in the width direction of the input image. This allows the vehicle to be driven so that the neutral reference line is positioned in the center between the positive and negative lane markings in the input image, in other words, so that the vehicle body is positioned in the center of the host vehicle driving lane.

[0037] (9) In the present invention, the target steering amount calculation means stores, as the center position, the position of the neutral reference line that makes the positive side area value and the negative side area value equal when the vehicle is traveling in the center of the width direction of a straight road. The target steering amount calculation means also sets the neutral reference line to the center position when the vehicle is traveling along the host vehicle's lane between the positive side marking line and the negative side marking line. This allows the vehicle to travel so that the vehicle body is positioned in the center of the host vehicle's lane, even if the camera is not installed in the center of the width direction of the vehicle body.

[0038] (10) In the present invention, when the vehicle is caused to travel from the host vehicle driving lane toward the adjacent driving lane on the positive side, the target steering amount calculation means moves the neutral reference line in the input image toward the positive side at a predetermined speed. This allows the vehicle to automatically move from the host vehicle driving lane to the adjacent driving lane on the positive side. Furthermore, when the vehicle is caused to travel from the host vehicle driving lane toward the adjacent driving lane on the negative side, the target steering amount calculation means moves the neutral reference line in the input image toward the negative side at a predetermined speed. This allows the vehicle to automatically move from the host vehicle driving lane to the adjacent driving lane on the negative side.

[0039] (11) In the present invention, when the target steering amount calculation means recognizes a forward-side lane marking and a forward-side obstacle that partially obscures it based on the input image, the target steering amount calculation means moves the neutral reference line to the negative side in the input image. This allows the vehicle to avoid the forward-side obstacle. Also, when the target steering amount calculation means recognizes a negative-side lane marking and a backward-side obstacle that partially obscures it based on the input image, the target steering amount calculation means moves the neutral reference line to the positive side in the input image. This allows the vehicle to avoid the backward-side obstacle.

[0040] (12) In the present invention, if the positive-side lane marking can be recognized but the negative-side lane marking cannot be recognized based on the input image, the target steering amount calculation means estimates the negative-side lane marking based on the positive-side lane marking. Furthermore, if the negative-side lane marking can be recognized but the positive-side lane marking cannot be recognized based on the input image, the target steering amount calculation means estimates the positive-side lane marking based on the negative-side lane marking. Therefore, according to the present invention, even if either the positive-side lane marking or the negative-side lane marking cannot be recognized, both the positive-side area value and the negative-side area value can be calculated, and thus the target steering amount can be calculated using both the positive-side area value and the negative-side area value.

[0041] (13) In the present invention, when the target steering amount calculation means recognizes a forward lane marking and a forward obstacle that partially obscures it based on the input image, the target steering amount calculation means interpolates the portion of the forward lane marking that is obscured by the forward obstacle with a line along the negative side of the forward obstacle. This makes it possible to reduce the positive side area value by the amount of the forward obstacle, allowing the vehicle to avoid the forward obstacle. Furthermore, when the target steering amount calculation means recognizes a negative lane marking and a backward obstacle that partially obscures it based on the input image, the target steering amount calculation means interpolates the portion of the negative lane marking that is obscured by the backward obstacle with a line along the positive side of the backward obstacle. This makes it possible to reduce the negative side area value by the amount of the backward obstacle, allowing the vehicle to avoid the backward obstacle.

[0042] (14) In the present invention, the target vehicle speed setting means calculates the target vehicle speed based on the input image, and the acceleration / deceleration control means operates the acceleration / deceleration device based on the target vehicle speed. The target vehicle speed setting means also recognizes the roadway based on the input image, sets a speed control reference line extending in the width direction of the input image, calculates the area of ​​the roadway above the speed control reference line as an upper area value, calculates the area of ​​the roadway below the speed control reference line as a lower area value, calculates the sum of these area values ​​as a total area value, calculates the ratio of the upper area value to the total area value as an upper / lower ratio value, and further calculates the target vehicle speed based on the upper / lower ratio value. Here, when the roadway ahead of the vehicle curves to the positive or negative side, i.e., when the vehicle speed needs to be reduced, the upper / lower ratio value is larger than when the roadway is straight. Therefore, according to the present invention, the vehicle speed can be controlled with simple calculations by utilizing the correlation between an appropriate vehicle speed according to the roadway conditions ahead and the upper / lower ratio value defined above.

[0043] (15) In the present invention, the target vehicle speed calculation means calculates the target vehicle speed so that it decreases as the upper / lower ratio value increases. This allows the vehicle speed to be changed in accordance with the condition of the road ahead of the vehicle.

[0044] (16) In the present invention, the target vehicle speed calculation means estimates a curvature parameter of the road based on the input image, selects one from a plurality of speed tables based on the curvature parameter, and calculates the target vehicle speed based on the selected speed table and the upper / lower ratio value. Thus, according to the present invention, the vehicle speed can be changed according to the curvature parameter of the road.

[0045] (17) In the present invention, the target vehicle speed calculation means acquires the vehicle's driving mode, selects one from a plurality of speed tables based on the driving mode, and calculates the target vehicle speed based on the selected speed table and the upper / lower ratio value. Thus, according to the present invention, the vehicle speed can be changed in accordance with the curvature parameter of the road.

[0046] (18) In the present invention, if the target vehicle speed calculation means can recognize the positive side lane marking but cannot recognize the negative side lane marking based on the input image, it estimates the negative side lane marking based on the positive side lane marking. Furthermore, if the target vehicle speed calculation means can recognize the negative side lane marking but cannot recognize the positive side lane marking based on the input image, it estimates the positive side lane marking based on the negative side lane marking. Therefore, according to the present invention, even if either the positive side lane marking or the negative side lane marking cannot be recognized, the upper side area value, lower side area value, upper / lower ratio value, etc. can be calculated, and the target vehicle speed can be calculated using these upper side area value, lower side area value, upper / lower ratio value, etc.

[0047] (19) In the present invention, the input image acquisition means acquires as an input image an image captured by a camera facing forward as viewed from the aircraft, the first target control variable calculation means calculates a first target control variable for the first-axis attitude control mechanism of the aircraft based on the input image, and the first-axis attitude control means operates the first-axis attitude control mechanism to the positive or negative side based on the first target control variable. Also in the present invention, the first target control variable calculation means recognizes a target to be followed by the aircraft based on the input image, sets a virtual flight path for the input image based on the position of the target to be followed in the input image, and calculates the first target control variable based on a first positive-side area value between the first control reference line and a first positive-side demarcation line that is the positive-side boundary of the flight path, and a first negative-side area value between the first control reference line and a first negative-side demarcation line that is the negative-side boundary of the flight path. Therefore, according to the present invention, the first target control variable for the first-axis attitude control mechanism of the aircraft can be calculated simply by calculating area values ​​from the input image, thereby reducing the burden on a computer that performs such calculations.

[0048] (20) In the present invention, the second target control amount calculation means calculates a second target control amount for the second-axis attitude control mechanism of the aircraft based on the input image, and the second-axis attitude control means operates the second-axis attitude control mechanism to the positive side or the negative side based on the second target control amount. Also in the present invention, the second target control amount calculation means calculates the second target control amount based on a second positive side area value between the second control reference line and a second positive side demarcation line that is the boundary on the positive side of the flight path, and a second negative side area value between the second control reference line and a second negative side demarcation line that is the negative side of the flight path. Therefore, according to the present invention, the second target control amount for the second-axis attitude control mechanism of the aircraft can be calculated simply by calculating area values ​​from the input image, thereby reducing the burden on the computer that performs such calculations.

[0049] (21) In the present invention, the third target control amount calculation means calculates a third target control amount for the third axis attitude control mechanism of the aircraft based on the input image, and the third axis attitude control means operates the third axis attitude control mechanism to the positive or negative side based on the third target control amount. Also in the present invention, the third target control amount calculation means virtually sets a target attitude line for the input image based on the attitude of the target to be tracked in the input image, calculates a third area value between the virtually defined third control reference line and the target attitude line in the input image, and calculates the third target control amount based on the third area value. Therefore, according to the present invention, the third target control amount for the third axis attitude control mechanism of the aircraft can be calculated simply by calculating the area value from the input image, thereby reducing the burden on the computer that performs such calculations. [Brief explanation of the drawings]

[0050] [Figure 1] 1 is a diagram schematically illustrating the configuration of a vehicle equipped with a vehicle control device according to a first embodiment of the present invention. [Figure 2] FIG. 2 is a functional block diagram of a vehicle control device. [Figure 3] 10 is a flowchart showing a specific procedure of a target steering amount calculation process. [Figure 4] FIG. 10 is a diagram showing an example of an input image (when both positive and negative lane markings are recognized); [Figure 5] 10 is a flowchart showing a specific procedure of a total area calculation process in a target steering amount calculation unit. [Figure 6] 10 is a flowchart showing a specific procedure of one-sided area calculation processing in a target steering amount calculation unit. [Figure 7] FIG. 10 is a diagram showing an example of an input image (when only the forward lane markings are recognized). [Figure 8] FIG. 10 is a diagram showing an example of an input image (when the road on which the host vehicle is traveling is turning significantly forward). [Figure 9] FIG. 10 is a diagram showing an example of an input image (in the case where a front-side obstacle exists that hides part of the front-side lane marking); [Figure 10] 10 is a flowchart showing a specific procedure for a target vehicle speed calculation process. [Figure 11] 10 is a flowchart showing a specific procedure of a total area calculation process in a target vehicle speed calculation unit. [Figure 12] FIG. 12 is a diagram showing an example of an input image for explaining the processing of FIG. 11. [Figure 13] FIG. 10 is a diagram illustrating an example of a speed table. [Figure 14] FIG. 10 is a diagram showing an example of an input image (when the road on which the host vehicle is traveling is turning to the negative side). [Figure 15] FIG. 10 is a diagram showing an example of an input image (when the road on which the host vehicle is traveling is turning forward). [Figure 16] 10A and 10B are diagrams illustrating an example of an input image for explaining a procedure for calculating a curvature parameter. [Figure 17] 10 is a flowchart showing a specific procedure of a target vehicle speed calculation process according to the second embodiment of the present invention. [Figure 18] FIG. 18 is a diagram showing an example of an input image for explaining the processing of FIG. 17. [Figure 19] FIG. 10 is a diagram schematically illustrating the configuration of an aircraft equipped with an aircraft control device according to a third embodiment of the present invention. [Figure 20] FIG. 2 is a functional block diagram of an aircraft control device. [Figure 21]10 is a flowchart showing a specific procedure for a yaw axis target control amount calculation process. [Figure 22] FIG. 22 is a diagram showing an example of an input image for explaining the processing of FIG. 21. [Figure 23] 10 is a flowchart showing a specific procedure for a pitch axis target control amount calculation process. [Figure 24] FIG. 24 is a diagram showing an example of an input image for explaining the processing of FIG. 23. [Figure 25] 10 is a flowchart showing a specific procedure of a roll axis target control amount calculation process. [Figure 26] FIG. 26 is a diagram showing an example of an input image for explaining the processing of FIG. 25. DETAILED DESCRIPTION OF THE INVENTION

[0051] First Embodiment A vehicle control device according to a first embodiment of the present invention will be described below with reference to the drawings.

[0052] FIG. 1 is a diagram schematically illustrating the configuration of a vehicle V equipped with a vehicle control device 1 according to this embodiment. The upper part of FIG. 1 shows a plan view of the vehicle V, and the lower part of FIG. 1 shows a side view. Note that the following description will be given assuming that the vehicle V is a so-called right-hand drive four-wheel vehicle in which the driver's seat is located on the right side in the vehicle width direction as viewed along the direction of travel, but the present invention is not limited to this. The vehicle V may also be a so-called left-hand drive four-wheel vehicle in which the driver's seat is located on the left side in the vehicle width direction as viewed along the direction of travel.

[0053] The vehicle V is equipped with an electric power steering device 9 as a steering mechanism that steers the left and right front wheels Wf, a power plant 8 as a driving device that generates a driving force to rotate the front wheels Wf, which are the driving wheels of the vehicle V, a braking device 7 that generates a braking force to stop the rotation of the front wheels Wf and rear wheels Wr, an on-board camera C that captures images of the surroundings of the vehicle V, and a vehicle control device 1 that controls the electric power steering device 9, the power plant 8, and the braking device 7 based on the images captured by the on-board camera C.

[0054] The electric power steering device 9 includes a gearbox 93 that connects a pinion shaft 92 extending from a steering wheel 91 that receives steering operations by the driver to the left and right front wheels Wf, an electric motor 94 provided in the gearbox 93, and a steering sensor 95 that detects the steering amount of the steering wheel 91.

[0055] The gearbox 93 includes a rack shaft extending along the vehicle width direction and meshing with the pinion shaft 92, tie rods connecting both ends of the rack shaft to the left and right front wheels Wf, and other components. The gearbox 93 converts the rotational motion of the steering wheel 91, generated by the driver's steering operation, into motion along the vehicle width direction, thereby steering the left and right front wheels Wf toward the vehicle's traveling direction. The electric motor 94 rotates in response to a control signal output from the vehicle control device 1 and generates a driving force for assisting the driver's steering operation or for automatically steering the front wheels Wf without the driver's steering operation. The steering sensor 95 detects the steering amount of the steering wheel 91 and transmits a signal corresponding to the detected value to the vehicle control device 1. In the following description, the steering amount is set to zero when traveling straight, positive when turning right, and negative when turning left. However, the present invention is not limited to this. The steering amount may be set to negative when turning right and positive when turning left.

[0056] The power plant 8 is a driving force generating source that generates a driving force for rotating the front wheels Wf to move the vehicle V forward or backward along the traveling direction in response to acceleration / deceleration operation of an accelerator pedal (not shown) by the driver and control signals output from the vehicle control device 1. In the following, a case will be described in which the power plant 8 is a drive motor that generates driving force by consuming electric power supplied from a high-voltage battery, a fuel cell stack, or the like (not shown), but the present invention is not limited to this. The power plant 8 may also be an engine that generates driving force by consuming fuel stored in a fuel tank (not shown), or a transmission that changes the speed of the engine output and transmits it to the front wheels Wf.

[0057] The braking device 7 includes a disc brake device that generates braking force to slow down or stop the rotation of each wheel Wf, Wr by tightening discs provided on the axles of each wheel Wf, Wr mainly when driving, based on braking operation of the brake pedal (not shown) by the driver or control signals output from the vehicle control device 1, and a parking brake that generates braking force to keep the rotation of each wheel Wr, Wf stopped mainly when parking.

[0058] The vehicle-mounted camera C is directed forward along the traveling direction as viewed from the vehicle V. In this embodiment, the vehicle-mounted camera C is installed in the center of the body of the vehicle V in the vehicle width direction, but the present invention is not limited to this.

[0059] The vehicle control device 1 controls the electric power steering device 9, the power plant 8, and the braking device 7 based on an image of the front side of the vehicle V captured by an on-board camera C. The vehicle control device 1 is a computer configured with hardware such as a calculation processing means such as a CPU, auxiliary storage means such as an HDD or SSD that stores programs that cause the calculation processing means to execute target steering amount calculation processing and target vehicle speed calculation processing, which will be described later, and a main storage means such as a RAM for storing data that is temporarily required when the calculation processing means executes the programs.

[0060] 2 is a functional block diagram of the vehicle control device 1. The vehicle control device 1 has the above-described hardware configuration, which includes an input image acquisition unit 2, a target steering amount calculation unit 3, a steering control unit 4, a target vehicle speed calculation unit 5, and an acceleration / deceleration control unit 6.

[0061] The input image acquisition unit 2 acquires an image of the front side of the vehicle V captured by the on-board camera C as an input image. The input image acquisition unit 2 transmits the acquired input image to the target steering amount calculation unit 3 and the target vehicle speed calculation unit 5.

[0062] The target steering amount calculation unit 3 calculates a target steering amount for the steering amount of the electric power steering device 9 based on the input image transmitted from the input image acquisition unit 2. The procedure for calculating the target steering amount based on the input image in the target steering amount calculation unit 3 will be described later with reference to Figs. 3 to 9 etc. The steering control unit 4 executes automatic steering control to automatically operate the electric motor 94 of the electric power steering device 9 so that the target steering amount calculated by the target steering amount calculation unit 3 matches the steering amount detected by the steering sensor 95.

[0063] The target vehicle speed calculation unit 5 calculates a target vehicle speed for the vehicle speed of the vehicle V based on the input image transmitted from the input image acquisition unit 2. The procedure for calculating the target vehicle speed based on the input image in the target vehicle speed calculation unit 5 will be described later with reference to Figs. 10 to 16, etc. The acceleration / deceleration control unit 6 executes automatic acceleration / deceleration control that automatically operates the power plant 8 and the braking device 7 so that the target vehicle speed calculated by the target vehicle speed calculation unit 5 matches the vehicle speed of the vehicle V detected by a vehicle speed sensor (not shown).

[0064] 3 is a flowchart showing the specific steps of the target steering amount calculation process for calculating the target steering amount based on the input image. This target steering amount calculation process is executed by the target steering amount calculation unit 3 every time the input image acquisition unit 2 acquires a new input image.

[0065] First, in step ST1, the target steering amount calculation unit 3 recognizes the vehicle travel path corresponding to the range in which the vehicle V is allowed to travel, based on the input image, and then proceeds to step ST2.

[0066] 4 is a diagram showing an example of an input image. In accordance with the definition of the steering amount described above, the right side of the input image will be referred to as the positive side, and the left side will be referred to as the negative side. In addition, in the following, the horizontal axis extending in the width direction of the input image will be referred to as the X axis, and the vertical axis of the input image that is perpendicular to the X axis and extends in the up-down direction will be referred to as the Y axis. In the following, the center point of the input image will be referred to as the origin of the X axis and the Y axis.

[0067] The target steering amount calculation unit 3 recognizes the vehicle's driving path by extracting, from the input image, characteristic lines such as white lines and curbs extending on both the positive and negative sides as viewed from the vehicle V (i.e., on both the left and right sides in FIG. 4). More specifically, the target steering amount calculation unit 3 recognizes a characteristic line extending on the positive side as viewed from the vehicle V (i.e., on the right side in FIG. 4) and having a length longer than a predetermined length as a positive-side lane marking line LP (i.e., right-side lane marking line in FIG. 4) corresponding to the positive-side end of the vehicle's driving path (i.e., the right end in FIG. 4), and recognizes a characteristic line extending on the negative side as viewed from the vehicle V (i.e., the left side in FIG. 4) and having a length longer than a predetermined length as a negative-side lane marking line LN (i.e., left-side lane marking line in FIG. 4) corresponding to the negative-side end of the vehicle's driving path (i.e., the left end in FIG. 4). Here, if a portion of a characteristic line extracted from the input image is discontinued, the target steering amount calculation unit 3 may recognize the positive-side lane marking line LP or the negative-side lane marking line LN by interpolating the characteristic line according to a known algorithm.

[0068] If the target steering amount calculation unit 3 can recognize both the positive side lane marking LP and the negative side lane marking LN from the input image, it recognizes the area between these positive side lane marking LP and negative side lane marking LN in the input image as the vehicle's driving path.If the target steering amount calculation unit 3 can recognize the positive side lane marking LP but cannot recognize the negative side lane marking LN from the input image, it recognizes the area on the negative side of the positive side lane marking LP in the input image as the vehicle's driving path.If the target steering amount calculation unit 3 can recognize the negative side lane marking LN but cannot recognize the positive side lane marking LP from the input image, it recognizes the area on the positive side of the negative side lane marking LN in the input image as the vehicle's driving path.

[0069] Returning to FIG. 3, in step ST2, the target steering amount calculation unit 3 determines whether or not the host vehicle's driving lane was recognized from the input image in step ST1. More specifically, if the target steering amount calculation unit 3 can recognize at least one of the positive-side lane marking LP and the negative-side lane marking LN that demarcate the host vehicle's driving lane from the input image, it determines that the host vehicle's driving lane has been recognized and proceeds to step ST3. If the target steering amount calculation unit 3 cannot recognize either the positive-side lane marking LP or the negative-side lane marking LN, it determines that the host vehicle's driving lane has not been recognized and proceeds to step ST7. Also, in step ST7, the target steering amount calculation unit 3 cancels the automatic steering control by the steering control unit 4 and ends the target steering amount calculation process of FIG. 3.

[0070] Next, in step ST3, the target steering amount calculation unit 3 determines whether or not both the positive side lane marking LP and the negative side lane marking LN have been recognized. If the determination result in step ST3 is YES, i.e., if both the positive and negative lane markings LP and LN have been recognized, the target steering amount calculation unit 3 executes a total area calculation process (see step ST4) which will be described with reference to FIG. 5, and calculates a target steering amount according to the input image. If the determination result in step ST3 is NO, i.e., if only one of the positive and negative lane markings LP and LN has been recognized, the target steering amount calculation unit 3 executes a one-side area calculation process (see step ST5) which will be described with reference to FIG. 6, and calculates a target steering amount according to the input image.

[0071] Next, in step ST6, the target steering amount calculation unit 3 transmits the target steering amount calculated by executing the total area calculation process (step ST4) or the one-sided area calculation process (step ST5) to the steering control unit 4, and terminates the target steering amount calculation process of Figure 3.

[0072] 5 is a flowchart showing a specific procedure for the total area calculation process in the target steering amount calculation unit 3. As described above, this total area calculation process is executed when both the positive and negative lane marking lines LP and LN can be recognized from the input image.

[0073] First, in step ST11, the target steering amount calculation unit 3 sets a virtual neutral reference line L0 in the input image, and then proceeds to step ST12. As described above, the on-board camera C is provided at the center in the width direction of the vehicle body. Therefore, when the vehicle V is to travel along the host vehicle driving lane between the positive side marking line LP and the negative side marking line LN, it is preferable that the target steering amount calculation unit 3 sets the neutral reference line L0 at the origin of the X axis (i.e., the center in the width direction of the input image) as shown in FIG. 4. Note that a case where the on-board camera C is not provided at the center in the width direction of the vehicle body will be described later.

[0074] In step ST12, the target steering amount calculation unit 3 sets multiple (six in the example of FIG. 4) lateral reference lines LT1, LT2, LT3, LT4, LT5, and LT6 in the input image that intersect with the neutral reference line L0 set in step ST11 and are parallel to one another, and then proceeds to step ST13. Note that the following description will be given of a case in which the target steering amount calculation unit 3 sets multiple lateral reference lines LT1 to LT6 that are perpendicular to the neutral reference line L0 and at equal intervals along the Y axis as shown in FIG. 4, but the present invention is not limited to this. These lateral reference lines LT1 to LT6 may intersect with the neutral reference line L0 at an angle other than 90°, and may not be at equal intervals. In addition, the target steering amount calculation unit 3 sets the multiple lateral reference lines LT1 to LT6 below the vanishing point in the input image where the distance along the X axis between the positive-side marking line LP and the negative-side marking line LN becomes approximately zero.

[0075] In step ST13, the target steering amount calculation unit 3 determines the intersections of the multiple lateral reference lines LT1 to LT6 set in the input image in step ST12 with the positive-side lane marking line LP and the negative-side lane marking line LN recognized from the input image as feature points, calculates the coordinate values ​​of these feature points, and proceeds to step ST14. Note that hereinafter, the intersections of the lateral reference lines LT1 to LT6 with the positive-side lane marking line LP are referred to as positive-side feature points, and the intersections of the lateral reference lines LT1 to LT6 with the negative-side lane marking line LN are referred to as negative-side feature points.

[0076] In the following, the coordinate value of the i-th (i is an integer from 1 to 6) positive side characteristic point (i.e., the intersection of the i-th horizontal reference line LTi and the positive side lane marking line LP) is expressed as (xp i ,yp i), and the coordinate value of the i-th negative side characteristic point (i.e., the intersection of the i-th horizontal reference line LTi and the negative side lane marking line LN) is expressed as (xn i ,yn i ) is written as

[0077] In step ST14, the target steering amount calculation unit 3 calculates the area value of the region between the neutral reference line L0 and the positive-side demarcation line LP in the input image as the positive-side area value by using the coordinate values ​​of the multiple positive-side feature points calculated in step ST13, and then proceeds to step ST15. More specifically, the target steering amount calculation unit 3 calculates the area value of the region surrounded by the positive-side demarcation line LP, the neutral reference line L0, and the multiple lateral reference lines LT1 to LT6 in the input image as the positive-side area value Sp, and calculates this positive-side area value Sp by the quadrature by division method as shown in the following equation (1). Note that, hereinafter, "a" is the coordinate value of the neutral reference line L0 along the X-axis.

number

[0078] In step ST15, the target steering amount calculation unit 3 calculates the area value of the region between the neutral reference line L0 and the negative-side demarcation line LN in the input image as a negative-side area value by using the coordinate values ​​of the multiple negative-side feature points calculated in step ST13, and proceeds to step ST16. More specifically, the target steering amount calculation unit 3 calculates the area value of the region surrounded by the negative-side demarcation line LN, the neutral reference line L0, and the multiple lateral reference lines LT1 to LT6 in the input image as a negative-side area value Sn, and calculates this negative-side area value Sn by the quadrature by division method as shown in the following equation (2).

number

[0079] In step ST16, the target steering amount calculation unit 3 calculates the target steering amount Str based on both the positive-side area value Sp and the negative-side area value Sn calculated by the above-described procedure, and ends the total area calculation process shown in Fig. 5. More specifically, the target steering amount calculation unit 3 calculates the target steering amount Str so that the larger the positive-side area value Sp is, the larger the target steering amount Str becomes toward the positive side, and the larger the negative-side area value Sn is, the larger the target steering amount Str becomes toward the negative side.

[0080] More specifically, the target steering amount calculation unit 3 calculates the target steering amount Str by dividing the difference (Sp-Sn) between the positive area value Sp and the negative area value Sn by the sum (Sp+Sn) of the positive area value Sp and the negative area value Sn, as shown in the following equation (3). Note that, although the following description will be given of a case where the gain Gain in the following equation (3) is a positive constant, the present invention is not limited to this. The gain Gain may also be a negative value. The value of this gain Gain may also be changed depending on the vehicle speed, the arrangement pattern of the feature points, etc.

number

[0081] As shown in Figure 4, the positive area value Sp increases as the distance along the X-axis between the neutral reference line L0 and the positive-side lane marking line LP increases, and the negative area value Sn increases as the distance along the X-axis between the neutral reference line L0 and the negative-side lane marking line LN increases. Therefore, by calculating the target steering amount Str according to the above equation (3), the target steering amount Str increases toward the positive side as the proportion of the positive area value Sp in the total area value (Sp + Sn) of the host vehicle's road in the input image increases—in other words, as the distance along the X-axis between the neutral reference line L0 and the positive-side lane marking line LP increases. Furthermore, by increasing the target steering amount Str toward the positive side, the neutral reference line L0 approaches the positive-side lane marking line LP in the input image. Furthermore, by calculating the target steering amount Str according to the above equation (3), the larger the proportion of the negative area value Sn to the total area value (Sp+Sn) of the vehicle's roadway in the input image, in other words, the longer the distance along the X-axis between the neutral reference line L0 and the negative side marking line LN, the larger the target steering amount Str becomes on the negative side. Also, by increasing the target steering amount Str on the negative side, the neutral reference line L0 moves closer to the negative side marking line LN on the input image. Therefore, by calculating the target steering amount Str according to the above equation (3), the electric power steering device 9 can be automatically operated so that the positive side area value Sp and the negative side area value Sn are equal, in other words, so that the neutral reference line L0 is maintained in the center between the positive side marking line LP and the negative side marking line LN.

[0082] 6 is a flowchart showing the specific steps of the one-sided area calculation process in the target steering amount calculation unit 3. As described above, this one-sided area calculation process is executed when only one of the positive and negative lane markings LP and LN can be recognized from the input image.

[0083] Fig. 7 is a diagram showing an example of an input image, in which the positive side marking LP can be recognized but the negative side marking LN cannot be recognized.

[0084] First, in step ST21, the target steering amount calculation section 3 sets a virtual neutral reference line L0 in the input image using the same procedure as in step ST11 in FIG. 5, and then the process proceeds to step ST22.

[0085] In step ST22, the target steering amount calculation unit 3 sets a plurality of (six in the example of FIG. 5) lateral reference lines LT1 to LT6 in the same procedure as in step ST12 of FIG. 5, and then proceeds to step ST23.

[0086] In step ST23, the target steering amount calculation unit 3 calculates the coordinate values ​​of a plurality of positive side feature points or a plurality of negative side feature points by the same procedure as in step ST13, and then proceeds to step ST24. More specifically, the target steering amount calculation unit 3 calculates the coordinate values ​​of a plurality of feature points defined on the lane markings recognized from the input image, and then proceeds to step ST24. In step ST22, the intersections of the multiple horizontal reference lines LT1 to LT6 set in the input image with the positive-side demarcation line LP and the negative-side demarcation line LN recognized from the input image are set as feature points, the coordinate values ​​of these feature points are calculated, and the process proceeds to step ST14. Note that hereinafter, the intersections of the horizontal reference lines LT1 to LT6 with the positive-side demarcation line LP are referred to as positive-side feature points, and the intersections of the horizontal reference lines LT1 to LT6 with the negative-side demarcation line LN are referred to as negative-side feature points.

[0087] In step ST24, the target steering amount calculation unit 3 calculates either the positive-side area value Sp or the negative-side area value Sn using the same procedure as in step ST14 or step ST15, and then proceeds to step ST26. More specifically, if the target steering amount calculation unit 3 can recognize the positive-side lane marking LP from the input image, it calculates the positive-side area value Sp using the same procedure as in step ST14. Furthermore, if the target steering amount calculation unit 3 can recognize the negative-side lane marking LN from the input image, it calculates the negative-side area value Sn using the same procedure as in step ST15.

[0088] In step ST26, the target steering amount calculation unit 3 calculates the target steering amount Str based on either the positive-side area value Sp or the negative-side area value Sn calculated by the above-described procedure, and ends the one-sided area calculation process shown in Fig. 6. More specifically, the target steering amount calculation unit 3 calculates the target steering amount Str so that the larger the positive-side area value Sp is, the larger the target steering amount Str becomes toward the positive side, and the larger the negative-side area value Sn is, the larger the target steering amount Str becomes toward the negative side.

[0089] More specifically, when the target steering amount calculation unit 3 calculates the positive side area value Sp, it divides the difference (Sp-Sn_tmp) between the positive side area value Sp and a predetermined negative side reference value Sn_tmp by the sum (Sp+Sn_tmp) of the positive side area value Sp and the negative side reference value Sn_tmp, and further multiplies the result by a predetermined gain Gain, as shown in the following equation (4). Note that, although the following description will be given of a case where the gain Gain in the following equation (4) is a positive constant, the present invention is not limited to this. The gain Gain may also be a negative value. Furthermore, the value of this gain Gain may be changed depending on the vehicle speed, the arrangement pattern of the feature points, etc.

number

[0090] Furthermore, when the target steering amount calculation unit 3 calculates the negative area value Sn, it divides the difference (Sp_tmp-Sn) between a predetermined positive reference value Sp_tmp and the negative area value Sn by the sum (Sp_tmp+Sn) of the positive reference value Sp_tmp and the negative area value Sn, as shown in the following equation (5), and further multiplies the result by a predetermined gain Gain to calculate the target steering amount Str. Note that, although the following description will be given of a case where the gain Gain in the following equation (5) is a positive constant, the present invention is not limited to this. The gain Gain may also be a negative value. Furthermore, the value of this gain Gain may be changed depending on the vehicle speed, the arrangement pattern of the feature points, etc.

number

[0091] Here, if the negative-side lane marking LN cannot be recognized from the input image, the target steering amount calculation unit 3 may set the negative-side area value Sn calculated from the input image when the negative-side lane marking LN can be recognized as the negative-side reference value Sn_tmp. Also, if the positive-side lane marking LP cannot be recognized from the input image, the target steering amount calculation unit 3 may set the positive-side area value Sp calculated from the input image when the positive-side lane marking LP can be recognized as the positive-side reference value Sp_tmp. Also, if the width of the lane on which the host vehicle is traveling can be obtained based on map information (not shown), the target steering amount calculation unit 3 may set the negative-side reference value Sn_tmp and the positive-side reference value Sp_tmp based on this width of the lane on which the host vehicle is traveling.

[0092] According to the one-side area calculation process described above, for the same reason as the total area calculation process shown in Figure 5, the electric power steering device 9 can be automatically operated so that the neutral reference line L0 is maintained in the center between the positive side demarcation line LP and the negative side demarcation line LN.

[0093] Above, the specific procedures for the total area calculation process and one-side area calculation process in the target steering amount calculation unit 3 according to this embodiment have been explained using the example of the case where the vehicle's driving path is straight, as shown in Figures 4 and 7, but the present invention can also be applied when the vehicle's driving path is significantly curved.

[0094] Fig. 8 is a diagram showing an example of an input image, in which the road on which the vehicle is traveling is turning sharply forward.

[0095] As shown in Figure 8, when the vehicle's traveling path curves significantly to the positive side, the negative-side lane marking line LN and the neutral reference line L0 intersect in the input image. When the negative-side lane marking line LN and the neutral reference line L0 intersect in the input image in this way, in the processing of step ST15 in Figure 5 or step ST24 in Figure 6, the target steering amount calculation unit 3 calculates the negative-side area value Sn by subtracting the area value of region R2 between the portion of the neutral reference line L0 above the intersection point P with the negative-side lane marking line LN and the negative-side lane marking line LN from the area value of region R1 between the portion of the neutral reference line L0 below the intersection point P with the negative-side lane marking line LN and the negative-side lane marking line LN.

[0096] Although not shown in the drawings, the same applies when the vehicle's roadway curves significantly to the negative side. That is, when the positive-side lane marking LP and the neutral reference line L0 intersect in the input image, in the processing of step ST14 in Fig. 5 or step ST24 in Fig. 6, the target steering amount calculation unit 3 calculates the positive-side area value Sp by subtracting the area value of the area between the portion of the neutral reference line L0 above the intersection point with the positive-side lane marking LP and the positive-side lane marking LP from the area value of the area between the portion of the neutral reference line L0 below the intersection point with the positive-side lane marking LP and the positive-side lane marking LP.

[0097] In this way, when the vehicle's driving path is significantly curved to the positive or negative side, the target steering amount calculation unit 3 can prompt the vehicle to steer in the direction in which the vehicle's driving path is curved by calculating the positive side area value Sp and the negative side area value Sn using the above-mentioned procedure.

[0098] In the above-described total area calculation process and one-side area calculation process, the target steering amount calculation unit 3 assumes that the vehicle V is traveling within the host vehicle driving lane and fixes the position of the neutral reference line L0, but the present invention is not limited to this. When the vehicle V is traveling from the host vehicle driving lane toward the positive-side adjacent driving lane adjacent to the positive-side lane marking LP, the target steering amount calculation unit 3 preferably moves the neutral reference line L0 from the center in the width direction of the input image toward the positive side at a predetermined speed in the processing of step ST11 in FIG. 5 and step ST21 in FIG. 6. When the vehicle V is traveling from the host vehicle driving lane toward the negative-side adjacent driving lane adjacent to the negative-side lane marking LN, the target steering amount calculation unit 3 preferably moves the neutral reference line L0 from the center in the width direction of the input image toward the negative side at a predetermined speed in the processing of step ST11 in FIG. 5 and step ST21 in FIG. 6.

[0099] In addition, in the above-described step ST11 of the total area calculation process and step ST21 of the one-side area calculation process, the neutral reference line L0 is set at the center of the width direction of the input image, assuming that the on-board camera C is installed at the center of the width direction of the vehicle body. However, the present invention is not limited to this. The on-board camera C may be installed at a position shifted from the center of the width direction of the vehicle body. In such a case, for example, when the vehicle V is traveling at the center of the width direction of a straight road, the target steering amount calculation unit 3 stores, as the center position, the position of the neutral reference line L0 in the input image that makes the positive side area value Sp and the negative side area value Sn defined as above equal. Thereafter, when the vehicle V is traveling along the host vehicle's driving lane, it is preferable that the target steering amount calculation unit 3 sets the neutral reference line L0 to the stored neutral position in step ST11 of the total area calculation process and step ST21 of the one-side area calculation process.

[0100] In addition, in the above-mentioned target steering amount calculation process, the target steering amount calculation unit 3 has been described as recognizing characteristic lines such as white lines and curbs extracted from the input image as positive side marking lines LP and negative side marking lines LN, but the present invention is not limited to this.

[0101] Fig. 9 is a diagram showing an example of an input image. Fig. 9 shows a case where a forward obstacle OBp (for example, a vehicle stopped at the road edge) that hides a part of the forward-side lane marking line LP is present at a position slightly forward as viewed from the vehicle V. In such a case, in order for the vehicle V to avoid the forward obstacle OBp, it is preferable to steer the vehicle V to the negative side.

[0102] In the processing of step ST1, when the target steering amount calculation unit 3 recognizes the front-side lane marking line LP and a front-side obstacle OBp that obscures a part of it based on the input image, it is preferable to interpolate the portion of the front-side lane marking line LP that is obscured by the front-side obstacle OBp (the portion shown by the dashed line in Fig. 9) with a line that follows the negative side of the front-side obstacle OBp, as shown by the thick dashed-dotted line in Fig. 9. As shown in Fig. 9, by interpolating the portion of the front-side lane marking line LP that is obscured by the front-side obstacle OBp with the thick dashed-dotted line, the front-side area value Sp can be made smaller than when interpolated with the dashed line, and therefore the vehicle V can be steered to the negative side so that the vehicle V avoids the front-side obstacle OBp.

[0103] Although not shown, the same applies when a negative-side obstacle that obscures a portion of the negative-side lane marking line LN is present in a position slightly to the negative side forward as viewed from the vehicle V. That is, when the target steering amount calculation unit 3 recognizes the negative-side lane marking line LN and the negative-side obstacle that obscures a portion of it based on the input image in the processing of step ST1, it is preferable that the portion of the negative-side lane marking line LP that is obscured by the negative-side obstacle be interpolated with a line along the positive side of the negative-side obstacle. This allows the vehicle V to be steered to the positive side so that the vehicle V avoids the negative-side obstacle.

[0104] Note that avoidance of such an obstacle that obscures the lane marking can also be achieved by shifting the neutral reference line L0. That is, if the target steering amount calculation unit 3 recognizes a forward-side lane marking LP and a forward-side obstacle OBp that obscures a portion of the lane marking LP based on the input image in the processing of step ST1, the target steering amount calculation unit 3 may shift the position of the neutral reference line L0 in the input image from the origin of the X-axis to the negative side in steps ST11 and ST21. This allows the vehicle V to be steered to the negative side so as to avoid the forward-side obstacle OBp. Also, if the target steering amount calculation unit 3 recognizes a negative-side lane marking LN and a backward-side obstacle that obscures a portion of the lane marking LP based on the input image in the processing of step ST1, the target steering amount calculation unit 3 may shift the position of the neutral reference line L0 in the input image from the origin of the X-axis to the positive side in steps ST11 and ST21. This allows the vehicle V to be steered to the positive side so as to avoid the backward-side obstacle.

[0105] In addition, in the above-mentioned target steering amount calculation process, the target steering amount calculation unit 3 executes the total area calculation process (step ST4) when both the positive and negative dividing lines LP and LN can be recognized from the input image, and executes the one-sided area calculation process (step ST5) when only one of the positive and negative dividing lines LP and LN can be recognized, but the present invention is not limited to this.

[0106] More specifically, if the target steering amount calculation unit 3 is able to recognize the positive-side lane marking LP but is unable to recognize the negative-side lane marking LN in the processing of step ST1, it may estimate the negative-side lane marking LN based on the shape of the recognized positive-side lane marking LP in the input image, and further use the recognized positive-side lane marking LP and the estimated negative-side lane marking LN to perform the total area calculation processing. Also, if the target steering amount calculation unit 3 is able to recognize the negative-side lane marking LN but is unable to recognize the positive-side lane marking LP in the processing of step ST1, it may estimate the positive-side lane marking LP based on the shape of the recognized negative-side lane marking LN in the input image, and further use the recognized negative-side lane marking LN and the estimated positive-side lane marking LP to perform the total area calculation processing.

[0107] 10 is a flowchart showing the specific steps of the target vehicle speed calculation process for calculating the target vehicle speed based on the input image. This target vehicle speed calculation process is executed by the target vehicle speed calculation unit 5 every time a new input image is acquired by the input image acquisition unit 2.

[0108] First, in step ST31, the target vehicle speed calculation unit 5 recognizes the vehicle's travel path based on the input image, and then proceeds to step ST32. Note that the procedure for the target vehicle speed calculation unit 5 to recognize the vehicle's travel path and the positive side lane marking LP and negative side lane marking LN corresponding to the boundary therebetween from the input image is the same as the procedure in the processing in the target steering amount calculation unit 3 (for example, step ST1), and therefore a detailed description thereof will be omitted.

[0109] In step ST32, the target vehicle speed calculation unit 5 determines whether or not the vehicle's traveling path has been recognized from the input image in step ST31. The procedure for determining whether or not the vehicle's traveling path has been recognized from the input image in the target vehicle speed calculation unit 5 is the same as the procedure in the process in the target steering amount calculation unit 3 (for example, step ST2), so a detailed description will be omitted. If the determination result in step ST32 is YES, the target vehicle speed calculation unit 5 proceeds to step ST33, and if the determination result in step ST32 is NO, the target vehicle speed calculation unit 5 proceeds to step ST37. Also, in step ST37, the target vehicle speed calculation unit 5 cancels the automatic acceleration / deceleration control by the acceleration / deceleration control unit 6, and ends the target vehicle speed calculation process of FIG. 10.

[0110] Next, in step ST33, the target vehicle speed calculation unit 5 determines whether both the positive side lane marking LP and the negative side lane marking LN were recognized. If the determination result in step ST33 is YES, i.e., if both the positive and negative lane markings LP and LN were recognized, the target vehicle speed calculation unit 5 executes a total area calculation process (see step ST35) described with reference to FIG. 11 to calculate the target vehicle speed according to the input image, and then proceeds to step ST36. If the determination result in step ST33 is NO, i.e., if only one of the positive and negative lane markings LP and LN was recognized, the target vehicle speed calculation unit 5 proceeds to step ST34.

[0111] Next, in step ST36, the target vehicle speed calculation unit 5 transmits the target vehicle speed calculated by executing the total area calculation process (step ST34) or the one-side area calculation process (step ST35) to the acceleration / deceleration control unit 6, and terminates the target vehicle speed calculation process of Figure 10.

[0112] 11 is a flowchart showing the specific steps of the total area calculation process in the target vehicle speed calculation unit 5. As described above, this total area calculation process is executed when both the positive and negative lane markings LP and LN can be recognized from the input image.

[0113] Fig. 12 is a diagram showing an example of an input image. In the following, a specific procedure for the total area calculation process of Fig. 11 will be described using the input image shown in Fig. 12 as an example.

[0114] First, in step ST41, the target vehicle speed calculation unit 5 virtually sets a speed control reference line LT3 on the input image that extends parallel to the X-axis (i.e., in the width direction of the input image), and then proceeds to step ST42. More specifically, the target vehicle speed calculation unit 5 virtually sets a lateral reference line setting section of a predetermined length SC on the Y-axis, and further sets a speed control reference line L3 at a position that divides this lateral reference line setting section vertically at a predetermined internal division ratio (in the example of FIG. 12, upper section length: lower section length = 2:3). In addition, at this time, the target vehicle speed calculation unit 5 preferably sets the lateral reference line setting section of length SC within a section on the Y-axis where both or either the positive side lane marking LP and the negative side lane marking LN are defined.

[0115] Next, in step ST42, the target vehicle speed calculation unit 5 sets multiple (five in the example of FIG. 12) lateral reference lines LT1, LT2, LT4, LT5, and LT6 in the input image within the above-mentioned lateral reference line setting section, which are parallel to the speed control reference line LT3 set in step ST41, and then proceeds to step ST43. Here, the target vehicle speed calculation unit 5 preferably sets the ratio between the number of lateral reference lines set in the upper section and the number of lateral reference lines set in the lower section of the lateral reference line setting section to be equal to the above-mentioned interior division ratio. Therefore, in the example shown in FIG. 12, two lateral reference lines LT1 and LT2 are set at positions on the positive side of the speed control reference line LT3 set in step ST41 along the Y axis, and three lateral reference lines LT4, LT5, and LT6 are set at positions on the negative side of the speed control reference line LT3 along the Y axis. In the following, a case will be described in which the target vehicle speed calculation unit 5 sets a plurality of lateral reference lines LT1, LT2, LT4, LT5, LT6 and a speed control reference line LT3 at equal intervals along the Y axis as shown in Fig. 12, but the present invention is not limited to this. Also, in the following, for convenience of calculation, the speed control reference line LT3 will also be referred to as the third lateral reference line.

[0116] In step ST43, the target vehicle speed calculation unit 5 determines the intersections of the multiple lateral reference lines LT1 to LT6 set in the input image in step ST42 with the positive-side lane marking LP and negative-side lane marking LN recognized from the input image as feature points, calculates the coordinate values ​​of these feature points, and proceeds to step ST44. Note that hereinafter, the intersections of the lateral reference lines LT1 to LT6 with the positive-side lane marking LP are referred to as positive-side feature points, and the intersections of the lateral reference lines LT1 to LT6 with the negative-side lane marking LN are referred to as negative-side feature points.

[0117] In the following, the coordinate value of the i-th (i is an integer from 1 to 6) positive side characteristic point (i.e., the intersection of the i-th horizontal reference line LTi and the positive side lane marking line LP) is expressed as (xp i ,yp i ), and the coordinate value of the i-th negative side characteristic point (i.e., the intersection of the i-th horizontal reference line LTi and the negative side lane marking line LN) is expressed as (xn i ,yn i ) is written as

[0118] In step ST44, the target vehicle speed calculation unit 5 calculates the area value of the area above the speed control reference line LT3 of the vehicle's driving path in the input image as an upper area value by using the coordinate values ​​of the multiple positive side feature points and negative side feature points calculated in step ST43, and proceeds to step ST45. More specifically, the target steering amount calculation unit 3 determines the area value of a region Rt surrounded by the positive side lane marking LP, the negative side lane marking LN, the speed control reference line LT3, and the multiple lateral reference lines LT1-LT2 in the input image as an upper area value St, and calculates this upper area value St by the quadrature by pieces as shown in the following equation (6).

number

[0119] In step ST45, the target vehicle speed calculation unit 5 calculates the area value of the area below the speed control reference line LT3 of the vehicle's driving path in the input image as a lower area value by using the coordinate values ​​of the multiple positive side feature points and negative side feature points calculated in step ST43, and proceeds to step ST46. More specifically, the target steering amount calculation unit 3 determines the area value of a region Rb surrounded by the positive side lane marking LP, the negative side lane marking LN, the speed control reference line LT3, and the multiple lateral reference lines LT4 to LT6 in the input image as a lower area value Sb, and calculates this lower area value Sb by the quadrature by pieces method as shown in the following equation (7).

number

[0120] In step ST46, the target vehicle speed calculation unit 5 calculates the sum of the upper area value St and the lower area value Sb as the total area value Stotal (=St+Sb), and then proceeds to step ST47.

[0121] In step ST47, the target vehicle speed calculation unit 5 calculates the ratio of the upper area value St to the total area value Stotal as an upper / lower ratio value r (=St / Stotal), and then proceeds to step ST48.

[0122] In step ST48, the target vehicle speed calculation unit 5 calculates the target vehicle speed Vtrgt based on the upper / lower ratio value r calculated by the above-mentioned procedure, and ends the total area calculation process in Fig. 11. More specifically, the target vehicle speed calculation unit 5 reads out a speed table that associates the upper / lower ratio value r with the target vehicle speed Vtrgt from a storage medium (not shown), and calculates the target vehicle speed Vtrgt according to the upper / lower ratio value r by searching the read speed table based on the upper / lower ratio value r calculated in step ST47.

[0123] Fig. 13 is a diagram showing an example of a speed table. According to the speed table shown in Fig. 13, the target vehicle speed calculation unit 5 calculates the target vehicle speed Vtrgt so that it decreases as the upper / lower ratio value r increases.

[0124] Here, since the on-board camera C captures the front side of the vehicle V from a position higher than the road surface, when the vehicle's road curves forward to the positive or negative side, the area of ​​the vehicle's road in the distance in the input image becomes larger than when the vehicle's road is straight. In other words, when the vehicle's road curves forward to the positive or negative side, the vertical ratio value r tends to be larger than when the vehicle's road is straight. Therefore, the target vehicle speed calculation unit 5 can calculate the target vehicle speed Vtrgt according to the vehicle's road with a simple calculation by calculating the target vehicle speed Vtrgt so that it decreases as the vertical ratio value r increases.

[0125] Returning to FIG. 10 , in step ST34, the target vehicle speed calculation unit 5 estimates the lane marking that could not be recognized, of the positive and negative lane markings LP and LN, based on the shape of the recognized lane marking in the input image, and proceeds to step ST35. More specifically, if the target vehicle speed calculation unit 5 can recognize the positive lane marking LP but cannot recognize the negative lane marking LN based on the input image, it estimates the negative lane marking LN based on the shape of the recognized positive lane marking LP in the input image. More specifically, the target vehicle speed calculation unit 5 estimates the negative lane marking LN so that it is parallel to the positive lane marking LP in a planar view. Furthermore, if the target vehicle speed calculation unit 5 can recognize the negative lane marking LN but cannot recognize the positive lane marking LP in the input image, it estimates the positive lane marking LP based on the shape of the recognized negative lane marking LN in the input image. More specifically, the target vehicle speed calculation unit 5 estimates the positive lane marking LP so that it is parallel to the negative lane marking LN in a planar view.

[0126] Next, the specific procedure for estimating either the positive or negative lane marking line LP, LN based on the shape of the other lane marking line in the input image in the target vehicle speed calculation unit 5 will be explained with reference to Figures 14 and 15.

[0127] Fig. 14 is a diagram showing an example of an input image. More specifically, Fig. 14 shows a case where the vehicle's roadway curves to the negative side. The following describes a case where the target vehicle speed calculation unit 5 is able to recognize the negative-side lane marking LN that curves to the negative side based on the input image shown in Fig. 14, but is unable to recognize the positive-side lane marking LP.

[0128] When the negative-side lane marking LN recognized from the input image is curved to the negative side, the target vehicle speed calculation unit 5 determines that the positive-side lane marking LP is a virtual line (see the thick dashed line in FIG. 14) that passes through two start points Ps and end points Pe defined in the input image. In this case, the start point Ps of the virtual line is set, for example, to a position obtained by inverting the sign of the X-axis component of the coordinate value of the bottom end of the recognized negative-side lane marking LN in the input image. The end point Pe of the virtual line is set, for example, to the origin of the input image.

[0129] While the above description of a case where the positive-side lane marking LP cannot be recognized has been given with reference to FIG. 14, the same applies when the negative-side lane marking LN cannot be recognized. That is, when the positive-side lane marking LP recognized from the input image curves toward the positive side, the target vehicle speed calculation unit 5 determines that the negative-side lane marking LN is a virtual line passing through two start points Ps and end points Pe defined in the input image. In this case, the start point Ps of the virtual line is set, for example, to a position obtained by inverting the sign of the X-axis component of the coordinate value of the bottom edge of the recognized positive-side lane marking LP in the input image. The end point Pe of the virtual line is set, for example, to the origin of the input image.

[0130] Fig. 15 is a diagram showing an example of an input image. More specifically, Fig. 15 shows a case where the vehicle's travel path is curving to the positive side. The following describes a case where the target vehicle speed calculation unit 5 is able to recognize the negative-side lane marking LN curving to the positive side based on the input image shown in Fig. 15, but is unable to recognize the positive-side lane marking LP.

[0131] When the negative-side lane marking LN recognized from the input image is curved toward the positive side, the target vehicle speed calculation unit 5 determines a virtual line (see the thick dashed line in FIG. 15) that passes through two start points Ps and end points Pe defined in the input image as the positive-side lane marking LP. In this case, the start point Ps of the virtual line is set, for example, at a position obtained by inverting the sign of the X-axis component of the coordinate value of the bottom end of the recognized negative-side lane marking LN in the input image. The end point Pe of the virtual line is set, for example, at the same position as the top end of the recognized negative-side lane marking LN in the input image.

[0132] While the above description of a case where the positive-side lane marking LP cannot be recognized has been given with reference to FIG. 15, the same applies when the negative-side lane marking LN cannot be recognized. That is, when the positive-side lane marking LP recognized from the input image is curved toward the negative side, the target vehicle speed calculation unit 5 determines that the negative-side lane marking LN is a virtual line passing through two start points Ps and end points Pe defined in the input image. In this case, the start point Ps of the virtual line is set, for example, to a position obtained by inverting the sign of the X-axis component of the coordinate value of the bottom end of the recognized positive-side lane marking LP in the input image. Furthermore, the end point Pe of the virtual line is set, for example, to the same position as the top end of the recognized positive-side lane marking LP in the input image.

[0133] Returning to Figure 10, in step ST34, if the target vehicle speed calculation unit 5 is unable to recognize either the positive or negative lane marking lines LP, LN, it estimates the lane marking that could not be recognized using the above-mentioned procedure, and uses this estimated lane marking to perform the above-mentioned total area calculation process (see step ST35).

[0134] The specific procedure for the total area calculation process in the target vehicle speed calculation unit 5 according to this embodiment has been described above, but the present invention is not limited to this. For example, in step ST48, the target vehicle speed calculation unit 5 calculates the target vehicle speed based on a single speed table as shown in Fig. 13, but the present invention is not limited to this. For example, a storage medium may store a plurality of speed tables that associate the upper / lower ratio value with the target vehicle speed, and the target vehicle speed calculation unit 5 may switch between the speed tables depending on various conditions.

[0135] For example, the target vehicle speed calculation unit 5 may estimate the curvature parameters of the vehicle's road ahead of the vehicle V based on the input image in accordance with the procedure described below, select one from a plurality of speed tables based on the estimated curvature parameters, and calculate the target vehicle speed based on the selected speed table and the up / down ratio value.

[0136] Fig. 16 is a diagram showing an example of an input image. Fig. 16 shows a case where the road on which the vehicle is traveling is curving forward. The procedure for calculating the curvature parameter in target vehicle speed calculation unit 5 will be described below with reference to Fig. 16.

[0137] The target vehicle speed calculation unit 5 calculates the coordinate values ​​of a plurality of average feature points as indicated by square marks in Fig. 16 by using the coordinate values ​​of the plurality of positive side feature points and negative side feature points acquired in step ST43. Here, the coordinate value xmid along the X axis of the jth (j is an integer equal to or greater than 1) average feature point counted sequentially from the bottom in the input image is j is defined by the following formula (8). In the following formula (8), "xp j " is the coordinate value along the X axis of the j-th positive feature point counting from the bottom in the input image, and "xn j " is the coordinate value along the X axis of the j-th negative feature point counting from the bottom in the input image.

number

[0138] Next, the target vehicle speed calculation unit 5 virtually sets a reference line (see the dashed line in FIG. 16 ) that passes through the first and second average feature points (i.e., the two average feature points that are closest to the vehicle V in a planar view) for the input image. The target vehicle speed calculation unit 5 also calculates the angle of each average feature point with respect to the reference line. Here, the angle θj of the jth average feature point with respect to the reference line refers to the angle of a line that passes through the jth average feature point and the first average feature point with respect to the reference line, as shown in FIG. 16 . The target vehicle speed calculation unit 5 also calculates a curvature parameter θall by calculating the sum of the absolute values ​​of the angles with respect to the reference line that are calculated for all average feature points, as shown in the following equation (9). This makes it possible to calculate the curvature parameter, which increases in proportion to the curvature of the road on which the vehicle is traveling, from the input image through a simple calculation.

number

[0139] In addition, if multiple driving modes that can be specified by the occupant for the vehicle V are defined, the target vehicle speed calculation unit 5 may acquire the driving mode specified by the occupant, select one from multiple speed tables based on the acquired driving mode, and calculate the target vehicle speed based on the selected speed table and the upper / lower ratio value.

[0140] The vehicle control device 1 according to this embodiment provides the following effects. (1) In the vehicle control device 1, the input image acquisition unit 2 acquires as an input image an image captured by an onboard camera C facing forward as viewed from the vehicle V, the target steering amount calculation unit 3 calculates a target steering amount for the electric power steering device 9 of the vehicle V based on the input image, and the steering control unit 4 operates the electric power steering device 9 to the positive or negative side based on the target steering amount. Furthermore, in the vehicle control device 1, the target steering amount calculation unit 3 recognizes the vehicle's driving path based on the input image and calculates at least one of a positive-side area value between the neutral reference line and the positive-side lane marking of the vehicle's driving path in the input image and a negative-side area value between the neutral reference line and the negative-side lane marking of the vehicle's driving path. Here, the positive-side area value tends to increase in proportion to the distance in the width direction between the neutral lane marking and the positive-side lane marking, and the negative-side area value tends to increase in proportion to the distance in the width direction between the neutral lane marking and the negative-side lane marking. That is, when moving the vehicle V along the host vehicle roadway, it is necessary to operate the electric power steering device 9 in the positive or negative direction so that the distance between the neutral lane marking and the positive or negative lane marking, i.e., the positive area value and the negative area value, remain approximately constant. Furthermore, when the host vehicle roadway ahead as viewed from the vehicle V curves toward the positive side, i.e., when the electric power steering device 9 needs to be gradually operated toward the positive side, the distance along the width direction between the neutral lane marking and the positive lane marking increases as the vehicle moves away from the vehicle V, and therefore the positive area value increases. Furthermore, when the host vehicle roadway ahead as viewed from the vehicle V curves toward the negative side, i.e., when the electric power steering device 9 needs to be gradually operated toward the negative side, the distance along the width direction between the neutral lane marking and the negative lane marking increases as the vehicle moves away from the vehicle V, and therefore the negative area value increases. In this way, the steering amount when traveling the vehicle V along the host vehicle roadway is correlated with the positive area value and the negative area value defined above. Therefore, the target steering amount calculation unit 3 calculates the target steering amount by utilizing the correlation between such steering amount and the area value in the input image, thereby making it possible to move the vehicle V along the vehicle travel path. In this way, the vehicle control device 1 can calculate an appropriate target steering amount simply by calculating the area value from the input image, thereby reducing the load on the computer that performs such calculations.

[0141] Furthermore, in the input image captured by the on-board camera C, the area of ​​objects that are far from the vehicle V appears smaller than the area of ​​objects that are in front of the vehicle V. Therefore, it can be said that the positive side area value and negative side area value calculated in the above manner are naturally weighted without setting parameters. Therefore, by calculating the target steering amount based on such positive side area value and negative side area value, it is possible to reduce the number of parameters that require tuning and the load on the computer compared to conventional methods, which can ultimately contribute to the development of sustainable transportation systems.

[0142] (2) The target steering amount calculation unit 3 calculates the target steering amount so that it increases toward the positive side as the positive side area value increases, and so that it increases toward the negative side as the negative side area value increases. This allows the vehicle V to move without deviating from the vehicle's driving path.

[0143] (3) The target steering amount calculation unit 3 calculates the area value of the region surrounded by the neutral reference line extending in the up-down direction in the input image, the multiple lateral reference lines intersecting with the neutral reference line, and the positive-side demarcation line as the positive-side area value, and calculates the area value of the region surrounded by the neutral reference line, the multiple lateral reference lines, and the negative-side demarcation line as the negative-side area value. Therefore, the vehicle control device 1 can calculate the positive-side area value and the negative-side area value with a simple calculation.

[0144] (4) The target steering amount calculation unit 3 calculates the target steering amount by subtracting a predetermined negative reference value from the positive area value, subtracting the negative area value from the predetermined positive reference value, or subtracting the negative area value from the positive area value. Therefore, according to the vehicle control device 1, the target steering amount can be calculated if at least one of the positive and negative lane markings can be recognized from the input image.

[0145] (5) When the positive side lane marking line and the neutral reference line intersect in the input image, i.e., when the positive side lane marking line is curved significantly to the negative side in front of the vehicle V, the target steering amount calculation unit 3 calculates the positive side area value by subtracting the area value between the portion of the neutral reference line above the intersection point with the positive side lane marking line (i.e., the farther side as viewed from the vehicle V) and the positive side lane marking line from the area value between the portion of the neutral reference line below the intersection point with the positive side lane marking line (i.e., the closer side as viewed from the vehicle V) and the positive side lane marking line. This makes it possible to calculate the positive side area value taking into account the curvature of the road on which the vehicle is traveling.

[0146] (6) When the negative-side lane marking and the neutral reference line intersect in the input image, i.e., when the negative-side lane marking is curved significantly to the positive side in front of the vehicle V, the target steering amount calculation unit 3 calculates the negative-side area value by subtracting the area value between the portion of the neutral reference line above the intersection point with the negative-side lane marking (i.e., the farther side as viewed from the vehicle V) and the negative-side lane marking from the area value between the portion of the neutral reference line below the intersection point with the negative-side lane marking (i.e., the closer side as viewed from the vehicle V) and the negative-side lane marking. This makes it possible to calculate the negative-side area value taking into account the curvature of the lane on which the vehicle is traveling.

[0147] (7) The target steering amount calculation unit 3 calculates the target steering amount by dividing the difference between the positive area value and the negative reference value by the sum of the positive area value and the negative reference value, dividing the difference between the positive reference value and the negative area value by the sum of the positive area value and the negative area value, or dividing the difference between the positive area value and the negative area value by the sum of the positive area value and the negative area value. That is, the vehicle control device 1 can improve robustness by normalizing the positive area value or the negative area value to calculate the target steering amount.

[0148] (8) The on-board camera C is provided at the widthwise center of the vehicle body. When the vehicle V is driven along the host vehicle driving lane between the positive and negative lane markings, the target steering amount calculation unit 3 sets the neutral reference line at the widthwise center of the input image. This allows the vehicle V to be driven so that the neutral reference line is positioned in the center between the positive and negative lane markings in the input image, in other words, so that the vehicle body is positioned in the center of the host vehicle driving lane.

[0149] (9) When the vehicle V is traveling in the center of the width direction of a straight road, the target steering amount calculation unit 3 stores the position of the neutral reference line L0 that makes the positive side area value Sp and the negative side area value Sn equal as the center position. Furthermore, when the vehicle V is traveling along the host vehicle driving lane, the target steering amount calculation unit 3 sets the neutral reference line L0 to the center position. This allows the vehicle V to travel so that the vehicle body is positioned in the center of the host vehicle driving lane even if the on-board camera C is not installed in the center of the vehicle body in the width direction.

[0150] (10) When the vehicle V is caused to travel from the host vehicle driving lane toward the adjacent driving lane on the positive side, the target steering amount calculation unit 3 moves the neutral reference line in the input image toward the positive side at a predetermined speed. This allows the vehicle V to automatically move from the host vehicle driving lane toward the adjacent driving lane on the positive side. Furthermore, when the vehicle V is caused to travel from the host vehicle driving lane toward the adjacent driving lane on the negative side, the target steering amount calculation unit 3 moves the neutral reference line in the input image toward the negative side at a predetermined speed. This allows the vehicle V to automatically move from the host vehicle driving lane toward the adjacent driving lane on the negative side.

[0151] (11) When the target steering amount calculation unit 3 recognizes a forward-side lane marking and a forward-side obstacle that partially obscures the lane marking based on the input image, the target steering amount calculation unit 3 moves the neutral reference line to the negative side in the input image. This allows the vehicle V to avoid the forward-side obstacle. Furthermore, when the target steering amount calculation unit 3 recognizes a negative-side lane marking and a backward-side obstacle that partially obscures the lane marking based on the input image, the target steering amount calculation unit 3 moves the neutral reference line to the positive side in the input image. This allows the vehicle V to avoid the backward-side obstacle.

[0152] (12) If the target steering amount calculation unit 3 can recognize the positive side lane marking but cannot recognize the negative side lane marking based on the input image, it estimates the negative side lane marking based on the positive side lane marking. Furthermore, if the target steering amount calculation unit 3 can recognize the negative side lane marking but cannot recognize the positive side lane marking based on the input image, it estimates the positive side lane marking based on the negative side lane marking. Therefore, even if the vehicle control device 1 cannot recognize either the positive side lane marking or the negative side lane marking, it is possible to calculate both the positive side area value and the negative side area value, and ultimately to calculate the target steering amount using both the positive side area value and the negative side area value.

[0153] (13) When the target steering amount calculation unit 3 recognizes a front-side lane marking line and a front-side obstacle that hides part of it based on the input image, it interpolates the part of the front-side lane marking line that is hidden by the front-side obstacle with a line along the negative-side side of the front-side obstacle. This makes it possible to reduce the positive-side area value by the amount of the front-side obstacle, allowing the vehicle V to avoid the front-side obstacle. Furthermore, when the target steering amount calculation means recognizes a negative-side lane marking line and a front-side obstacle that hides part of it based on the input image, it interpolates the part of the negative-side lane marking line that is hidden by the negative-side obstacle with a line along the positive-side side of the negative-side obstacle. This makes it possible to reduce the negative-side area value by the amount of the negative-side obstacle, allowing the vehicle V to avoid the front-side obstacle.

[0154] (14) The target vehicle speed setting unit 5 calculates the target vehicle speed based on the input image, and the acceleration / deceleration control unit 6 operates the power plant 8 and the braking device 7 based on the target vehicle speed. The target vehicle speed setting unit 5 also recognizes the vehicle's roadway based on the input image, sets a speed control reference line extending in the width direction of the input image, calculates the area of ​​the vehicle's roadway above the speed control reference line as an upper area value, calculates the area of ​​the vehicle's roadway below the speed control reference line as a lower area value, calculates the sum of these area values ​​as a total area value, calculates the ratio of the upper area value to the total area value as an upper / lower ratio value, and further calculates the target vehicle speed based on the upper / lower ratio value. Here, when the vehicle's roadway ahead of the vehicle V curves to the positive or negative side, i.e., when the vehicle speed needs to be reduced, the upper / lower ratio value is larger than when the roadway is straight. Therefore, the vehicle control device 1 can control the vehicle speed with simple calculations by utilizing the correlation between an appropriate vehicle speed according to the roadway conditions ahead and the upper / lower ratio value defined above.

[0155] (15) The target vehicle speed calculation unit 5 calculates the target vehicle speed so that it decreases as the upper / lower ratio value increases. This allows the vehicle speed to be changed according to the condition of the road ahead of the vehicle V.

[0156] (16) The target vehicle speed calculation unit 5 estimates a curvature parameter of the road on which the vehicle is traveling based on the input image, selects one of a plurality of speed tables based on the curvature parameter, and calculates the target vehicle speed based on the selected speed table and the upper / lower ratio value. Thus, the vehicle control device 1 can change the vehicle speed according to the curvature parameter of the road on which the vehicle is traveling.

[0157] (17) The target vehicle speed calculation unit 5 acquires the driving mode of the vehicle V, selects one from a plurality of speed tables based on the driving mode, and calculates the target vehicle speed based on the selected speed table and the up / down ratio value. Therefore, the vehicle control device 1 can change the vehicle speed according to the curvature parameter of the road on which the vehicle is traveling.

[0158] (18) If the target vehicle speed calculation unit 5 can recognize the positive side lane marking but cannot recognize the negative side lane marking based on the input image, it estimates the negative side lane marking based on the positive side lane marking. Furthermore, if the target vehicle speed calculation unit 5 can recognize the negative side lane marking but cannot recognize the positive side lane marking based on the input image, it estimates the positive side lane marking based on the negative side lane marking. Therefore, even if the vehicle control device 1 cannot recognize either the positive side lane marking or the negative side lane marking, it is possible to calculate the upper area value, lower area value, upper / lower ratio value, etc., and ultimately calculate the target vehicle speed using these upper area value, lower area value, upper / lower ratio value, etc.

[0159] Second Embodiment Next, a vehicle control device according to a second embodiment of the present invention will be described with reference to the drawings. The vehicle control device according to this embodiment differs from the vehicle control device 1 according to the first embodiment in the configuration of the target vehicle speed calculation unit.

[0160] 17 is a flowchart showing a specific procedure of the target vehicle speed calculation process in the target vehicle speed calculation unit according to this embodiment. This target vehicle speed calculation process is executed by the target vehicle speed calculation unit every time a new input image is acquired by the input image acquisition unit 2.

[0161] Fig. 18 is a diagram showing an example of an input image. In the following, a specific procedure of the target vehicle speed calculation process of Fig. 17 will be described using the input image shown in Fig. 18 as an example.

[0162] First, in step ST51, the target vehicle speed calculation unit recognizes the vehicle's travel path based on the input image, and then proceeds to step ST52. Note that the procedure for the target vehicle speed calculation unit to recognize the vehicle's travel path and the positive side lane marking LP and negative side lane marking LN corresponding to its boundaries from the input image is the same as the procedure in the processing in the target steering amount calculation unit 3 (for example, step ST1), and therefore a detailed description thereof will be omitted.

[0163] In step ST52, the target vehicle speed calculation unit determines whether or not a leading vehicle VF that is set as a follow-up target for the vehicle V is present on the vehicle's travel path recognized in step ST51. If the determination result in step ST52 is YES, the target vehicle speed calculation unit proceeds to step ST53, and if the determination result in step ST52 is NO, the target vehicle speed calculation unit proceeds to step ST58. Also, in step ST58, the target vehicle speed calculation unit cancels the automatic acceleration / deceleration control by the acceleration / deceleration control unit 6, and ends the target vehicle speed calculation process of FIG. 17.

[0164] Next, in step ST53, the target vehicle speed calculation unit calculates the area value of the vehicle-in-front VF in the input image acquired in the current control cycle. Note that in Figure 18, the vehicle-in-front VF in the input image acquired in the previous control cycle is shown by a dashed line.

[0165] Next, in step ST54, the target vehicle speed calculation unit calculates the area change rate dS(t) of the vehicle in front VF based on the following equation (10), and proceeds to step ST55. In the following equation (10), "S(t)" indicates the area value of the vehicle in front VF calculated in the current control cycle (i.e., the area value of the solid line portion in FIG. 18), "S(t-1)" indicates the area value of the vehicle in front VF calculated in the previous control cycle (i.e., the area value of the dashed line portion in FIG. 18), and "Δt" indicates the control cycle.

number

[0166] Next, in step ST55, the target vehicle speed calculation unit calculates a speed change amount dv(t) based on the area value S(t) and area change rate dS(t) of the vehicle-in-front VF calculated in step ST53, and then proceeds to step ST56. More specifically, the target vehicle speed calculation unit reads out a predetermined speed change amount table from a storage medium (not shown), and calculates the speed change amount dv(t) by searching the read speed change amount table based on the area value S(t) and area change rate dS(t) of the vehicle-in-front VF. Note that this speed change amount table is set so that the area value S(t) of the vehicle-in-front VF in the input image is maintained approximately constant when the target vehicle speed is calculated according to equation (11) described below.

[0167] Next, in step ST56, the target vehicle speed calculation unit calculates the target vehicle speed v(t) by adding the speed change amount dv(t) calculated in step ST55 to the target vehicle speed v(t-1) in the previous control cycle as shown in the following equation (11), and then proceeds to step ST57. Here, if the target vehicle speed v(t) calculated according to the following equation (11) is a negative value, the target vehicle speed calculation unit sets the target vehicle speed v(t) to 0. Furthermore, if the target vehicle speed v(t) calculated according to the following equation (11) is greater than a predetermined vehicle speed upper limit vlim, the target vehicle speed calculation unit sets the target vehicle speed v(t) to the vehicle speed upper limit vlim.

number

[0168] Next, in step ST57, the target vehicle speed calculation section transmits the calculated target vehicle speed to the acceleration / deceleration control section 6, and the target vehicle speed calculation process shown in FIG. 17 ends.

[0169] According to the target vehicle speed calculation process of this embodiment as described above, the target vehicle speed can be calculated so that the area value of the vehicle in front VF in the input image is maintained approximately constant. Here, the area value of the vehicle in front VF in the input image is proportional to the inter-vehicle distance between the vehicle V and the vehicle in front VF. Therefore, according to this embodiment, the vehicle V can be made to follow the vehicle in front VF by simple calculations using the input image.

[0170] <Third embodiment> Next, an aircraft control device according to a third embodiment of the present invention will be described with reference to the drawings.

[0171] FIG. 19 is a diagram showing a schematic configuration of an aircraft F equipped with an aircraft control device 1A according to this embodiment. FIG. 20 is a functional block diagram of the aircraft control device 1A.

[0172] The flying object F is, for example, a drone, an airplane, a helicopter, a vertical take-off and landing aircraft, etc. The flying object F includes a propulsion mechanism (not shown) that propels the airframe along a propulsion direction parallel to the roll axis Or, a camera CA that is mounted on the airframe facing forward along this propulsion direction, a yaw axis attitude control mechanism 9Y that changes the attitude of the airframe about the yaw axis Oy during flight, a pitch axis attitude control mechanism 9P that changes the attitude of the airframe about the pitch axis Op during flight, a roll axis control mechanism 9R that changes the attitude of the airframe about the roll axis Or during flight, and an airframe control device 1A that controls the yaw axis attitude control mechanism 9Y, pitch axis attitude control mechanism 9P, and roll axis attitude control mechanism 9R based on images captured by the camera CA.

[0173] 19, the roll axis Or extends parallel to the propulsion direction of the aircraft, and the yaw axis Oy and pitch axis Op extend in a plane perpendicular to the roll axis Or. The yaw axis Oy extends parallel to the up-down direction of the aircraft, and the pitch axis Op extends parallel to the width direction of the aircraft, which is perpendicular to the yaw axis Oy.

[0174] The yaw axis attitude control mechanism 9Y changes the attitude of the aircraft around the yaw axis Oy during flight in response to a control signal transmitted from the aircraft control device 1 A. More specifically, the yaw axis attitude control mechanism 9Y rotates the aircraft during flight around the yaw axis Oy in a positive direction (for example, clockwise when viewed from above) or a negative direction (for example, counterclockwise when viewed from above).

[0175] The pitch axis attitude control mechanism 9P changes the attitude of the aircraft around the pitch axis Op during flight in response to a control signal transmitted from the aircraft control device 1 A. More specifically, the pitch axis attitude control mechanism 9P rotates the aircraft during flight around the pitch axis Op in a positive direction (for example, clockwise when viewed from the right) or a negative direction (for example, counterclockwise when viewed from the right).

[0176] The roll axis attitude control mechanism 9R changes the attitude of the airframe around the roll axis Or during flight in response to a control signal transmitted from the aircraft control device 1 A. More specifically, the roll axis attitude control mechanism 9R rotates the airframe during flight around the roll axis Or in the positive direction (for example, clockwise when viewed from the rear) or the negative direction (for example, counterclockwise when viewed from the rear).

[0177] The camera CA faces forward along the propulsion direction as viewed from the aircraft F. In this embodiment, the camera CA is described as being installed in the center of the aircraft F's body in the width direction, but the present invention is not limited to this.

[0178] The aircraft control device 1A controls the yaw axis attitude control mechanism 9Y, pitch axis attitude control mechanism 9P, and roll axis attitude control mechanism 9R based on an image of the front side of the aircraft F captured by the camera CA. The aircraft control device 1A is a computer configured with hardware such as a processing means such as a CPU, auxiliary storage means such as an HDD or SSD that stores programs that cause the processing means to execute processes such as a yaw axis target control amount calculation process, a pitch axis target control amount calculation process, and a roll axis target control amount calculation process, which will be described later, and a main storage means such as a RAM for storing data temporarily required for the processing means to execute the programs.

[0179] As shown in FIG. 20, the aircraft control device 1A has the above-described hardware configuration, which includes an input image acquisition unit 2A, a yaw axis target control amount calculation unit 31, a pitch axis target control amount calculation unit 32, a roll axis target control amount calculation unit 33, a yaw axis attitude control unit 41, a pitch axis attitude control unit 42, and a roll axis attitude control unit 43.

[0180] The input image acquisition unit 2A acquires an image of the front side of the aircraft F captured by the camera CA as an input image. The input image acquisition unit 2A transmits the acquired input image to the yaw axis target control amount calculation unit 31, the pitch axis target control amount calculation unit 32, and the roll axis target control amount calculation unit 33.

[0181] The yaw axis target control amount calculation unit 31 calculates a yaw axis target control amount for the yaw axis control amount of the yaw axis attitude control mechanism 9Y, based on the input image transmitted from the input image acquisition unit 2A. The procedure for calculating the yaw axis target control amount in the yaw axis target control amount calculation unit 31 based on the input image will be described later with reference to Figures 21 and 22, etc. The yaw axis attitude control unit 41 executes automatic attitude control to operate the yaw axis attitude control mechanism 9Y positively or negatively, based on the yaw axis target control amount calculated by the yaw axis target control amount calculation unit 31.

[0182] The pitch axis target control amount calculation unit 32 calculates a pitch axis target control amount for the pitch axis control amount of the pitch axis attitude control mechanism 9P, based on the input image transmitted from the input image acquisition unit 2A. The procedure for calculating the pitch axis target control amount based on the input image in the pitch axis target control amount calculation unit 32 will be described later with reference to Figures 23 and 24, etc. The pitch axis attitude control unit 42 executes automatic attitude control to operate the pitch axis attitude control mechanism 9P positively or negatively, based on the pitch axis target control amount calculated by the pitch axis target control amount calculation unit 32.

[0183] The roll axis target control amount calculation unit 33 calculates a roll axis target control amount for the roll axis control amount of the roll axis attitude control mechanism 9R based on the input image transmitted from the input image acquisition unit 2A. The procedure for calculating the roll axis target control amount based on the input image in the roll axis target control amount calculation unit 33 will be described later with reference to Figures 25 and 26, etc. The roll axis attitude control unit 43 executes automatic attitude control to operate the roll axis attitude control mechanism 9R to the positive side or negative side based on the roll axis target control amount calculated by the roll axis target control amount calculation unit 33.

[0184] 21 is a flowchart showing the specific steps of the yaw axis target control amount calculation process for calculating the yaw axis target control amount based on an input image. This yaw axis target control amount calculation process is executed by the yaw axis target control amount calculation unit 31 every time a new input image is acquired by the input image acquisition unit 2A.

[0185] Fig. 22 is a diagram showing an example of an input image. In the following, a specific procedure for the yaw axis target control amount calculation process of Fig. 21 will be described using the input image shown in Fig. 22 as an example.

[0186] First, in step ST61, the yaw axis target control amount calculation unit 31 recognizes the target T to be followed by the flying object F based on the input image, and then proceeds to step ST62.

[0187] Next, in step ST62, the yaw axis target control amount calculation unit 31 determines whether or not the target object T was recognized from the input image in step ST61. If the determination result in step ST62 is YES, the yaw axis target control amount calculation unit 31 proceeds to step ST63, and if the determination result in step ST62 is NO, the yaw axis target control amount calculation unit 31 proceeds to step ST69. Also, in step ST69, the yaw axis target control amount calculation unit 31 cancels the automatic attitude control by the yaw axis attitude control unit 41, and ends the yaw axis target control amount calculation process of FIG.

[0188] Next, in step ST63, the yaw-axis target control amount calculation unit 31 sets a virtual yaw-axis control reference line Ly in the input image, and then proceeds to step ST64. Here, it is preferable that the yaw-axis target control amount calculation unit 31 sets the yaw-axis control reference line Ly at the origin of the X-axis in the input image (i.e., the center in the width direction of the input image) so that the line is parallel to the Y-axis in the input image, as shown in Fig. 22.

[0189] Next, in step ST64, the yaw axis target control amount calculation unit 31 virtually sets a plurality of flight lane lines LF1, LF2, LF3, and LF4 extending radially from the target T based on the position and attitude of the target T in the input image, thereby setting a virtual flight path for the input image, and proceeds to step ST65.

[0190] More specifically, as shown in Fig. 22, the yaw axis target control amount calculation unit 31 sets a plurality of lines connecting a plurality of reference points P1, P2, P3, and P4 defined at the four corners of the input image with the target T to be tracked as flight lane lines LF1, LF2, LF3, and LF4 that demarcate the flight path of the aircraft F. As shown in Fig. 22, the first reference point P1 is set, for example, at the extreme end of the positive side of the X axis and the extreme end of the positive side of the Y axis, the second reference point P2 is set, for example, at the extreme end of the positive side of the X axis and the extreme end of the negative side of the Y axis, the third reference point P3 is set, for example, at the extreme end of the negative side of the X axis and the extreme end of the negative side of the Y axis, and the fourth reference point P4 is set, for example, at the extreme end of the positive side of the X axis and the extreme end of the positive side of the Y axis.

[0191] Next, in step ST65, the yaw axis target control amount calculation unit 31 calculates the upper positive side area value Sypt, the upper negative side area value Synt, the lower positive side area value Sypb, and the lower negative side area value Synb as the area values ​​of the multiple regions Rypt, Rynt, Rypb, Rynb defined between the yaw axis control reference line Ly and the multiple flight lane lines LF1 to LF4 in the input image, and proceeds to step ST66.

[0192] More specifically, the yaw-axis target control amount calculation unit 31 calculates the area value of the region Rypt between the yaw-axis control reference line Ly and the first lane marking LF1 in the input image as an upper positive area value Sypt, calculates the area value of the region Rynt between the yaw-axis control reference line Ly and the fourth lane marking LF4 in the input image as an upper negative area value Synt, calculates the area value of the region Rypb between the yaw-axis control reference line Ly and the second lane marking LF2 in the input image as a lower positive area value Sypb, and calculates the area value of the region Rynb between the yaw-axis control reference line Ly and the third lane marking LF3 in the input image as a lower negative area value Synb. The yaw-axis target control amount calculation unit 31 calculates these area values ​​Sypt, Synt, Sypb, and Synb based on, for example, the quadrature by pieces method described in steps ST12 to ST15. In the following, the sum of the upper positive side area value Sypt and the lower positive side area value Sypb will be referred to as the positive side area value Syp (= Sypt + Sypb), and the sum of the upper negative side area value Synt and the lower negative side area value Synb will be referred to as the negative side area value Syn (= Synt + Synb).

[0193] Next, in step ST66, the yaw axis target control amount calculation unit 31 calculates the yaw axis target control amount uy based on the positive area value Syp and the negative area value Syn calculated by the above-described procedure, and then proceeds to step ST67. More specifically, the yaw axis target control amount calculation unit 31 calculates the yaw axis target control amount uy so that the yaw axis target control amount uy increases toward the positive side as the positive area value Syp increases, and increases toward the negative side as the negative area value Syn increases. As a result, the larger the positive area value Syp, the more the attitude of the aircraft around the yaw axis during flight can be changed toward the positive side (i.e., clockwise when viewed from above). Also, the larger the negative area value Syn, the more the attitude of the aircraft around the yaw axis during flight can be changed toward the negative side (i.e., counterclockwise when viewed from above).

[0194] More specifically, the yaw axis target control amount calculation unit 31 calculates the yaw axis target control amount uy by dividing the difference (Syp-Syn) between the positive side area value Syp and the negative side area value Syn by the sum (Syp+Syn) of the positive side area value Syp and the negative side area value Syn, as shown in the following equation (12). The following description will be given assuming that the gain Gain_y in the following equation (12) is a positive constant, but the present invention is not limited to this. The gain Gain_y may also be a negative value. The value of this gain Gain_y may also be changed depending on some parameter such as the speed of the aircraft.

number

[0195] In step ST67, the yaw axis target control amount calculation unit 31 transmits the yaw axis target control amount uy calculated by the above procedure to the yaw axis attitude control unit 41, and ends the yaw axis target control amount calculation process in FIG.

[0196] 23 is a flowchart showing a specific procedure for the pitch axis target control amount calculation process for calculating the pitch axis target control amount based on an input image. This pitch axis target control amount calculation process is executed by the pitch axis target control amount calculation unit 32 every time a new input image is acquired by the input image acquisition unit 2A.

[0197] Fig. 24 is a diagram showing an example of an input image. In the following, a specific procedure of the pitch axis target control amount calculation process of Fig. 23 will be described using the input image shown in Fig. 24 as an example.

[0198] First, in step ST71, the pitch axis target control amount calculation unit 32 recognizes the target T to be followed by the flying object F based on the input image, and then proceeds to step ST72.

[0199] Next, in step ST72, the pitch axis target control amount calculation unit 32 determines whether or not the target T to be tracked was recognized from the input image in step ST71. If the determination result in step ST72 is YES, the pitch axis target control amount calculation unit 32 proceeds to step ST73, and if the determination result in step ST72 is NO, the pitch axis target control amount calculation unit 32 proceeds to step ST79. Also, in step ST79, the pitch axis target control amount calculation unit 32 cancels the automatic attitude control by the pitch axis attitude control unit 42, and ends the pitch axis target control amount calculation process in FIG.

[0200] Next, in step ST73, the pitch axis target control amount calculation unit 32 sets a virtual pitch axis control reference line Lp in the input image, and then proceeds to step ST74. Here, it is preferable that the pitch axis target control amount calculation unit 32 sets the pitch axis control reference line Lp so that it is perpendicular to the yaw axis control reference line Ly in the input image, as shown in Fig. 24.

[0201] Next, in step ST74, the pitch axis target control amount calculation unit 32 sets a virtual flight path for the input image by virtually setting a plurality of flight lane lines LF1, LF2, LF3, and LF4 extending radially from the target T using the same procedure as in step ST64, and then proceeds to step ST75.

[0202] Next, in step ST75, the pitch axis target control amount calculation unit 32 calculates the right positive area value Sppr, the right negative area value Spnr, the left positive area value Sppl, and the left negative area value Spnl as the area values ​​of the multiple regions Rppr, Rpnr, Rppl, and Rpnl defined between the pitch axis control reference line Lp and the multiple flight lane lines LF1 to LF4 in the input image, and then proceeds to step ST76.

[0203] More specifically, the pitch axis target control amount calculation unit 32 calculates the area value of the region Rppr between the pitch axis control reference line Lp and the second lane marking LF2 in the input image as a right positive area value Sppr, calculates the area value of the region Rpnr between the pitch axis control reference line Lp and the first lane marking LF1 in the input image as a right negative area value Spnr, calculates the area value of the region Rppl between the pitch axis control reference line Lp and the third lane marking LF3 in the input image as a left positive area value Sppl, and calculates the area value of the region Rpnl between the pitch axis control reference line Lp and the fourth lane marking LF4 in the input image as a left negative area value Spnl. The pitch axis target control amount calculation unit 32 calculates these area values ​​Sppr, Spnr, Sppl, and Spnl based on, for example, the quadrature by section method described in steps ST12 to ST15. In the following, the sum of the right positive area value Sppr and the left positive area value Sppl will be referred to as the positive area value Spp (=Sppr+Sppl), and the sum of the right negative area value Spnr and the left negative area value Spnl will be referred to as the negative area value Spn (=Spnr+Spnl).

[0204] Next, in step ST76, the pitch axis target control amount calculation unit 32 calculates the pitch axis target control amount up based on the positive side area value Spp and the negative side area value Spn calculated by the above-described procedure, and then proceeds to step ST77. More specifically, the pitch axis target control amount calculation unit 32 calculates the pitch axis target control amount up so that the larger the positive side area value Spp, the larger the pitch axis target control amount up, and the larger the negative side area value Spn, the larger the pitch axis target control amount up. As a result, the larger the positive side area value Spp, the more the attitude of the aircraft around the pitch axis during flight can be changed to the positive side (i.e., clockwise when viewed from the right). Also, the larger the negative side area value Spn, the more the attitude of the aircraft around the pitch axis during flight can be changed to the negative side (i.e., counterclockwise when viewed from the right).

[0205] More specifically, the pitch axis target control amount calculation unit 32 calculates the pitch axis target control amount up by dividing the difference (Spp-Spn) between the positive side area value Spp and the negative side area value Spn by the sum (Spp+Spn) of the positive side area value Spp and the negative side area value Spn, as shown in the following equation (13). The following description will be given assuming that the gain Gain_p in the following equation (13) is a positive constant, but the present invention is not limited to this. The gain Gain_p may also be a negative value. The value of this gain Gain_p may also be changed depending on some parameter such as the speed of the aircraft.

number

[0206] In step ST77, the pitch axis target control amount calculation unit 32 transmits the pitch axis target control amount up calculated by the above procedure to the pitch axis attitude control unit 42, and ends the pitch axis target control amount calculation process in FIG.

[0207] 25 is a flowchart showing a specific procedure for the roll axis target control amount calculation process for calculating the roll axis target control amount based on an input image. This roll axis target control amount calculation process is executed by the roll axis target control amount calculation unit 33 every time a new input image is acquired by the input image acquisition unit 2A.

[0208] Fig. 26 is a diagram showing an example of an input image. In the following, a specific procedure of the roll axis target control amount calculation process of Fig. 25 will be described using the input image shown in Fig. 26 as an example.

[0209] First, in step ST81, the roll axis target control amount calculation unit 33 recognizes the target T to be followed by the flying object F based on the input image, and then proceeds to step ST82.

[0210] Next, in step ST82, the roll axis target control amount calculation unit 33 determines whether or not the target T to be tracked was recognized from the input image in step ST81. If the determination result in step ST82 is YES, the roll axis target control amount calculation unit 33 proceeds to step ST83, and if the determination result in step ST82 is NO, the roll axis target control amount calculation unit 33 proceeds to step ST89. Also, in step ST89, the roll axis target control amount calculation unit 33 cancels the automatic attitude control by the roll axis attitude control unit 43, and ends the roll axis target control amount calculation process of FIG. 25.

[0211] Next, in step ST83, the roll axis target control amount calculation unit 33 virtually sets, for the input image, a target attitude line Lt that passes through the center position Ot of the target T to be followed and a reference circle Ct that has the center at the center Ot, based on the position and attitude of the target T to be followed in the input image, and then proceeds to step ST84. As shown in Fig. 26, the roll axis target control amount calculation unit 33 sets a line that is parallel to the horizontal direction of the airframe of the target T to be followed and that passes through the center position Ot, as the target attitude line Lt.

[0212] Next, in step ST84, the roll axis target control amount calculation unit 33 sets two virtual roll axis control reference lines Lr1, Lr2 in the input image, and then proceeds to step ST85. Here, the roll axis target control amount calculation unit 33 sets a line that passes through the center position Ot of the target T and is parallel to the X-axis as the first roll axis control reference line Lr1, and sets a line that passes through the center position Ot of the target T and is perpendicular to the first roll axis control reference line Lr1 as the second roll axis control reference line Lr2. In the following, the input image is divided into a first quadrant, a second quadrant, a third quadrant, and a fourth quadrant by these two roll axis control reference lines Lr1, Lr2. In the following, the area above the first roll axis control reference line Lr1 and to the right of the second roll axis control reference line Lr2 will be referred to as the first quadrant, the area below the first roll axis control reference line Lr1 and to the right of the second roll axis control reference line Lr2 will be referred to as the second quadrant, the area below the first roll axis control reference line Lr1 and to the left of the second roll axis control reference line Lr2 will be referred to as the third quadrant, and the area above the first roll axis control reference line Lr1 and to the left of the second roll axis control reference line Lr2 will be referred to as the fourth quadrant.

[0213] Next, in step ST85, the roll axis target control amount calculation unit 33 calculates a first area value Sr1, a second area value Sr2, a third area value Sr3, and a fourth area value Sr4 as the area values ​​of multiple regions partitioned between the first roll axis control reference line Lr1 and the target posture line Lt in each quadrant of the input image, and then proceeds to step ST86.

[0214] More specifically, the roll axis target control amount calculation unit 33 calculates the area value of the region between the first roll axis control reference line Lr1 and the target posture line Lt in the first quadrant of the input image as a first area value Sr1, calculates the area value of the region between the first roll axis control reference line Lr1 and the target posture line Lt in the second quadrant of the input image as a second area value Sr2, calculates the area value of the region between the first roll axis control reference line Lr1 and the target posture line Lt in the third quadrant of the input image as a third area value Sr3, and calculates the area value of the region between the first roll axis control reference line Lr1 and the target posture line Lt in the fourth quadrant of the input image as a fourth area value Sr4.

[0215] Here, the region between the first roll axis control reference line Lr1 and the target posture line Lt refers to the triangular portion (the hatched portion in FIG. 26 ) defined by the first roll axis control reference line Lr1, the target posture line Lt, and the auxiliary line when auxiliary lines parallel to the second roll axis control reference line Lr2 are set from two intersections of the target posture line Lt and the reference circle Ct, as shown in FIG. 26 . The area value in the quadrant where the target posture line Lt does not exist is set to 0. That is, in the example shown in FIG. 26 , the target posture line Lt exists only in the second and fourth quadrants, but not in the first and third quadrants. Therefore, in the example shown in FIG. 26 , the first area value Sr1 and the third area value Sr3 are both 0, and the second area value Sr2 and the fourth area value Sr4 are the area values ​​of the hatched portions in FIG. 26 . In the following, the sum of the second area value Sr2 and the fourth area value Sr4 is referred to as the positive side area value Srp (=Sr2+Sr4), and the sum of the first area value Sr1 and the third area value Sr3 is referred to as the negative side area value Srn (=Sr1+Sr3).

[0216] Next, in step ST86, the roll axis target control amount calculation unit 33 calculates the roll axis target control amount ur based on the positive side area value Srp and the negative side area value Srn calculated by the above procedure, and then proceeds to step ST87. More specifically, the roll axis target control amount calculation unit 33 calculates the roll axis target control amount ur so that the larger the positive side area value Srp, the larger the positive side area value, and so that the larger the negative side area value Srn, the larger the negative side. As a result, the larger the positive side area value Srp, the more the attitude of the aircraft around the roll axis during flight can be changed to the positive side (i.e., clockwise when viewed from the rear). Also, the larger the negative side area value Srn, the more the attitude of the aircraft around the roll axis during flight can be changed to the negative side (i.e., counterclockwise when viewed from the rear).

[0217] More specifically, the roll axis target control amount calculation unit 33 calculates the roll axis target control amount ur by dividing the difference (Srp-Srn) between the positive side area value Srp and the negative side area value Srn by the sum (Srp+Srn) of the positive side area value Srp and the negative side area value Srn, as shown in the following equation (14), and further multiplying the result by a predetermined gain Gain_r. Note that, although the following description will be given of a case where the gain Gain_r in the following equation (14) is a positive constant, the present invention is not limited to this. The gain Gain_r may also be a negative value. Furthermore, the value of this gain Gain_r may be changed depending on some parameter such as the speed of the aircraft.

number

[0218] In step ST87, the roll axis target control amount calculation unit 33 transmits the roll axis target control amount ur calculated by the above procedure to the roll axis attitude control unit 43, and transmits it to the roll axis attitude control unit 43 in FIG. 25, and then ends the roll axis target control amount calculation process in FIG. 25.

[0219] The aircraft control device 1A according to this embodiment has the following advantages. (19) The input image acquisition unit 2A acquires as an input image an image captured by a camera CA directed forward as viewed from the aircraft F. The yaw-axis target control amount calculation unit 31 calculates a yaw-axis target control amount for the yaw-axis attitude control mechanism 9Y of the aircraft F based on the input image. The yaw-axis attitude control unit 41 operates the yaw-axis attitude control mechanism 9Y to the positive or negative side based on the yaw-axis target control amount. The yaw-axis target control amount calculation unit 31 also recognizes the target T to be tracked of the aircraft F based on the input image, sets a virtual flight path for the input image based on the position of the target T to be tracked in the input image, and calculates a yaw-axis target control amount uy based on a positive-side area value Syp between the yaw-axis control reference line Ly and flight lane lines LF1, LF2, which are the positive-side boundaries of the flight path, and a negative-side area value Syn between the yaw-axis control reference line Ly and flight lane lines LF3, LF4, which are the negative-side boundaries of the flight path. Therefore, according to the aircraft control device 1A, the yaw axis target control amount uy for the yaw axis attitude control mechanism 9Y of the aircraft F can be calculated simply by calculating the area value from the input image, thereby reducing the burden on the computer responsible for such calculations.

[0220] (20) The pitch axis target control amount calculation unit 32 calculates a pitch axis target control amount for the pitch axis attitude control mechanism 9P of the aircraft F based on the input image, and the pitch axis attitude control unit 42 operates the pitch axis attitude control mechanism 9P to the positive or negative side based on the pitch axis target control amount. The pitch axis target control amount calculation unit 32 also calculates the pitch axis target control amount up based on the positive-side area value Spp between the pitch axis control reference line Lp and flight lane lines LF2, LF3, which are the positive-side boundaries of the flight path, and the negative-side area value Spn between the pitch axis control reference line Lp and flight lane lines LF1, LF4 on the negative side of the flight path. Therefore, according to the aircraft control device 1A, the pitch axis target control amount up for the pitch axis attitude control mechanism 9P of the aircraft F can be calculated simply by calculating area values ​​from the input image, thereby reducing the burden on the computer that performs such calculations.

[0221] (21) The roll axis target control amount calculation unit 33 calculates the roll axis target control amount for the roll axis attitude control mechanism 9R of the aircraft F based on the input image, and the roll axis attitude control unit 43 operates the roll axis attitude control mechanism 9R to the positive or negative side based on the roll axis target control amount. The roll axis target control amount calculation unit 33 also sets a virtual target attitude line Lt for the input image based on the attitude of the target T to be tracked in the input image, calculates a positive-side area value Srp and a negative-side area value Srn between the virtual first roll axis control reference line Lr1 and the target attitude line Lt in the input image, and calculates the roll axis target control amount ur based on these area values ​​Srp and Srn. Therefore, according to the aircraft control device 1A, the roll axis target control amount ur for the roll axis attitude control mechanism 9R of the aircraft F can be calculated simply by calculating the area values ​​from the input image, thereby reducing the burden on the computer that performs such calculations.

[0222] Although the first to third embodiments of the present invention have been described above, the present invention is not limited to these and the detailed configuration may be changed as appropriate within the scope of the spirit of the present invention. [Explanation of symbols]

[0223] V...Vehicle C...In-vehicle camera 1...Vehicle control device 2... Input image acquisition unit (input image acquisition means) 3...Target steering amount calculation unit (target steering amount calculation means) 4...Steering control unit (steering control means) 5…Target vehicle speed calculation unit (target vehicle speed calculation means) 6...Acceleration / deceleration control section (acceleration / deceleration control means) 7...Braking device (acceleration / deceleration device) 8...Power plant (acceleration / deceleration device) 9...Electric power steering device (steering mechanism) F...Flying object T...Target to follow CA...camera 1A...Flight control device 2A... Input image acquisition unit (input image acquisition means) 31... Yaw axis target control amount calculation unit (first target control amount calculation means) 32... Pitch axis target control amount calculation unit (second target control amount calculation means) 33... Roll axis target control amount calculation unit (third target control amount calculation means) 41... Yaw axis attitude control unit (first axis attitude control means) 42... Pitch axis attitude control unit (second axis attitude control means) 43...Roll axis attitude control unit (third axis attitude control means) 9Y...Yaw axis attitude control mechanism (first axis attitude control mechanism) 9P...Pitch axis attitude control mechanism (second axis attitude control mechanism) 9R...Roll axis attitude control mechanism (third axis attitude control mechanism)

Claims

1. an input image acquisition means for acquiring, as an input image, an image captured by a camera facing forward as viewed from the vehicle; a target steering amount calculation means for calculating a target steering amount for a steering mechanism of the vehicle based on the input image; a steering control means for operating the steering mechanism to a positive side or a negative side based on the target steering amount, The target steering amount calculation means A process of recognizing a traveling path of the vehicle based on the input image; a process of calculating at least one of a positive side area value between a virtual neutral reference line defined in the input image and a positive side lane marking that is a boundary on the positive side of the roadway, and a negative side area value between the neutral reference line and a negative side lane marking that is a boundary on the negative side of the roadway; and calculating the target steering amount based on at least one of the positive side area value and the negative side area value.

2. 2. The vehicle control device according to claim 1, wherein the target steering amount calculation means calculates the target steering amount so that it increases toward a positive side as the positive side area value increases, and calculates the target steering amount so that it increases toward a negative side as the negative side area value increases.

3. 3. The vehicle control device according to claim 2, wherein the target steering amount calculation means virtually sets the neutral reference line extending in the vertical direction in the input image and a plurality of lateral reference lines intersecting the neutral reference line in the input image, calculates the area value of the area surrounded by the positive side demarcation line, the neutral reference line, and the lateral reference line as the positive side area value, and calculates the area value of the area surrounded by the negative side demarcation line, the neutral reference line, and the lateral reference line as the negative side area value.

4. The target steering amount calculation means subtracting a predetermined negative reference value from the positive area value; subtracting the negative area value from a predetermined positive reference value, or 2. The vehicle control device according to claim 1, wherein the target steering amount is calculated by subtracting the negative area value from the positive area value.

5. The vehicle control device described in claim 4, characterized in that when the positive side dividing line and the neutral reference line intersect in the input image, the target steering amount calculation means calculates the positive side area value by subtracting the area value between the portion of the neutral reference line above the intersection point with the positive side dividing line and the positive side dividing line from the area value between the portion of the neutral reference line below the intersection point with the positive side dividing line and the positive side dividing line.

6. The vehicle control device described in claim 4, characterized in that when the negative side dividing line and the neutral reference line intersect in the input image, the target steering amount calculation means calculates the negative side area value by subtracting the area value between the portion of the neutral reference line above the intersection point with the negative side dividing line and the negative side dividing line from the area value between the portion of the neutral reference line below the intersection point with the negative side dividing line and the negative side dividing line.

7. The target steering amount calculation means Dividing the difference between the positive area value and the negative reference value by the sum of the positive area value and the negative reference value, or Dividing the difference between the positive reference value and the negative area value by the sum of the positive reference value and the negative area value, or 7. The vehicle control device according to claim 5, wherein the target steering amount is calculated by dividing a difference between the positive area value and the negative area value by a sum of the positive area value and the negative area value.

8. the camera is provided at the center of the vehicle body in the width direction, The vehicle control device according to claim 1, characterized in that the target steering amount calculation means is configured to execute a process of setting the neutral reference line at the center of the width direction of the input image when the vehicle is driven along the vehicle driving lane between the positive side lane marking and the negative side lane marking.

9. The target steering amount calculation means a process of storing, as a center position, the position of the neutral reference line that makes the positive side area value and the negative side area value equal when the vehicle is traveling in the center in the width direction of a straight road; 2. The vehicle control device according to claim 1, further comprising: a process for setting the neutral reference line at the center position when the vehicle is traveling along the vehicle driving lane between the positive side lane marking and the negative side lane marking.

10. The target steering amount calculation means When the vehicle is driven from the own vehicle driving lane toward the forward adjacent driving lane adjacent to the forward lane marking, a process of moving the neutral reference line in the input image toward the forward side at a predetermined speed; 10. The vehicle control device according to claim 8, wherein when the vehicle is driven from the own vehicle driving lane toward the negative side adjacent driving lane adjacent to the negative side dividing line, the vehicle control device is configured to execute a process of moving the neutral reference line to the negative side in the input image at a predetermined speed.

11. The target steering amount calculation means a process of moving the neutral reference line to the negative side in the input image when the positive side lane marking and a positive side obstacle that conceals a part of the positive side lane marking are recognized based on the input image; 10. The vehicle control device according to claim 8 or 9, characterized in that, when the negative side lane marking and a negative side obstacle obscuring part of the negative side lane marking are recognized based on the input image, the vehicle control device is configured to execute a process of moving the neutral reference line to the positive side in the input image.

12. The target steering amount calculation means a process of estimating the negative-side lane marking based on the positive-side lane marking when the positive-side lane marking can be recognized based on the input image but the negative-side lane marking cannot be recognized; 2. The vehicle control device according to claim 1, further comprising: a process for estimating the positive side lane marking based on the negative side lane marking when the negative side lane marking can be recognized based on the input image and the positive side lane marking cannot be recognized.

13. The target steering amount calculation means When the front-side lane marking line and a front-side obstacle that conceals a part of the front-side lane marking line are recognized based on the input image, a process of interpolating the part of the front-side lane marking line that is concealed by the front-side obstacle with a line along the negative side of the front-side obstacle; The vehicle control device described in claim 1 is configured to execute a process in which, when the negative side lane marking line and a negative side obstacle that hides part of it are recognized based on the input image, the part of the negative side lane marking line that is hidden by the negative side obstacle is interpolated with a line along the positive side of the negative side obstacle.

14. a target vehicle speed calculation means for calculating a target vehicle speed of the vehicle based on the input image; an acceleration / deceleration control means for operating an acceleration / deceleration device of the vehicle based on the target vehicle speed, The target vehicle speed calculation means A process of recognizing the road based on the input image; a process of setting a speed control reference line extending in a width direction of the input image; a process of calculating an area value of the travel path above the speed control reference line as an upper area value; a process of calculating an area value of the roadway below the speed control reference line as a lower area value; A process of calculating a sum of the upper area value and the lower area value as a total area value; a process of calculating a ratio of the upper area value to the total area value as an upper / lower ratio value; 2. The vehicle control device according to claim 1, further comprising: a process for calculating the target vehicle speed based on the upper / lower ratio value.

15. 15. The vehicle control device according to claim 14, wherein the target vehicle speed calculation means calculates the target vehicle speed so that the target vehicle speed decreases as the upper / lower ratio value increases.

16. a storage medium for storing a plurality of speed tables that associate the upper / lower ratio value with the target vehicle speed; The target vehicle speed calculation means a process of estimating a curvature parameter of the road based on the input image; 15. The vehicle control device according to claim 14, further comprising: a process of selecting one of the plurality of speed tables based on the curvature parameter; and a process of calculating the target vehicle speed based on the selected speed table and the up / down ratio value.

17. a storage medium for storing a plurality of speed tables that associate the upper / lower ratio value with the target vehicle speed; The target vehicle speed calculation means A process of obtaining a driving mode of the vehicle; 15. The vehicle control device according to claim 14, further comprising: a process of selecting one of the plurality of speed tables based on the driving mode; and a process of calculating the target vehicle speed based on the selected speed table and the up / down ratio value.

18. The target vehicle speed calculation means a process of estimating the negative-side lane marking based on the positive-side lane marking when the positive-side lane marking can be recognized based on the input image but the negative-side lane marking cannot be recognized; 15. The vehicle control device according to claim 14, further comprising: a process for estimating the positive side lane marking based on the negative side lane marking if the negative side lane marking can be recognized based on the input image but the positive side lane marking cannot be recognized.

19. A vehicle control method for controlling a vehicle by a computer based on an image captured by a camera facing forward as viewed from the vehicle, comprising: acquiring an image captured by the camera as an input image; recognizing a travel path of the vehicle based on the input image; Calculating at least one of a positive side area value between a virtual neutral reference line defined in the input image and a positive side lane marking that is a boundary on the positive side of the roadway, and a negative side area value between the neutral reference line and a negative side lane marking that is a boundary on the negative side of the roadway; calculating a target steering amount for a steering mechanism of the vehicle based on at least one of the positive side area value and the negative side area value; and operating the steering mechanism to a positive or negative direction based on the target steering amount.

20. A program for controlling a vehicle by a computer based on an image captured by a camera facing forward as viewed from the vehicle, The computer, acquiring an image captured by the camera as an input image; recognizing a travel path of the vehicle based on the input image; Calculating at least one of a positive side area value between a virtual neutral reference line defined in the input image and a positive side lane marking that is a boundary on the positive side of the roadway, and a negative side area value between the neutral reference line and a negative side lane marking that is a boundary on the negative side of the roadway; calculating a target steering amount for a steering mechanism of the vehicle based on at least one of the positive side area value and the negative side area value; and operating the steering mechanism to the positive or negative side based on the target steering amount.

21. an input image acquisition means for acquiring an image captured by a camera facing forward as viewed from the aircraft as an input image; a first target control amount calculation means for calculating a first target control amount for a first axis attitude control mechanism that changes the attitude of the aircraft about a first axis based on the input image; a first axis attitude control means for operating the first axis attitude control mechanism to a positive side or a negative side based on the first target control amount, The first target control amount calculation means A process of recognizing a target to be followed by the aircraft based on the input image; A process of virtually setting a flight path for the input image based on the position of the target in the input image; A process of calculating a first positive area value between a first control reference line virtually defined in the input image and a first positive division line that is a boundary on the positive side of the flight path, and a first negative area value between the first control reference line and a first negative division line that is a boundary on the negative side of the flight path; and calculating the first target control amount based on the first positive area value and the first negative area value.

22. a second target control amount calculation means for calculating a second target control amount for a second axis attitude control mechanism that changes the attitude of the aircraft about the second axis based on the input image; a second axis attitude control means for operating the second axis attitude control mechanism to a positive side or a negative side based on the second target control amount, The second target control amount calculation means A process of calculating a second positive area value between a second control reference line virtually defined so as to be perpendicular to the first control reference line in the input image and a second positive demarcation line that is a boundary on the positive side of the flight path, and a second negative area value between the second control reference line and a second negative demarcation line that is a boundary on the negative side of the flight path; and calculating the second target control amount based on the second positive area value and the second negative area value.

23. a third target control amount calculation means for calculating a third target control amount for a third axis attitude control mechanism that changes the attitude of the aircraft about the third axis based on the input image; a third axis attitude control means for operating the third axis attitude control mechanism to a positive side or a negative side based on the third target control amount, The third target control amount calculation means a process of virtually setting a target posture line for the input image based on the posture of the target to be tracked in the input image; a process of calculating a third area value between a third control reference line virtually defined in the input image and the target attitude line; 23. The aircraft control device according to claim 22, further comprising: a process for calculating the third target control amount based on the third area value.

Images