Route recognition system for autonomous vehicles

Abstract

To provide a route recognition system which performs automatic operation following a stable travel route by grasping route lines provided on a road surface with a camera provided on a vehicle body.SOLUTION: In a route recognition system of automatic operation traveling on route lines provided on a road surface, a camera is mounted at such a position that the route lines are visible without exposed to disturbance of natural environment such as influences of sunlight in the vehicle body and shades and a relative position with the center of gravity of a vehicle body can be recognized, a lateral deviation (e2) between the route lines and the center of gravity of the vehicle body, and angle deviation (e3) of a body shaft to the route lines are detected while referring to the position of the camera, and steering is controlled so that the detected lateral deviation (e2) and the angle deviation (e3) are eliminated, and the center of gravity of the vehicle body coincides with the route lines.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a route recognition system for an autonomous vehicle that follows a driving route using a camera installed in the vehicle. [Background technology]

[0002] A driver's driving actions are a repeated sequence of cognition, judgment, and operation. Research has been conducted on improving automobile safety by elucidating the driver's driving behavior, on automobiles and control systems that are suited to human characteristics, and on driver steering, such as where the driver looks on the route and how he or she steers.

[0003] Non-Patent Document 1 proposes a forward-gaze primary prediction model in which "the steering wheel is operated in proportion to the deviation between the target course at a gaze point a certain distance L ahead in the vehicle's traveling direction and the predicted future position of the vehicle."

[0004] In chapter 11 of Non-Patent Document 2, it is stated that the key to automatic steering is to detect the deviation of the front and rear ends of the vehicle from the desired course using an appropriate method, and to perform steering that is sensitive to the sum and difference of the two.

[0005] In addition, in Non-Patent Document 3, when driving around sharp curves, the primary prediction model is unable to follow the course well. Therefore, as a more realistic model, a secondary prediction model is proposed, which considers that "the future vehicle position is regarded as a position predicted based on the current motion state, and steering is performed according to the deviation from the target course." The forward gaze models in Non-Patent Documents 1 and 3 use a camera mounted on a vehicle to recognize the forward direction of travel, leading to technologies for driving assistance and self-driving.

[0006] Patent Document 1 describes that the camera is equipped with a means for notifying the user of conditions under which recognition processing is impossible, such as when the image is too bright due to backlighting or after rain, when the image is too dark at night, or when the image has low contrast due to twilight or snow.

[0007] The key points of the automatic steering described in Non-Patent Document 2 are connected to path following control that recognizes lanes using a side camera, and further to a magnetic sensor method that follows a route using magnets buried in the road surface. Patent document 2 states that a magnetic sensor installed under the vehicle floor detects the position of magnets buried in the road, making it possible to recognize the position without using GPS or IMU, and to travel without derailing from the magnetic track, from straight ahead to all curvature radii including the minimum turning radius.

[0008] Patent Document 3 explains a method for detecting changes in the center of gravity position due to changes in the load, and shows a redundant system that obtains the sideslip angle and yaw rate of the center of gravity of the vehicle from a vehicle model, calculates its coordinates and direction, and coordinates them with the detection values ​​from an IMU (inertial navigation unit), GPS, and magnetic markers.

[0009] Patent Document 4 describes an autonomous vehicle equipped with multiple sensors to recognize the surroundings of the vehicle, and describes a method of external recognition using each sensor, in which the surrounding recognition coordinates recognized by the multiple sensors attached at different positions are always replaced with the vehicle center of gravity position. In other words, the coordinates detected by each sensor are converted and captured on the vehicle coordinate system with the vehicle center of gravity, which is the origin of vehicle motion, as the origin. [Prior art documents] [Patent documents]

[0010] [Patent Document 1] JP 2000-207563 A [Patent Document 2] JP 2020-098566 A [Patent Document 3] JP 2021-25925 A [Patent Document 4] JP 2022-0343345 A [Non-patent literature]

[0011] [Non-Patent Document 1] Masaichi Kondo: On the fundamental relationship between steering and motion of an automobile, Transactions of the Society of Automotive Engineers of Japan, No. 5 (1958), p. 40-43. [Non-Patent Document 2] Masaichi Kondo: Basic Automotive Engineering, Late Edition, Yokendo Co., Ltd., 1st edition published on April 20, 1962, p.21. [Non-Patent Document 3] Kenichi Yoshimoto: Simulation of the human-vehicle system using a steering model including prediction, Journal of the Japan Society of Mechanical Engineers, Vol. 71, No. 596 (September 1968), pp. 13-18. DISCLOSURE OF THEINVENTION [Problem to be solved by the invention]

[0012] For self-driving vehicles, white line recognition devices have been developed that photograph the left and right white lines and recognize them. The white line recognition system photographs the shapes of the left and right white lines using cameras inside the windshield and on top of the vehicle while driving, but there are cases where the white lines cannot be recognized due to exposure to disturbances from the natural environment such as sunlight and shadows.

[0013] Non-Patent Documents 1 to 3 relate to a forward-focused model that recognizes the driving direction ahead using a camera mounted on a vehicle, but make no proposal for resolving the difficulty of recognizing markers on the road, such as white lines.

[0014] Patent Document 1 also relates to a forward gaze model, and this prior document only notifies the driver when it becomes difficult to recognize an index due to twilight or the like, but does not propose any means for resolving the situation.

[0015] In Patent Document 2, magnets buried in the road are provided with information such as route coordinates, curvature, and azimuth, resulting in excellent route tracking performance. However, while the magnetic sensor method solves the problem of being unable to recognize white lines, it has concerns about its resistance to lightning strikes and the cost of burying magnetic markers. [Means for solving the problem]

[0016] In order to solve the above problems, the route recognition system for automated driving according to the present invention sets a route line on the road surface, and attaches a camera to the vehicle body in a position where the route line can be viewed without being exposed to disturbances from the natural environment such as the effects of sunlight or shadows, and where its relative position with respect to the center of gravity of the vehicle body can be determined. By referring to the position of this camera, the lateral deviation (e2) between the route line and the center of gravity of the vehicle and the angular deviation (e3) of the vehicle axle relative to the route line are detected, and steering control is performed to eliminate the detected lateral deviation (e2) and angular deviation (e3) and so as to align the center of gravity of the vehicle with the route line.

[0017] The route lines may be dotted or colored lines, in addition to existing white lines, and the route lines themselves may have information such as curvature change information, road gradient change information, planned vehicle speed information, route branch information, route junction information, and longitude, latitude, and orientation. In this way, look-ahead control becomes possible.

[0018] Regarding the position of the center of gravity of the vehicle body, the position of the center of gravity when the vehicle is unladen can be calculated by multiplying the wheelbase by the ratio of the front axle load to the rear axle load written on the vehicle inspection certificate, and the position of the center of gravity when the vehicle is loaded or has passengers on board can be calculated by detecting the front axle load to the rear axle load from the air pressure of the air suspension of the front and rear axles and multiplying the ratio by the wheelbase, so the positional relationship between the installation position of the camera itself and the position of the center of gravity can be calculated. It is preferable that the changing position of the center of gravity falls within the field of view of the camera.

[0019] Also, as an example of a mounting position where the route line can be visually confirmed and the relative position with respect to the center of gravity of the vehicle can be known, it is possible to mount the sensor under the floor of the vehicle body. In this position, the route line can be recognized and the lateral deviation (e2) between the route line and the center of gravity of the vehicle can be detected accurately. However, measures against stone chipping, water intrusion, etc. are required.

[0020] Also, the cameras may be installed in two locations, one where the route line can be seen and the other where the relative position to the center of gravity can be determined. In this case, the positional relationship between the two cameras must be corrected when calculating the lateral deviation (e2) and angular deviation (e3). Effect of the Invention

[0021] According to the present invention, by mounting a camera on the front of the vehicle, on the underfloor surface and within the range of change of the center of gravity of the vehicle, and illuminating this range with a lighting device provided on the vehicle body, the route line can be recognized even in twilight or bad weather. Then, the lateral and angular displacements of the vehicle body at the center of gravity of the vehicle relative to the route line can be captured to determine the steering angle and control the following of the route. Furthermore, by associating the type, shape and color of the route line with the route information, the steering angle can be determined, and the route following can be controlled in response to operation management such as acceleration / deceleration, stopping, waiting and U-turns. [Brief description of the drawings]

[0022] [Figure 1] Diagram of underfloor camera field of view [Diagram 2] A diagram illustrating how the robot follows a route by detecting the deviation of its own position from the route line and the curvature of the route. [Diagram 3] Diagram of how the vehicle recognizes its own location from route information and follows the route [Figure 4] Diagram of detecting junctions and other intersections from route information DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] Hereinafter, an embodiment of the present invention will be described with reference to FIGS. Figure 1 is an explanatory diagram of the field of view arrangement of an underfloor camera, showing a vehicle turning with a minimum turning radius R0. The position of the center of gravity of the vehicle changes depending on the load or number of passengers. The camera is placed at a position (camera field of view (W) in the figure) where the locus of the center of gravity is within the camera's field of view. The axis of the camera passes through the center of the camera's field of view (W).

[0024] Incidentally, the center of gravity trajectory is not visible to the camera, but rather it is the route line that is projected in the camera's field of view (W). In this embodiment, the center of gravity trajectory drawn by the center of gravity of the vehicle in a planar view is included in the camera's field of view (W) which projects the route line. In other words, by aligning the x-axis (vehicle body front-rear central axis) of the vehicle's xy coordinate system with the camera axis and determining and mounting the camera's front-rear position (y value of the xy coordinate system), the center of gravity trajectory drawn by the center of gravity of the vehicle is included in the camera's field of view (W).

[0025] The path traced by the center of gravity is an arc with its center at "Point O" on the extension of the rear axle when turning with the minimum turning radius. The field of view must be long and wide enough to include this arc, and the length (lsf-lsr) and width (B) of the field of view must be secured to capture the path traced by the center of gravity over the entire range from straight driving to the minimum turning radius, and the field of view must be set to capture the change in curvature of the path line on the road surface.

[0026] Although the underfloor camera can "capture the trajectory of the center of gravity over the entire range from straight driving to the minimum radius," it is better that the front-facing camera is limited to the trajectory of the center of gravity that falls within the field of view captured by the front-facing camera. Therefore, automatic driving using only the front-facing camera is limited to the turning range within which the trajectory of the center of gravity of the vehicle can be captured in the field of view. The fields of view of the front-facing camera and the underfloor camera are matched by recognizing them on the xy plane coordinate system with the center of gravity of the vehicle as the origin. The origin of the field of view image may be the intersection of the vehicle's front-to-rear center line and the rear axle center line instead of the vehicle's center of gravity position.

[0027] FIG. 2 is an explanatory diagram of detecting the self-position deviation from the route line and the route curvature, and tracing the route by detecting the route curvature. The route line is shown by a dotted line. The route line is captured at points P1, Pc, and P2. Capturing point Pc requires that the center of gravity position Gc is known. The distance between the center of gravity position Gc and point Pc is taken as the lateral deviation e2, and the angle between the route tangent at point Pc (a line perpendicular to the normal line) and the vehicle center line (x-axis) is taken as the angular deviation e3. The actual front wheel steering angle for tracing the route is calculated from the captured e2 and e3 and the vehicle's longitudinal speed vx.

[0028] In other words, the differential value e'2 of the lateral deviation e2 is divided by the vehicle longitudinal speed vx to obtain the yaw rate r e2 , that is, equation (1). The differential value of e3 is the yaw rate r e3 , that is, equation (2). The intersection point 0 of the normal of point p1 and the normal of point p2 12 Find the radius of curvature (R 1-= R2) and take the inverse of it to determine the path curvature ρroute Then, multiplying the path curvature by the path tangential speed (i.e., the vehicle longitudinal speed divided by the cosine of the angle deviation e3) gives the path yaw rate r route , i.e., equation (3). Thus, the sum of equations (1), (2), and (3) is the yaw rate r of the vehicle, i.e., equation (4).

[0029]

number

[0030] The yaw rate of the vehicle is expressed as the vehicle speed v (i.e., v x Dividing this by ρ / cos e2) gives the vehicle turning curvature ρ, i.e., equation (5). The actual front wheel steering angle δ to align the vehicle's center of gravity with this path is given by the equation The result is (6).

[0031]

number

[0032] FIG. 3 is an explanatory diagram of how the vehicle recognizes its own position from route line information and follows a route. The origin of the coordinate system (earth coordinates) is placed at the starting point (starting point), and the coordinates x and y of the length of the dashed line and the interval between the dashed lines are accumulated (integrated) from there to obtain the coordinates of the current location (x n ,y n ) and the azimuth angle (Φ n ) can be calculated. Each broken line is associated with an ID number, and map data such as latitude, longitude, and azimuth angle and other information are associated with it, and table lookup control is performed.

[0033]

number

[0034] Now, the lateral deviation of the vehicle with respect to the point pn(xn,yn) on the route and its tangent is e2, and the angular deviation is e3. The x-coordinate of the vehicle's center of gravity is given by equation (10), and the y-coordinate is given by equation (11). The straight-line distance l from the vehicle's center of gravity coordinate (xGC,yGC) to the junction pm on the route isGCm and the path arc angle φ GCm The angle of side slip β included in equation (13) is expressed as equation (14). Thus, the path curvature ρ GCm is expressed as equation (15), and the actual front wheel steering angle δ to follow the path curvature from the center of gravity to the junction is expressed as equation (16). In equation (16), l is the wheelbase of the vehicle and the path curvature ρ GCm The product of this and corresponds to the actual steering angle at low speed, and the product of this and (1+K SF v^2) indicates the speed dependency of the actual steering angle. The position of the merging point is located farther away at higher speeds and smaller curvatures, and closer at lower speeds and larger curvatures. Here, the side slip coefficient β included in equation (14) and the stability factor K included in equation (16) SF are given by equation (17) and equation (18). In equation (17), lf is the distance from the center of gravity of the vehicle to the front axle, Ccr is the rear tire cornering coefficient, Nr is the rear axle load, lr is the distance from the center of gravity of the vehicle to the rear axle, and m is the vehicle mass. In equation (18), Ccf is the front tire cornering coefficient, and Nf is the front axle load.

[0035]

number

[0036] 4 is an explanatory diagram for detecting branching and merging from route information. In the present invention, route lines are solid or dotted, white or colored, and laid in the center of the route, making it possible to preemptively transmit curvature change information, road gradient change information, planned vehicle speed information, route branching information, and route merging information.

[0037] Figure 4(A) is an example of distinguishing between cruising on a single track section and branching and merging in a waiting area section by the type and color of the line. The single track section is drawn with a dashed white line, and waiting areas are set up for up and down trains to pass each other, with the up and down passing lines being dashed white lines, the up waiting lines being solid white lines, and the down waiting lines being solid colored (e.g. blue) lines, ensuring control over determining branching and merging.

[0038] Figure 4(B) is an example of distinguishing between entering and leaving a large curvature curved road from a straight line or a small curvature curved road by the type of line. Straight lines and small curvature curved roads are shown as white dotted lines, which meet the requirements for branching and merging as described above, while large curvature curved roads, which require relatively strict route control, are shown as solid lines in a color (e.g., blue).

[0039] As mentioned above, the dashed route drawn on the road surface is captured by cameras installed under the floor and in front of the vehicle along with lighting, and the route is followed. In addition to the current location and current lateral deviation, angular deviation, and route curvature of the vehicle relative to the route line captured by the camera, route information is expressed by the type of route line (solid line, dashed line) and color of the route line (white, blue), making it possible to control the vehicle to predict and prepare route information ahead of time.

[0040] In order to travel along the route at the planned vehicle speed and with appropriate acceleration / deceleration and lateral acceleration, the system controls the drive and braking systems (controls drive gear selection, accelerator opening, engine brake, auxiliary brake, and wheel brake) to accommodate changes in the gradient of the route, and controls the drive, braking, and steering systems to accommodate changes in the curvature of the route.

[0041] The route includes branching and stopping for entering, stopping, and starting at bus stops, as well as deceleration, steering, stopping, starting, steering, and accelerating, boarding and alighting, merging for starting, cruising on single-track sections, and diverging, waiting, and merging at single-track waiting areas.

[0042] In both Figures 4(A) and (B), the main route is represented by a dotted line, and the dotted lines are counted and associated with route information such as distance from the starting point, gradient, curvature, and command speed according to their rank. Branches are represented by white solid lines and colored solid lines. In addition, solid lines are used for sections with large curvature to further improve the detection accuracy of the lateral deviation e2, angular deviation e3, and curvature by the camera. Instead of counting dashed lines, wheel pulses may be counted. The dashed lines may be solid lines.

[0043] As described above, by installing cameras under the floor in addition to the front, and following colored route lines painted on the road surface, the problem with the camera system, which is that it is exposed to disturbances from the natural environment and cannot recognize white lines, is solved.

[0044] The camera is mounted in a position that can accommodate the route line having a curvature from the minimum turning radius of the vehicle to straight ahead in its field of view, and detects the lateral deviation e2, angular deviation e3, and route line curvature between the vehicle's center of gravity position and the route line to perform feedback control, and also obtains route information such as distance, gradient, curvature, planned speed, branch position, merging position, and latitude and longitude orientation from the route line to perform feedforward control. Thus, it functions as a route recognition system for an autonomous vehicle equipped with a visual device that follows a driving route laid on a road. This corresponds to the case where it functions as a route recognition system using a front-of-vehicle camera, or the case where it functions as a parallel redundant system using a front-of-vehicle camera and an under-floor camera.

Claims

1. A route recognition system for autonomous driving that travels along a route line marked on the road surface, comprising: a camera attached under the floor of a vehicle body that can visually recognize the route line without being exposed to disturbances from the natural environment such as the effects of sunlight or shadows, can determine its relative position with respect to the vehicle's center of gravity, and can keep within the visible range in a planar view the center of gravity point locus traced by the vehicle's center of gravity; the system detects the lateral deviation (e2) between the route line and the vehicle's center of gravity and the angular deviation (e3) of the vehicle body axle relative to the route line by referring to the position of this camera; and performs steering control to eliminate the detected lateral deviation (e2) and angular deviation (e3) so that the vehicle's center of gravity coincides with the route line.

2. 2. The route recognition system for automatic driving according to claim 1, wherein the lateral deviation (e2) of the center of gravity of the vehicle and the angular deviation (e3) of the vehicle axle relative to the route line are substituted into equation (4) to calculate the yaw angular velocity (yaw rate) of the vehicle, the yaw angular velocity (yaw rate) is substituted into equation (5) to calculate the turning curvature of the vehicle, and the yaw angular velocity (yaw rate) and the turning curvature are substituted into equation (6) to calculate the front wheel steering angle. [0010]

3. 2. The route recognition system for autonomous driving according to claim 1, wherein the lateral deviation (e2) of the center of gravity of the vehicle and the angular deviation (e3) of the vehicle axle relative to the coordinates and orientation of the route line are substituted into equations (10) and (11) to calculate the coordinates of the center of gravity position (center of gravity point) of the vehicle, the straight-line distance from the center of gravity point coordinates to the route junction coordinates are calculated using equation (12), the route arc angle is calculated using equations (13), (14), (17), and (18), and the route curvature is calculated using equation (15), and the actual front wheel steering angle reaching the route junction is calculated using equation (16). [0025]

4. 2. The route recognition system for automatic driving according to claim 1, wherein the route lines provided on the road surface have different colors and shapes for each piece of route information.

5. An autonomous driving route recognition system as described in claim 1, characterized in that the route lines are dotted or colored lines, enabling anticipatory transmission of curvature change information, road gradient change information, planned vehicle speed information, route branching information, route merging information, and longitude, latitude and orientation, thereby enabling look-ahead control.

6. 2. The route recognition system for an autonomous vehicle according to claim 1, wherein the camera has a field of view of a length and width sufficient to capture the arc trajectory traced by the center of gravity of the vehicle, the center of gravity being on the extension of the rear axle, when turning with the minimum turning radius.

Images