Axis deviation estimation device

Abstract

This invention provides an axle offset estimation device capable of accurately estimating the axle offset of an on-board sensor. A road surface recognition unit (50) identifies a road surface (30) via a laser imaging detection and ranging device (17) mounted on a vehicle (1). An estimation unit (52) estimates the axle offset of the laser imaging detection and ranging device (17) mounted on the vehicle (1). A plane creation unit (51) creates an imaginary plane (30A) of the road surface (30) based on the recognition result of the road surface recognition unit (50). The estimation unit (52) estimates the axle offset of the laser imaging detection and ranging device (17) based on the inclination of the imaginary plane (30A) relative to the reference plane (17A) of the laser imaging detection and ranging device (17).

Description

Technical Field

[0001] The present invention relates to an axial offset estimation device. Background Art

[0002] Conventionally, there is known a technique for detecting an axial misalignment of a vehicle-mounted sensor based on a detection result of a lane (white line) on which a vehicle is traveling.

[0003] Patent document 1 describes a driving assistance device for a vehicle, which uses images from a vehicle-mounted radar and a vehicle-mounted camera to identify the environment outside the vehicle and assist the driver's driving operations, wherein the left and right lanes on the road are identified based on an image of the vehicle's travel direction captured by the vehicle-mounted camera, and the intersection of the approximate straight lines in the captured image plane of the left and right lanes is used as the horizontal line position in the image plane. The vehicle's posture angle is estimated based on changes in the horizontal line position in the image plane, and an actuator for adjusting the beam radiation angle of the vehicle-mounted radar in the up and down directions is controlled based on the posture angle.

[0004] Patent document 2 describes an axial deviation measuring device of a vehicle-mounted sensor, which includes a position measuring device mounted on a vehicle and measuring the position of a front object existing in front, capturing an image in the same direction, and when it is determined that the vehicle is moving straight based on the detection of a white line based on the image data, calculating the movement vector of the stopped object based on the object position data, and measuring the axial deviation of the position measuring device based on the movement vector.

[0005] Patent document 3 describes an external sensor device for a vehicle, which corrects the deviation angle between the axial direction of a vehicle-mounted camera and an obstacle detection sensor mounted on the vehicle on the same axis and the vehicle's traveling direction, wherein at least two lanes drawn on the road are identified based on a captured image obtained by the vehicle-mounted camera capturing the vehicle's traveling direction, and it is determined whether the vehicle is traveling straight. When the vehicle is traveling straight, the deviation angle is calculated from the at least two identified lanes, and based on the calculated deviation angle, the position of the obstacle identified from the captured image and the position of the obstacle detected by the obstacle detection sensor are corrected.

[0006] Patent document 4 describes an axial offset detection device for a vehicle-mounted laser imaging detection and ranging device, which detects the axial offset of the laser imaging detection and ranging device mounted on the vehicle, wherein a first white line is detected based on shooting information obtained by a camera installed inside the vehicle cabin, a second white line is detected based on point information obtained by a laser imaging detection and ranging device installed outside the vehicle cabin, and the axial offset angle of the laser imaging detection and ranging device relative to the camera is estimated. If a comparison result between a detection result of the first white line and a detection result after rotating the detection result of the second white line at a specified angle with the laser imaging detection and ranging device as the center of the installation position of the vehicle satisfies specified conditions, the specified angle is estimated as the axial offset angle of the laser imaging detection and ranging device relative to the camera.

[0007] Prior art literature

[0008] Patent Literature

[0009] Patent Document 1: Japanese Patent Application Publication No. 2003-307561

[0010] Patent Document 2: Japanese Patent Application Publication No. 2004-198159

[0011] Patent Document 3: Japanese Patent Application Publication No. 2014-228943

[0012] Patent Document 4: Japanese Patent Application Publication No. 2020-143969 Summary of the invention

[0013] Problems to be solved by the invention

[0014] In the previous technology for detecting the axial offset of the vehicle-mounted sensor based on the detection results of the white line (lane), the axial offset of the vehicle-mounted sensor cannot be estimated with high accuracy when one of the left or right white lines cannot be detected, the detected white line is not a straight line, the vehicle is not moving straight, etc.

[0015] An object of the present invention is to provide an axial misalignment estimating device capable of estimating the axial misalignment of an on-vehicle sensor with high accuracy.

[0016] Means for solving problems

[0017] The present invention provides an axle offset estimation device for a vehicle-mounted sensor, which comprises:

[0018] A road surface recognition unit that recognizes the road surface by using a laser radar installed in the vehicle; and

[0019] an estimating unit configured to estimate an axial misalignment amount of the vehicle-mounted sensor mounted on the vehicle,

[0020] The axial deviation estimation device further comprises a plane creating unit, which creates a virtual plane of the road surface based on the recognition result of the road surface recognition unit.

[0021] The estimating unit estimates the axis offset based on the inclination of the virtual plane with respect to a reference plane of the laser radar.

[0022] Effects of the Invention

[0023] According to the axial misalignment estimation device of the present invention, it is possible to estimate the axial misalignment of the vehicle-mounted sensor with high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 FIG. 1 is a diagram showing an example of a vehicle 1 to which the axial deviation estimating device of the present invention is applied.

[0025] Figure 2 Yes means Figure 1 A diagram showing an example of the scanning range of a laser imaging detection and ranging device.

[0026] Figure 3 : is a flowchart showing an example of the processing performed by the axis deviation estimating unit 55 .

[0027] Figure 4 1 is a diagram showing an example of detection of three points on the road surface by the axial deviation estimating unit 55 .

[0028] Figure 5 1 is a diagram showing an example of the laser imaging detection and distance measuring device 17 in a state where no axial deviation occurs.

[0029] Figure 6 This is a diagram showing an example of creation of a virtual plane in a state where the laser imaging detection and distance measuring device 17 is not misaligned.

[0030] Figure 7 1 is a diagram showing an example of the laser imaging detection and distance measuring device 17 in a state where an axial deviation has occurred.

[0031] Figure 8 This is a diagram showing an example of creation of a virtual plane in a state where the laser imaging detection and distance measuring device 17 is misaligned.

[0032] Fig. 9 1 is a diagram showing an example of the laser imaging detection performed by the estimating unit 52 and the estimation of the axial deviation of the distance measuring device 17 .

[0033] Fig.10 1 is a diagram showing an example of continuous detection of points on the road surface by the axial deviation estimating unit 55 .

[0034] Fig.111 is a diagram showing an example of updating of the axis misalignment amount by the axis misalignment estimating unit 55 .

[0035] Fig.12 : is a flowchart showing an example of the updating process of the axis deviation amount performed by the axis deviation estimating unit 55 .

[0036] Fig.13 1 is a diagram showing an example of estimating the axis misalignment amount based on a plurality of virtual planes by the axis misalignment estimating unit 55 .

[0037] Fig.14 This is a diagram showing an example of the virtual planes 30A to 30C when there is no axis deviation of the laser imaging detection and distance measuring device 17 .

[0038] Fig.15 It is a diagram showing an example of the virtual planes 30A to 30C when there is an axial deviation of the laser imaging detection and distance measuring device 17 .

[0039] Fig.16 1 is a flowchart showing an example of a process of estimating the axial misalignment amount based on a plurality of virtual planes, which is performed by the axial misalignment estimating unit 55 .

[0040] Description of Reference Numerals

[0041] 1 Vehicle

[0042] 17 Laser imaging detection and ranging device (LiDAR)

[0043] 30 Road surface

[0044] 30A~30C Imaginary plane

[0045] 50 Road surface recognition unit

[0046] 51 Graphic Production Department

[0047] 52 Presumption

[0048] 55 axis deviation estimation unit (estimation device) DETAILED DESCRIPTION

[0049] Hereinafter, an embodiment of the axial misalignment estimation device of the present invention will be described with reference to the accompanying drawings. Hereinafter, for the convenience of description, the center of the vehicle body is taken as the origin, the front-rear (vehicle length) direction is defined as the X-axis, the left-right (vehicle width) direction is defined as the Y-axis, and the up-down direction is defined as the Z-axis. Furthermore, the X-axis or Y-axis direction is appropriately described as the horizontal axis direction, and the Z-axis direction is appropriately described as the vertical axis direction.

[0050] The rotation direction with the X-axis as the axis corresponds to the roll direction, the rotation direction with the Y-axis as the axis corresponds to the pitch direction, the rotation direction with the Z-axis as the axis corresponds to the yaw direction, the X-axis corresponds to the roll axis, the Y-axis corresponds to the pitch axis, and the Z-axis corresponds to the yaw axis.

[0051] <Vehicle 1 to which the axle misalignment estimation device of the present invention is applied>

[0052] Figure 1 FIG. 1 is a diagram showing an example of a vehicle 1 to which the axial deviation estimating device of the present invention is applied. Figure 2 FIG. 1 is a diagram showing an example of the scanning range of a millimeter wave radar. Figure 1 As shown, the vehicle 1 includes a propulsion device 3, a braking device 4, a steering device 5, an external sensor 6, a vehicle sensor 7, a navigation device 8, a driving operating member 9, and a vehicle control system 14 of the control device 10. The components of the vehicle control system 14 are connected to each other through a communication unit such as a CAN (Controller Area Network) in a manner capable of transmitting signals.

[0053] The propulsion device 3 is a device that applies driving force to the vehicle 1, and includes, for example, a power source and a transmission. The power source includes at least one of an internal combustion engine such as a gasoline engine, a diesel engine, and an electric motor. In the present embodiment, the propulsion device 3 includes an automatic transmission and a shift actuator that changes the shift position (gear) of the automatic transmission. The braking device 4 is a device that applies braking force to the vehicle 1, and includes, for example, a brake caliper that presses a brake pad against a brake rotor and an electric cylinder that supplies hydraulic pressure to the brake caliper. The braking device 4 may also include an electric parking brake device that limits the rotation of the wheel through a cable. The steering device 5 is a device for changing the steering angle of the wheel, and includes, for example, a rack and pinion mechanism that steers the wheel and an electric motor that drives the rack and pinion mechanism. The propulsion device 3, the braking device 4, and the steering device 5 are controlled by the control device 10.

[0054] The external sensor 6 is a device (external acquisition device) that captures electromagnetic waves, sound waves, etc. from the periphery of the vehicle 1 to detect objects outside the vehicle and obtain peripheral information of the vehicle 1. The external sensor 6 includes an external camera 16 and a laser imaging detection and ranging device 17.

[0055] The external vehicle camera 16 is, for example, a digital camera that utilizes a solid-state imaging element such as a CCD or CMOS. The external vehicle camera 16 is installed on the vehicle 1 (more specifically, a rearview mirror) with the optical axis in the front direction of the vehicle 1, i.e., in front of the vehicle 1, and captures the front of the vehicle 1 (X-axis direction).

[0056] The laser imaging detection and ranging device 17 is a LIDAR (Laser Imaging Detection and Ranging), which is an example of a laser radar installed in the vehicle 1. In addition, the laser imaging detection and ranging device 17 is an example of a vehicle-mounted sensor for estimating the axis offset. Figure 2 As shown, the laser imaging detection and ranging device 17 transmits electromagnetic waves (transmitting waves) to the outside of the vehicle with the optical axis A as the center, and receives reflected waves from surrounding objects to scan the surroundings of the vehicle 1. Thus, the laser imaging detection and ranging device 17 obtains ranging data and detects the positions of objects around the vehicle 1. The ranging data includes the direction of the object observed by the laser imaging detection and ranging device 17, and the distance between the laser imaging detection and ranging device 17 and the object. The electromagnetic waves transmitted from the laser imaging detection and ranging device 17 can be electromagnetic waves of any wavelength, such as ultraviolet rays, visible rays, near infrared rays, etc.

[0057] The laser imaging detection and ranging device 17 is installed at a predetermined position on the front of the vehicle 1. When mounted on the vehicle 1 (when shipped from the factory), the optical axis A of the laser imaging detection and ranging device 17 is set in the front, and its scanning range (scan range) is centered on the optical axis A, is set within a predetermined angle θz around the Z axis (yaw axis), and is set within a predetermined angle θy around the Y axis (pitch axis).

[0058] The vehicle sensor 7 includes a vehicle speed sensor 18 for detecting the speed of the vehicle 1. In addition to the vehicle speed sensor 18, the vehicle sensor 7 may also include an acceleration sensor for detecting the acceleration of the vehicle 1, a yaw rate sensor for detecting the angular velocity of the vehicle 1 around a vertical axis (Z axis), an azimuth sensor for detecting the direction of the vehicle 1, and the like.

[0059] The navigation device 8 is a device for obtaining the current position of the vehicle 1 and providing route guidance to the destination, etc., and includes a GPS receiving unit 20 and a map storage unit 21. The GPS receiving unit 20 determines the position (latitude, longitude) of the vehicle 1 based on signals received from artificial satellites (positioning satellites). The map storage unit 21 is composed of a known storage device such as a flash memory and a hard disk, and stores map information.

[0060] The driving operation elements 9 are provided in the vehicle interior and receive input operations performed by a user to control the vehicle 1. The driving operation elements 9 include a steering wheel 22, an accelerator pedal 23, a brake pedal 24 (brake operation element), and a shift lever 25.

[0061] The control device 10 is an electronic control unit (ECU) including a processor such as a CPU, a non-volatile memory (ROM), and a volatile memory (RAM). The control device 10 performs various vehicle controls by executing arithmetic processing according to a program by the processor. In addition, the processing of the axle offset estimation device of the present invention is performed by the processor of the control device 10. The control device 10 can be configured as a single hardware or as a unit composed of multiple hardware. In addition, at least a part of each functional unit of the control device 10 can be implemented by hardware such as LSI (large scale integrated circuit), ASIC (application specific integrated circuit), FPGA (field programmable gate array), or by a combination of software and hardware.

[0062] The control device 10 controls the propulsion device 3, the braking device 4, and the steering device 5 in a manner that at least avoids objects existing around the vehicle 1 based on the image obtained by the vehicle exterior camera 16 and the distance measurement data obtained by the laser imaging detection and distance measurement device 17. For example, the control device 10 can control the vehicle 1 by performing autonomous driving processing for making the vehicle 1 autonomously drive and performing auxiliary processing for assisting the driver in driving while avoiding objects existing around the vehicle 1.

[0063] In order to perform such control of the vehicle 1 , the control device 10 includes an outside world recognition unit 41 , a vehicle position identification unit 42 , an action planning unit 43 , and a travel control unit 44 .

[0064] The external recognition unit 41 appropriately controls the external sensor 6 and obtains the detection result from the external sensor 6. The external recognition unit 41 recognizes objects such as pedestrians and vehicles 1 that exist around the vehicle 1 based on the detection result of the external sensor 6. The external recognition unit 41 obtains the position of the object based on the vehicle 1 based on the distance measurement data obtained by the laser imaging detection and distance measurement device 17. In addition, the external recognition unit 41 obtains the size of the object based on the detection result of the external sensor 6 including the image obtained by the vehicle-external camera 16, the distance measurement data obtained by the laser imaging detection and distance measurement device 17, etc., and uses a well-known method such as machine learning to determine the type of the object (for example, the object is a bridge tower, a street lamp, etc.) according to the detection result of the external sensor 6.

[0065] The vehicle position determination unit 42 detects the position of the vehicle 1 based on the signal from the GPS receiving unit 20 of the navigation device 8. In addition, the vehicle position determination unit 42 may obtain the vehicle speed and yaw rate from the vehicle sensor 7 in addition to the signal from the GPS receiving unit 20, and determine the position and posture of the vehicle 1 using so-called inertial navigation.

[0066] The driving control unit 44 controls the propulsion device 3, the braking device 4, and the steering device 5 to drive the vehicle 1 according to the driving control instruction from the action planning unit 43. More specifically, when the action planning unit 43 indicates the trajectory that the vehicle 1 should travel, the driving control unit 44 controls the propulsion device 3, the braking device 4, and the steering device 5 to avoid the objects located around the vehicle 1 based on the positions and sizes of the objects acquired by the external recognition unit 41, and drives the vehicle 1 as far as possible along the trajectory.

[0067] The action planning unit 43 performs a following driving process to follow the vehicle traveling ahead, and an evasive process to safely stop the vehicle 1 when the driver cannot take over the driving operation when switching from automatic driving to manual driving (a process for performing a so-called minimum risk strategy (MRM)). The action planning unit 43 calculates the trajectory that the vehicle 1 should travel in each process, and outputs an instruction to the driving control unit 44 to make the vehicle 1 travel along the trajectory.

[0068] However, after the vehicle 1 starts traveling, the action planning unit 43 appropriately instructs the external recognition unit 41 to estimate the axis deviation of the laser imaging detection and distance measuring device 17, except for the case where the following driving process and the retreat process are preferably performed. The external recognition unit 41 obtains the axis deviation amount of the laser imaging detection and distance measuring device 17 in the axis deviation estimation, and outputs a restriction signal to the action planning unit 43 when it is determined that there is an axis deviation that should limit the processing that the action planning unit 43 can perform.

[0069] If a restriction signal is input, the action planning unit 43 restricts the executable processing. For example, if a restriction signal is input, the action planning unit 43 prohibits the execution of the follow-up driving processing, but enables the execution of the avoidance processing. On the other hand, when no restriction signal is input, the action planning unit 43 can execute the follow-up driving processing and the avoidance processing.

[0070] The external recognition unit 41 includes an axis deviation estimation unit 55, which performs the axis deviation estimation of the laser imaging detection and distance measuring device 17. The axis deviation estimation unit 55 is an example of the axis deviation estimation device of the present invention, and includes a road surface recognition unit 50, a plane preparation unit 51, and an estimation unit 52. The road surface recognition unit 50 recognizes the road surface on which the vehicle 1 is traveling based on the distance measurement data obtained by the laser imaging detection and distance measuring device 17.

[0071] The plane creating unit 51 creates a virtual plane of the road surface based on the recognition result of the road surface recognition unit 50. The virtual plane of the road surface is a plane in the three-dimensional external space recognized by the external recognition unit 41, which has a position and direction substantially consistent with the road surface.

[0072] The estimating unit 52 estimates the axis offset of the laser imaging detection and ranging device 17 based on the inclination of the imaginary plane produced by the plane producing unit 51 relative to the reference plane of the laser imaging detection and ranging device 17. The reference plane of the laser imaging detection and ranging device 17 is a reference plane for ranging of the laser imaging detection and ranging device 17. The laser imaging detection and ranging device 17 is installed on the vehicle 1 in such a manner that the reference plane is in a predetermined direction (for example, a horizontal direction).

[0073] The estimating unit 52 obtains the axis deviation amount of the laser imaging detection and distance measuring device 17 by the above-mentioned axis deviation estimation, and outputs a limit signal to the action planning unit 43 indicating that the axis deviation exists to such an extent that the processing that the action planning unit 43 can perform should be limited.

[0074] Next, the operation of the vehicle 1 thus configured will be described. When the vehicle 1 starts to travel or during travel, the action planning unit 43 appropriately instructs the outside recognition unit 41 to start the laser imaging detection and the axial offset estimation of the distance measuring device 17. The estimation unit 52 of the outside recognition unit 41 receives the instruction, performs the axial offset estimation process, and outputs a limit signal to the action planning unit 43 when it is estimated that there is a large axial offset. In other words, the control device 10 functions as an axial offset estimation device that performs the laser imaging detection and the axial offset estimation of the distance measuring device 17 (vehicle-mounted sensor), and performs vehicle control based on the estimation result.

[0075] <Processing by Axis Displacement Estimation Unit 55>

[0076] Figure 3 2 is a flowchart showing an example of processing performed by the axis deviation estimation unit 55. The axis deviation estimation unit 55 of the control device 10 executes, for example, Figure 3 Processing shown. Figure 3 The processing shown may be performed, for example, when the engine of the vehicle 1 is started, or may be repeatedly performed while the vehicle 1 is traveling.

[0077] First, the axial misalignment estimation unit 55 identifies the road surface on which the vehicle 1 is traveling based on the laser imaging detection and the distance measurement result of the distance measuring device 17 through the road surface recognition unit 50 (step S31). Next, the axial misalignment estimation unit 55 detects three or more points on the road surface identified in step S31 through the plane creation unit 51 (step S32). The road surface on which the vehicle 1 is traveling and the detection of three or more points on the road surface will be described later (see Figure 4 wait).

[0078] Next, the axis deviation estimation unit 55 generates a virtual plane based on the three or more points detected in step S32 through the plane generation unit 51 (step S33). Next, the axis deviation estimation unit 55 calculates the inclination of the virtual plane generated in step S32 relative to the reference plane of the laser imaging detection and ranging device 17 (step S34). Next, the axis deviation estimation unit 55 estimates the axis deviation amount of the laser imaging detection and ranging device 17 based on the inclination calculated in step S34 through the estimation unit 52 (step S35), and ends a series of processing. It should be noted that in step S35, the axis deviation estimation unit 55 may also calculate an average (including a weighted average) of a plurality of axis deviation amounts, the average of the plurality of axis deviation amounts including the axis deviation amount of the laser imaging detection and ranging device 17 estimated based on the inclination calculated in step S34 this time and the axis deviation amount of the laser imaging detection and ranging device 17 estimated based on the inclination calculated in step S34 in the past. The creation of the virtual plane and the estimation of the axis offset amount based on the inclination will be described later (see Figures 5 to 9 ).

[0079] <Detection of Three Points on the Road Surface by the Axis Displacement Estimation Unit 55>

[0080] Figure 4 3 is a diagram showing an example of detection of three points on a road surface by the axis deviation estimation unit 55. The road surface 30 is a road surface of a single-sided two-lane road on which the vehicle 1 travels. The road marking 30a is a white road marking such as a lane marking, and is a high-brightness reflective band with high reflection intensity. In addition, the road marking 30a is not limited to a white road marking, and may be a road marking of other colors such as yellow or blue. The road surface recognition unit 50 of the axis deviation estimation unit 55 recognizes the road surface 30 based on the laser imaging detection and the distance measurement result of the distance measuring device 17.

[0081] The plane generating unit 51 of the axis deviation estimating unit 55 detects three or more points on the road surface 30 identified by the road surface identifying unit 50. For example, the plane generating unit 51 uses the information of the reflection intensity included in the sensing data of the laser imaging detection and distance measuring device 17 to detect three or more points where the reflection intensity of each point on the road surface 30 is greater than or equal to a predetermined value. Figure 4 In the example of FIG. 1 , points 31 to 33 of a road sign 30 a which are high-brightness reflection bands on a road surface 30 are detected.

[0082] Here, a case where the portion of the road surface 30 where the vehicle 1 is located is parallel to the portion of the road surface 30 including the points 31 to 33 will be described.

[0083] <Laser imaging detection and distance measuring device 17 in a state where no axis deviation occurs>

[0084] Figure 51 is a diagram showing an example of the laser imaging detection and distance measuring device 17 in a state where no axial deviation occurs. Figure 6 1 is a diagram showing an example of the creation of a virtual plane in a state where the axis offset of the laser imaging detection and ranging device 17 is not generated. The laser imaging detection and ranging device 17 (X) is the laser imaging detection and ranging device 17 observed from the front of the vehicle 1 (X-axis direction). The laser imaging detection and ranging device 17 (Y) is the laser imaging detection and ranging device 17 observed from the side of the vehicle 1 (Y-axis direction). The reference plane 17A is the reference plane for ranging of the above-mentioned laser imaging detection and ranging device 17.

[0085] like Figure 5 As shown, the laser imaging detection and ranging device 17 is arranged on the vehicle 1 in such a manner that the reference plane 17A is parallel to the horizontal direction (XY plane) of the vehicle 1 when no axial deviation occurs, that is, in such a manner that it is orthogonal to the vertical direction (Z-axis direction) of the vehicle 1.

[0086] Figure 6 The virtual plane 30A shown is a virtual plane of the road surface 30 created by the plane creation unit 51. The plane creation unit 51 creates the virtual plane 30A including the detected points 31 to 33. Here, the space identified by the distance measurement data from the laser imaging detection and distance measurement device 17 is set to an xyz space composed of an x-axis, a y-axis, and a z-axis. The virtual plane 30A is an xy plane including the x-axis and the y-axis.

[0087] like Figure 6 As shown, in a state where no axis deviation of the laser imaging detection and distance measuring device 17 occurs, the virtual plane 30A coincides with the reference plane 17A.

[0088] <Laser imaging detection and distance measuring device 17 for generating axis deviation>

[0089] Figure 7 1 is a diagram showing an example of the laser imaging detection and distance measuring device 17 in a state where an axial deviation has occurred. Figure 8 1 is a diagram showing an example of creating a virtual plane in a state where an axis deviation occurs in the laser imaging detection and distance measuring device 17. In this example, an axis deviation in the roll direction and an axis deviation in the pitch direction occur in the laser imaging detection and distance measuring device 17.

[0090] like Figure 7 As shown in the laser imaging detection and ranging device 17 (X), when an axis offset in the roll direction occurs in the laser imaging detection and ranging device 17, the reference plane 17A is tilted relative to the horizontal direction (XY plane) of the vehicle 1 in the YZ plane. The tilt amount on the YZ plane is set as the axis offset θ Roll .

[0091] In addition, if Figure 7 As shown in the laser imaging detection and ranging device 17 (Y), when an axis offset in the pitch direction occurs in the laser imaging detection and ranging device 17, the reference plane 17A is tilted relative to the horizontal direction (XY plane) of the vehicle 1 in the XZ plane. The tilt amount on the XZ plane is set as the axis offset θ Pitch .

[0092] like Figure 8 As shown, in a state where the axis deviation of the laser imaging detection and distance measuring device 17 occurs, the virtual plane 30A including the points 31 to 33 is inclined relative to the reference plane 17A.

[0093] <Laser Imaging Detection by Estimation Unit 52 and Estimation of Axis Misalignment of Distance Measuring Device 17>

[0094] Fig. 9 1 is a diagram showing an example of the laser imaging detection performed by the estimation unit 52 and the estimation of the axis deviation of the distance measuring device 17. Fig. 9 In Figure 8 The following describes an example in which an axis offset in the roll direction and an axis offset in the pitch direction are generated in the laser imaging detection and ranging device 17 as in the example of FIG.

[0095] The virtual plane 30A in the xyz space can be represented by, for example, ax+by+cz+d=0. That is, the plane creation unit 51 creates the virtual plane 30A by calculating coefficients a, b, c, and d so that the virtual plane 30A represented by ax+by+cz+d=0 passes through the three points 31 to 33.

[0096] The vector n0 perpendicular to the reference plane 17A can be expressed as the following formula (1). The vector n1 perpendicular to the virtual plane 30A can be expressed as the following formula (2).

[0097]

Number 1

[0098]

[0099]

Number 2

[0100]

[0101] The vectors of equations (1) and (2) are decomposed into the yz plane and the xz plane. The vector n0 shown in equation (1) can be expressed as the vector n in equation (3) below in the yz plane. yz0 The vector n1 shown in equation (2) can be expressed in the yz plane as the vector n shown in equation (4) below: yz1 The vector n0 shown in formula (1) can be expressed in the xz plane as the vector n shown in formula (5) below. xz0The vector n1 shown in formula (2) can be expressed in the xz plane as the vector n shown in formula (6) below. xz1 Express it that way.

[0102]

Number 3

[0103]

[0104]

Number 4

[0105]

[0106]

Number 5

[0107]

[0108]

Number 6

[0109]

[0110] Axis offset in the roll direction θ Roll The vector n shown in formula (3) can be used yz0 The vector n shown in (4) yz1 The inner product of is calculated as follows (7). The axis offset θ in the pitch direction Pitch We can use the vector n shown in formula (5) xz0 The vector n shown in (6) xz1 The inner product of is calculated as shown in the following formula (8).

[0111]

Number 7

[0112]

[0113]

Number 8

[0114]

[0115] The estimation unit 52 calculates the above-mentioned axis deviation amount θ Roll ,θ Pitch As the estimated value of the axis offset of the laser imaging detection and distance measuring device 17. Alternatively, the estimating unit 52 may also use the above-mentioned axis offset θ Roll ,θ Pitch The sum or average is calculated as the estimated value of the axis offset of the laser imaging detection and ranging device 17.

[0116] In this way, the axial offset estimation unit 55 recognizes the road surface 30 by the laser imaging detection and ranging device 17 installed in the vehicle 1, creates an imaginary plane 30A of the road surface 30 according to the recognition result, and estimates the axial offset of the laser imaging detection and ranging device 17 mounted on the vehicle 1 based on the inclination of the imaginary plane 30A relative to the reference plane 17A of the laser imaging detection and ranging device 17.

[0117] Therefore, even if the left and right white lines of the road surface 30 cannot be detected, or the white lines of the road surface 30 are not straight lines, or the vehicle 1 is not moving straight, as long as the portion of the road surface 30 where the vehicle 1 is located is parallel to the portion of the road surface 30 where the imaginary plane 30A is made, the axial offset of the laser imaging detection and ranging device 17 can be estimated with high precision.

[0118] In addition, by detecting points (e.g., points 31 to 33) whose reflection intensity is greater than or equal to a predetermined value as points on the road surface 30 for creating the virtual plane 30A, the detection accuracy of the points on the road surface 30 can be improved even at points far from the vehicle 1. Therefore, the axis offset of the laser imaging detection and distance measuring device 17 can be estimated with high accuracy.

[0119] <Continuous Detection of Points on the Road Surface by the Axis Displacement Estimation Unit 55>

[0120] Fig.10 1 is a diagram showing an example of continuous detection of points on the road surface by the axial deviation estimating unit 55 . Fig.10 The scanning trajectory 17B shown is the scanning trajectory of the laser imaging detection and distance measuring device 17. As described above, the plane creating unit 51 detects three or more points where the reflection intensity of each point on the road surface 30 is greater than a predetermined value. Therefore, the points on the road marking 30a such as the white line on the road surface 30 are detected. Fig.10 In the example, points 61 to 76 are detected.

[0121] When more than four points on the road surface 30 can be detected, the plane generating unit 51 can generate multiple virtual planes of the road surface 30. For example, the plane generating unit 51 can generate a virtual plane including points 61 to 63, a virtual plane including points 64 to 66, a virtual plane including points 67 to 69, etc. Alternatively, the plane generating unit 51 can overlap points between virtual planes to generate a virtual plane including points 61 to 63, a virtual plane including points 62 to 64, a virtual plane including points 63 to 65, etc.

[0122] In such a case, the estimating unit 52 estimates the axis misalignment amount based on, for example, a virtual plane that is closest to the vehicle 1. Fig.10In the example shown, the laser imaging detection and distance measuring device 17 estimates the axis deviation based on the virtual plane including the points 61 to 63. As a result, the virtual plane can be created in a state where it is more difficult to generate an angle difference between the portion of the road surface 30 where the vehicle 1 is located and the portion of the road surface 30 where the virtual plane is created. Therefore, the axis deviation of the laser imaging detection and distance measuring device 17 can be estimated with high accuracy.

[0123] Alternatively, the plane preparation unit 51 may prepare a plurality of virtual planes of the road surface 30 as described above, and the estimation unit 52 may calculate the average of the axis offsets estimated from the plurality of virtual planes as the axis offset of the laser imaging detection and ranging device 17. Thus, even if there are irregularities on the road surface 30 or there is a deviation in the ranging result of the laser imaging detection and ranging device 17 due to vibration of the vehicle 1 or the accuracy of the laser imaging detection and ranging device 17, the axis offset of the laser imaging detection and ranging device 17 can be estimated with high accuracy.

[0124] Alternatively, the average of the axis misalignment may be a weighted average in which a higher weight is set for the axis misalignment estimated from the virtual plane closer to the vehicle 1. Thus, while being able to prevent the influence of the unevenness of the road surface 30 and the deviation of the distance measurement result of the laser imaging detection and distance measurement device 17, the axis misalignment can be estimated by giving importance to the virtual plane that is unlikely to have an angle difference with the portion of the road surface 30 where the vehicle 1 is located, and the axis misalignment of the laser imaging detection and distance measurement device 17 can be estimated with high accuracy.

[0125] <Update of Axis Misalignment Amount by Axis Misalignment Estimation Unit 55>

[0126] Fig.11 : is a diagram showing an example of updating of the axis deviation amount by the axis deviation estimating unit 55. Fig.11 In the figure, the horizontal axis represents the travel distance of the vehicle 1, and the vertical axis represents the axis offset amount (for example, the axis offset amount θ) estimated by the estimation unit 52 based on the inclination of the virtual plane 30A relative to the reference plane 17A. Roll Or the axis offset θ Pitch ).

[0127] The axial misalignment estimation unit 55 continuously and repeatedly executes, for example, Figure 3 The axis misalignment amount change 111 is a change in the axis misalignment amount estimated by the estimating unit 52 of the axis misalignment estimating unit 55 based on the virtual plane 30A, according to the travel distance of the vehicle 1 .

[0128] The estimating unit 52 updates the estimation result of the axis misalignment based on the axis misalignment change 111 when a predetermined condition is satisfied. The updating of the estimation result of the axis misalignment means reflecting the new estimation result of the axis misalignment in the process using the estimation result of the axis misalignment by the estimating unit 52. The process using the estimation result of the axis misalignment by the estimating unit 52 is, for example, a process that limits the process that can be executed by the action planning unit 43 when the axis misalignment is large. Specifically, when the predetermined condition is satisfied, the estimating unit 52 determines whether the axis misalignment at this time is greater than a threshold value, and outputs a restriction signal to the action planning unit 43 when the axis misalignment is greater than the threshold value.

[0129] The predetermined condition is that the change in the axis offset amount based on the inclination of the imaginary plane 30A relative to the reference plane 17A during the travel of the vehicle 1 over a predetermined distance d is less than or equal to a threshold value. The change in the axis offset amount is, for example, the size of the difference between the minimum axis offset amount and the maximum axis offset amount (the change range of the axis offset amount).

[0130] exist Fig.11 In the example, at the time point of the travel distance p1, the vehicle 1 has traveled a predetermined distance d while the change in the axle misalignment amount is substantially 0, so the estimating unit 52 updates the result of the axle misalignment estimation. Specifically, at the time point of the travel distance p1, the estimating unit 52 determines whether the axle misalignment amount estimated at that time point is greater than a threshold value, and outputs a restriction signal to the action planning unit 43 when the axle misalignment amount is greater than the threshold value.

[0131] In addition, at the time point of the travel distance p2 (> p1), the distance traveled by the vehicle 1 reaches the predetermined distance d in a state where the change amount of the axle misalignment is substantially 0, so the estimating unit 52 updates the estimation result of the axle misalignment. Specifically, at the time point of the travel distance p2, the estimating unit 52 determines whether the axle misalignment estimated at the time point is greater than a threshold value, and outputs a restriction signal to the action planning unit 43 when the axle misalignment is greater than the threshold value.

[0132] In addition, the estimating unit 52 does not update the estimation result of the axis misalignment amount during the period between the time point of the travel distance p1 and the time point of the travel distance p2. That is, even if the axis misalignment amount estimated during this period is greater than the threshold value, the estimating unit 52 does not output a restriction signal to the action planning unit 43. Thus, even if the portion of the road surface 30 where the vehicle 1 is located is temporarily not parallel to the portion of the road surface 30 where the virtual plane is created due to a change in the inclination of the road surface 30 during the travel of the vehicle 1, the processing that can be executed by the action planning unit 43 can be kept from being restricted.

[0133] <Axis Misalignment Amount Update Processing by Axis Misalignment Estimation Unit 55>

[0134] Fig.12: is a flowchart showing an example of the updating process of the axis deviation amount by the axis deviation estimation unit 55. The axis deviation estimation unit 55 executes, for example, Fig.12 Processing shown.

[0135] First, the axis deviation estimation unit 55 estimates the axis deviation amount of the laser imaging detection and distance measuring device 17 (step S121). The axis deviation amount estimation in step S121 is performed, for example, by Figure 3 Next, the axis deviation estimating unit 55 sets the axis deviation amount estimated in step S121 as a reference value (step S122).

[0136] Next, the axis deviation estimation unit 55 waits for a predetermined time (step S123). Next, the axis deviation estimation unit 55 estimates the axis deviation amount of the laser imaging detection and distance measuring device 17 again (step S124). The estimation of the axis deviation amount in step S124 is performed, for example, by Figure 3 The processing shown is performed.

[0137] Next, the axis deviation estimation unit 55 determines whether the difference between the axis deviation estimated in step S124 and the currently set reference value is less than a threshold value as the variation of the axis deviation of the laser imaging detection and ranging device 17 (step S125). As an example, the threshold value can be set to a value of about 0.02°. If the difference between the estimated axis deviation and the reference value is not less than the threshold value (step S125: No), since the variation of the axis deviation of the laser imaging detection and ranging device 17 is large, the process returns to step S121.

[0138] In step S125, when the difference between the estimated axle misalignment and the reference value is less than the threshold value (step S125: Yes), the axle misalignment estimation unit 55 updates the travel distance based on the travel distance during the standby period in step S123 (step S126), because the change amount of the axle misalignment of the laser imaging detection and distance measuring device 17 is small. The travel distance here is the distance that the vehicle 1 continuously travels when the change amount of the axle misalignment is small. Fig.12 At the start of the process, φ is set to 0. In step S126, the axial misalignment estimating unit 55 updates the travel distance by adding the distance traveled during the standby period in step S123 to the current travel distance.

[0139] Next, the axle misalignment estimation unit 55 determines whether the travel distance updated in step S126 is greater than or equal to a predetermined distance (predetermined distance d) (step S127). If the travel distance after the update is greater than or equal to the predetermined distance (step S127: Yes), the vehicle 1 has traveled the predetermined distance or greater in a state where the change in the axle misalignment amount is small. In this case, the axle misalignment estimation unit 55 determines whether the axle misalignment amount is greater than or equal to a threshold value based on the estimated result of the current axle misalignment amount (step S128). The threshold value is a value of the axle misalignment amount that serves as a reference for whether the processing based on the axle misalignment amount estimation result of the axle misalignment amount by the axle misalignment estimation unit 55, which can be executed by the action planning unit 43, should be limited.

[0140] In step S128, when the axis misalignment amount is not equal to or greater than the threshold value (step S128: No), the axis misalignment estimation unit 55 ends a series of processing. When the axis misalignment amount is equal to or greater than the threshold value (step S128: Yes), the axis misalignment estimation unit 55 outputs a restriction signal indicating that the axis misalignment exists to such an extent that the processing that the action planning unit 43 can execute should be restricted to the action planning unit 43 (step S129), and ends a series of processing.

[0141] In step S127 , when the updated travel distance is not greater than the predetermined distance (step S127 : No), the axial misalignment estimating unit 55 sets the axial misalignment amount estimated in step S124 as a new reference value (step S130 ), and the process returns to step S123 .

[0142] like Fig.11 , Fig.12 As described above, the axis misalignment estimation unit 55 updates the estimation result of the axis misalignment amount when a predetermined condition is satisfied. The predetermined condition is that the change in the axis misalignment amount based on the inclination of the virtual plane 30A with respect to the reference plane 17A of the laser imaging detection and distance measurement device 17 is less than a threshold value when the vehicle 1 travels for a predetermined distance or more.

[0143] The fact that the vehicle 1 travels a predetermined distance or more while the change in the axis misalignment is less than the threshold value means that during this period, the inclination of the portion of the road surface 30 where the vehicle 1 travels and the portion of the road surface 30 where the imaginary plane is created are substantially constant. Therefore, by updating the estimation result of the axis misalignment in this case, the estimation result of the axis misalignment is not updated when the portion of the road surface 30 where the vehicle 1 is located is not parallel to the portion of the road surface 30 where the imaginary plane is created, and the axis misalignment of the laser imaging detection and distance measuring device 17 can be estimated with high accuracy. In addition, the temporary change in the axis misalignment caused by the state of the road surface 30, vibration, etc. can be prevented from affecting the processing based on the estimation result of the axis misalignment.

[0144] As the processing based on the estimation result of the axis offset, the processing that can be executed by the action planning unit 43 is limited when the axis offset is equal to or greater than the threshold value, but the present invention is not limited thereto. For example, the processing based on the estimation result of the axis offset may be the correction of the distance measurement data obtained by the laser imaging detection and distance measurement device 17. For example, the outside world recognition unit 41 may correct the distance measurement data obtained by the laser imaging detection and distance measurement device 17 based on the axis offset estimated by the axis offset estimation unit 55, and perform the outside world recognition processing based on the corrected distance measurement data.

[0145] <Estimation of Axis Misalignment Amount Based on Multiple Virtual Planes by Axis Misalignment Estimation Unit 55>

[0146] Fig.13 1 is a diagram showing an example of estimating the axis misalignment amount based on a plurality of virtual planes by the axis misalignment estimating unit 55 .

[0147] At a certain time t0 , the plane creating unit 51 creates a virtual plane 30A including points 31 to 33 ahead of the vehicle 1 on the road surface 30 and a virtual plane 30B including points 34 to 36 further ahead of the vehicle 1 on the road surface 30 .

[0148] Next, at time t0+t1 when the vehicle 1 advances to the portion of the road surface 30 corresponding to the virtual plane 30A, the plane creation unit 51 creates a virtual plane 30C including points 37 to 39 of the portion of the road surface 30 corresponding to the virtual plane 30B. That is, the virtual plane 30C is created by creating a virtual plane of the same portion of the road surface 30 as the virtual plane 30B based on the distance measurement data from the position at which the virtual plane 30B was created. The points 37 to 39 may be the same three points as the points 34 to 36, or may be three points different from the points 34 to 36.

[0149] <Virtual planes 30A to 30C in the absence of axis deviation of the laser imaging detection and distance measuring device 17>

[0150] Fig.14 30A to 30C when there is no axis deviation of the laser imaging detection and distance measuring device 17. Fig.13 The virtual planes 30A to 30C shown are, for example, Fig.14 The plane shown.

[0151] The inclination of the imaginary plane 30A made at time t0 relative to the reference plane 17A and the inclination of the imaginary plane 30B made at time t0 relative to the reference plane 17A are both θ0. This means that regardless of whether the laser imaging detection and distance measuring device 17 has an axis offset, the portion of the road surface 30 corresponding to the imaginary plane 30A and the portion of the road surface 30 corresponding to the imaginary plane 30B are parallel.

[0152] However, only by using the imaginary planes 30A and 30B, it is impossible to determine whether an axial offset has occurred in the laser imaging detection and ranging device 17, or whether the part of the road surface 30 where the vehicle 1 is located at time t0 is not parallel to the part of the road surface 30 corresponding to the imaginary planes 30A and 30B, or whether they are generated in combination.

[0153] exist Fig.14 In the example shown, the inclination of the virtual plane 30C created at time t0+t1 relative to the reference plane 17A is 0. This means that based on the distance measurement from the portion of the virtual plane 30A in the road surface 30, the inclination of the portion of the road surface 30 corresponding to the virtual plane 30B (virtual plane 30C) can be correctly detected.

[0154] That is, in this case, it can be determined that an angle θ0 exists between the portion of the road surface 30 where the vehicle 1 is located at time t0 and the portion of the road surface 30 corresponding to the imaginary planes 30A and 30B, and no axial deviation occurs in the laser imaging detection and distance measuring device 17 .

[0155] <Virtual planes 30A to 30C in the presence of axis deviation of laser imaging detection and distance measuring device 17>

[0156] Fig.15 30A to 30C when there is an axial deviation of the laser imaging detection and distance measuring device 17. Fig.13 The imaginary planes 30A and 30B shown are, for example, Fig.15 The plane shown.

[0157] exist Fig.15 In the example shown, the inclination of the imaginary plane 30C created at time t0+t1 relative to the reference plane 17A is θ1. This means that based on the distance measurement from the imaginary plane 30A portion of the road surface 30, the detection of the inclination of the portion of the road surface 30 corresponding to the imaginary plane 30B (imaginary plane 30C) is offset by θ1.

[0158] That is, in this case, it can be determined that there is an angle of θ1 between the portion of the road surface 30 where the vehicle 1 is located at time t0 and the portion corresponding to the imaginary planes 30A and 30B in the road surface 30, and an axial offset of θ1 occurs in the laser imaging detection and ranging device 17. It should be noted that, in this case, the angle between the portion of the road surface 30 where the vehicle 1 is located at time t0 and the portion corresponding to the imaginary planes 30A and 30B in the road surface 30 is θ0 to θ1.

[0159] <Axis Misalignment Estimation Processing Based on Virtual Planes 30A to 30C by Axis Misalignment Estimation Unit 55>

[0160] Fig.16 1 is a flowchart showing an example of the process of estimating the axis deviation amount based on a plurality of virtual planes by the axis deviation estimating unit 55. The axis deviation estimating unit 55 may also replace Figure 3 The processing shown is performed Fig.16 Processing shown.

[0161] Fig.16 Steps S161 and S162 shown in FIG. Figure 3 Steps S31 and S32 are the same as those shown in FIG. 16. However, in step S162, the axial deviation estimation unit 55 detects four or more points on the road surface 30 so that two virtual planes can be created. Fig.13 In the example of FIG. 3 , six points 31 to 36 on the road surface 30 are detected.

[0162] Next, the axis deviation estimation unit 55 creates a first virtual plane and a second virtual plane based on the plurality of points detected in step S162 (step S163). The first virtual plane is a virtual plane of the portion of the road surface 30 in front of the vehicle 1, for example, Fig.13 In the example of , the second imaginary plane is the imaginary plane of the portion of the road surface 30 that is ahead of the first imaginary plane when viewed from the vehicle 1. For example, Fig.13 In the example of FIG. 3 , the imaginary plane 30B is shown.

[0163] Next, the axis misalignment estimation unit 55 determines whether the angle difference between the first virtual plane and the second virtual plane (each virtual plane) created in step S163 is less than a threshold value (step S164). The threshold value is the angle between two virtual planes that are considered to be substantially parallel. If the angle difference between each virtual plane is not less than the threshold value (step S164: No), the axis misalignment estimation unit 55 returns to step S161.

[0164] In step S164 , when the angle difference between the virtual planes is equal to or smaller than the threshold value (step S164 : Yes), the axial misalignment estimating unit 55 waits until the vehicle 1 moves to a position corresponding to the first virtual plane on the road surface 30 (step S165 ).

[0165] Next, the axis deviation estimation unit 55 identifies the road surface 30 based on the distance measurement result of the laser imaging detection and the distance measuring device 17 (step S166). Next, the axis deviation estimation unit 55 detects three points of the portion of the road surface 30 corresponding to the second virtual plane identified in step S166 (step S167). Next, the axis deviation estimation unit 55 creates a third virtual plane including the three points detected in step S167 (step S168). The third virtual plane is a virtual plane of the portion of the road surface 30 corresponding to the second virtual plane. Fig.13 In the example shown in FIG. 3 , the imaginary plane 30C is shown.

[0166] Next, the axis deviation estimation unit 55 calculates the inclination of the third virtual plane created in step S168 relative to the reference plane of the laser imaging detection and distance measurement device 17 (step S169). Next, the axis deviation estimation unit 55 estimates the axis deviation amount of the laser imaging detection and distance measurement device 17 based on the inclination calculated in step S169 (step S170), and a series of processing is terminated.

[0167] like Figure 13 to Figure 16 As shown, the axial misalignment estimating unit 55 creates a first virtual plane (e.g., virtual plane 30A) of the road surface 30 and a second virtual plane (e.g., virtual plane 30B) whose angle difference with the first virtual plane is less than a threshold value and which is farther from the vehicle 1 than the first virtual plane in the road surface 30 based on the recognition result of the road surface 30 at a certain timing (e.g., time t0). Then, the axial misalignment estimating unit 55 estimates the axial misalignment amount based on the third virtual plane (e.g., virtual plane 30C) created at a timing (e.g., time t0+t1) when the vehicle 1 is present in the portion of the road surface 30 corresponding to the first virtual plane.

[0168] That is, the axis deviation estimation unit 55 detects the mutually parallel first and second regions (parts corresponding to the imaginary planes 30A and 30B) in the road surface 30 based on the recognition result of the road surface 30, and when the vehicle 1 moves to the first region, the axis deviation amount is estimated based on the imaginary plane of the second region recognized from the first region. As a result, the portion of the road surface 30 where the vehicle 1 is located is parallel to the portion of the road surface 30 where the imaginary plane is created, so that the axis deviation of the laser imaging detection and distance measurement device 17 can be estimated with high accuracy.

[0169] It should be noted that, in step S164, the process of determining whether the angle difference between the first virtual plane and the second virtual plane is less than or equal to a threshold value has been described, but the axis deviation estimating unit 55 may also determine whether the ratio of the angle difference between the first virtual plane and the second virtual plane and the angle relative to the reference plane 17A is less than or equal to a threshold value in step S164. The angle relative to the reference plane 17A may be, for example, the angle of the first virtual plane (virtual plane 30A) relative to the reference plane 17A, the angle of the second virtual plane (virtual plane 30B) relative to the reference plane 17A, or the average value of the angle of the first virtual plane (virtual plane 30A) relative to the reference plane 17A and the angle of the second virtual plane (virtual plane 30B) relative to the reference plane 17A.

[0170] As mentioned above, although embodiment of this invention was described, this invention is not limited to the said embodiment, and it can make deformation|transformation, improvement, etc. as needed.

[0171] For example, the case where the axial offset of the laser imaging detection and ranging device 17 is estimated using the recognition result of the laser imaging detection and ranging device 17 has been described, but the axial offset of other onboard sensors mounted on the vehicle 1 may also be estimated using the recognition result of the laser imaging detection and ranging device 17. For example, when the axial offset similar to that of the laser imaging detection and ranging device 17 is generated in the off-vehicle camera 16 because the off-vehicle camera 16 is installed in the same part as the laser imaging detection and ranging device 17, the estimating unit 52 may also calculate the axial offset amount θ Roll ,θ Pitch It is estimated as the axis deviation amount of the exterior camera 16 .

[0172] In addition, although the structure of detecting three points (for example, points 31 to 33) on the road surface 30 and creating the virtual plane 30A including the three points has been described, it is also possible to detect four or more points on the road surface 30 and create the virtual plane 30A based on the four or more points. In creating the virtual plane 30A based on the four or more points, for example, robust estimation such as RANSAC (Random Sample Consensus) can be used.

[0173] Although a structure in which the processing of the axial offset estimation device of the present invention is executed by the processor of the control device 10 is described, a structure in which the processing of the axial offset estimation device of the present invention is executed by the processor of the vehicle-mounted sensor (laser imaging detection and ranging device 17, external camera 16) of the object of estimation of the axial offset can also be used.

[0174] In addition, at least the following matters are described in this specification. It should be noted that, although corresponding components and the like in the above-mentioned embodiment are shown in brackets, the present invention is not limited to these.

[0175] (1) An axial deviation estimation device (axial deviation estimation unit 55) of a vehicle-mounted sensor, comprising:

[0176] A road surface recognition unit (road surface recognition unit 50) that recognizes the road surface (road surface 30) by means of a laser radar (laser imaging detection and ranging device 17) provided in the vehicle (vehicle 1); and

[0177] an estimating unit (estimating unit 52) ​​for estimating the axis offset of the vehicle-mounted sensor (laser imaging detection and ranging device 17, vehicle-exterior camera 16) mounted on the vehicle,

[0178] The axial deviation estimation device further includes a plane creating unit (plane creating unit 51) for creating a virtual plane (virtual plane 30A) of the road surface based on the recognition result of the road surface recognition unit.

[0179] The estimating unit estimates the axis offset based on the inclination of the virtual plane with respect to a reference plane of the laser radar.

[0180] According to (1), even if the left and right white lines on the road surface cannot be detected, or the white lines on the road surface are not straight, or the vehicle is not moving straight, as long as the part of the road surface where the vehicle is located is parallel to the part of the road surface 30 where the imaginary plane is created, the axial offset of the vehicle-mounted sensor can be estimated with high accuracy.

[0181] (2) The axial deviation estimation device according to (1), wherein:

[0182] The plane creating unit detects three or more points (points 31 to 39 and 61 to 76) on the road surface, and creates the virtual plane based on the three or more points.

[0183] According to (2), a virtual plane parallel to the road surface can be created.

[0184] (3) The axial misalignment estimation device according to (2), wherein:

[0185] The three or more points are points on the road surface where the reflection intensity is equal to or greater than a predetermined value.

[0186] According to (3), the detection accuracy of points on the road surface can be improved.

[0187] (4) The axial deviation estimation device according to any one of (1) to (3), wherein:

[0188] When the plane creating unit creates a plurality of the virtual planes, the estimating unit estimates the axis misalignment amount based on the virtual plane closest to the vehicle.

[0189] According to (4), the virtual plane can be created in a state where an angle difference between the portion of the road surface where the vehicle is located and the portion of the road surface where the virtual plane is created is less likely to occur. Therefore, the axial misalignment of the vehicle-mounted sensor can be estimated with high accuracy.

[0190] (5) The axial misalignment estimation device according to any one of (1) to (3), wherein:

[0191] When the plane creating unit creates a plurality of the virtual planes, the estimating unit calculates an average value of the axis deviation amounts estimated from the respective virtual planes.

[0192] According to (5), even if there are bumps on the road surface or there are deviations in the distance measurement results of the laser radar due to vehicle vibration or the accuracy of the laser radar, the axial offset of the vehicle-mounted sensor can be estimated with high accuracy.

[0193] (6) The axial misalignment estimation device according to (5), wherein:

[0194] The average is a weighted average in which a higher weight is set for the axis misalignment amount estimated from the virtual plane closer to the vehicle.

[0195] According to (6), while being able to prevent the influence caused by the unevenness of the road surface and the deviation of the laser radar's ranging results, the axial offset of the vehicle-mounted sensor can be estimated by paying attention to a virtual plane that is difficult to produce an angle difference with the part of the road where the vehicle is located, thereby estimating the axial offset of the vehicle-mounted sensor with high precision.

[0196] (7) The axial misalignment estimation device according to any one of (1) to (6), wherein:

[0197] The estimating unit updates the estimation result of the axis offset amount when a predetermined condition is satisfied,

[0198] The predetermined condition is that a change amount of the axis misalignment amount due to the inclination is equal to or less than a threshold value while the vehicle travels for a predetermined distance or more.

[0199] According to (7), the estimated result of the axis offset can be updated when the inclination of the portion of the road surface where the vehicle is traveling and the portion where the imaginary plane is created is approximately constant. Therefore, the estimated result of the axis offset is not updated when the portion of the road surface where the vehicle is located is not parallel to the portion of the road surface where the imaginary plane is created, and the axis offset of the vehicle-mounted sensor can be estimated with high accuracy.

[0200] (8) The axial misalignment estimation device according to any one of (1) to (3), wherein:

[0201] The plane creating unit creates, based on the recognition result at a certain timing, a first virtual plane (virtual plane 30A) of the road surface and a second virtual plane (virtual plane 30B) of a portion of the road surface that has an angle difference with the first virtual plane that is less than a threshold value and is farther from the vehicle than the first virtual plane,

[0202] The estimating unit estimates the axis misalignment amount based on a third virtual plane (virtual plane 30C) created at a timing when the vehicle is present on a portion of the road surface corresponding to the first virtual plane.

[0203] According to (8), the portion of the road surface where the vehicle is located is parallel to the portion of the road surface where the third virtual plane is created, so the axial offset of the vehicle-mounted sensor can be estimated with high accuracy.

[0204] (9) The axial misalignment estimation device according to (8), wherein:

[0205] The third virtual plane is a virtual plane of a portion of the road surface corresponding to the second virtual plane.

[0206] (10) The axial deviation estimation device according to (8) or (9), wherein:

[0207] The estimating unit estimates the axis offset amount based on an inclination of the third virtual plane with respect to the reference plane.

Claims

1. An axial deviation estimation device, which is an axial deviation estimation device of a vehicle-mounted sensor, comprising: A road surface recognition unit that recognizes the road surface by using a laser radar installed in the vehicle; a plane making unit for making a virtual plane of the road surface according to the recognition result of the road surface recognition unit; and an estimating unit that estimates an axial offset of the vehicle-mounted sensor mounted on the vehicle based on an inclination of the virtual plane relative to a reference plane of the laser radar, in, The plane generating unit generates a first virtual plane of the road surface and a second virtual plane of a portion of the road surface farther from the vehicle than the first virtual plane, the second virtual plane having an angle difference with the first virtual plane being less than a threshold value, based on the recognition result at a certain time. The estimating unit estimates the axis misalignment amount based on a third virtual plane created at a timing when the vehicle is present on a portion of the road surface corresponding to the first virtual plane.

2. The axial deviation estimation device according to claim 1, wherein: The third virtual plane is a virtual plane of a portion of the road surface corresponding to the second virtual plane.

3. The axial deviation estimation device according to claim 1 or 2, wherein: The estimating unit estimates the axis offset amount based on an inclination of the third virtual plane with respect to the reference plane.

4. The axial deviation estimation device according to claim 1 or 2, wherein: The estimating unit updates the estimation result of the axis offset amount when a predetermined condition is satisfied, The predetermined condition is that a change amount of the axis misalignment amount due to the inclination is equal to or less than a threshold value while the vehicle travels for a predetermined distance or more.

5. The axial deviation estimation device according to claim 1, wherein: The plane creating unit creates the virtual plane based on points on the road surface recognized by the road surface recognition unit at which the reflection intensity of the laser radar is equal to or greater than a predetermined value.

6. The axial deviation estimation device according to claim 5, wherein: The plane creating unit detects three or more points on the road surface and creates the virtual plane based on the three or more points.

7. The axial deviation estimation device according to claim 5, wherein: Among the points on the road surface recognized by the road surface recognition unit, the points at which the reflection intensity of the laser radar is equal to or greater than a predetermined value are points on the road surface corresponding to road signs.

8. The shaft deviation estimation device according to any one of claims 5 to 7, wherein: When the plane creating unit creates a plurality of the virtual planes, the estimating unit calculates an average value of the axis deviation amounts estimated from the respective virtual planes.

9. The axial deviation estimation device according to claim 8, wherein: The average value is a weighted average value in which a higher weight is set for the axis misalignment amount estimated from the virtual plane closer to the vehicle.

10. The shaft deviation estimation device according to any one of claims 5 to 7, wherein: The estimating unit updates the estimation result of the axis offset amount when a predetermined condition is satisfied, The predetermined condition is that a change amount of the axis misalignment amount due to the inclination is equal to or less than a threshold value while the vehicle travels for a predetermined distance or more.

Images