Road surface waviness estimation device and road surface waviness estimation method

Abstract

Road surface waviness estimating device configured to input a vehicle peripheral image captured by a camera (1), wherein the camera (1) is installed on a vehicle traveling on a road surface to be capable of capturing an image of a periphery of the vehicle, wherein the road surface waviness estimating device comprises: a storage unit (21) designed to store pre-acquired camera installation information; an object detection unit (22) designed to detect regularly occurring objects installed on the road surface at a constant installation interval from the vehicle peripheral image captured by the camera (1), calculate positions of the respective detected, regularly occurring objects in the vehicle peripheral image and output a calculation result as position information; a position transformation unit (23) configured to transform, based on the position information input by the object detection unit (22) and the camera installation information acquired by the storage unit (21), the positions of the respective regularly occurring objects in the vehicle peripheral image into positions of the respective regularly occurring objects in a real space and to output a transformation result as detection positions (Pd0 to Pd3) of the respective regularly occurring objects; and a road surface estimation unit (24) configured to calculate respective detection intervals (a0 to a2) between adjacent detection positions of the respective detection positions (Pd0 to Pd3) input by the position transformation unit (23), and to determine, based on the respective detection positions (Pd0 to Pd3), the respective calculated detection intervals (a0 to a2) and the constant installation interval of the regularly occurring objects, an angle of inclination (θn) between an actual road surface and a detected road surface obtained by the transformation calculation by the position transformation unit (23), wherein the actual road surface is obtained by connecting the actual positions (Pt0 to Pt3) in the installation interval of the regularly occurring objects, such that the actual positions (Pt0 to Pt3) each exist on straight lines,which pass through a lens center (O) of the camera (1) and the detection positions (Pd0 to Pd3).

Description

BACKGROUND OF THE INVENTION 1. Field of the invention

[0001] The present invention relates to a road surface waviness estimating device and a road surface waviness estimating method for estimating the waviness of a road surface on which a vehicle is driving. 2. Description of the related technology

[0002] In recent years, an increasing number of preventative safety systems have been proposed. These systems are designed to use sensors installed on a vehicle to detect objects in its periphery or surroundings, thereby preventing potential hazards. In such systems, the object's position must be acquired with extreme precision.

[0003] If, on this occasion, a camera is used as a device for perceiving the periphery of the vehicle in order to calculate the position of the object with extreme accuracy, it is important to precisely calibrate the installation state of an in-vehicle camera on the vehicle, namely the installation angle of the in-vehicle camera with respect to a road surface.

[0004] When the in-vehicle camera is calibrated before delivery at a location where the road surface is horizontal or has a constant slope, an installation angle of the in-vehicle camera relative to the road surface is generally acquired in advance, assuming the road surface is horizontal. During a vehicle journey, an object on the road surface is detected that appears in the camera image captured by the in-vehicle camera, and the object's position within the image is acquired. Then, the object's position in real space is acquired from its position in the image, based on the pre-acquired installation angle of the in-vehicle camera.

[0005] The procedure described above assumes that the pre-acquired installation angle of the in-vehicle camera and the actual installation angle of the in-vehicle camera relative to the road surface on which the vehicle is traveling are identical. However, the road surface on which the vehicle travels in the real environment has undulations, and therefore the road surface is not always perfectly horizontal. Furthermore, the undulation of the road surface changes dynamically depending on the vehicle's location. In practice, therefore, the pre-acquired installation angle of the in-vehicle camera and the actual installation angle of the in-vehicle camera relative to the road surface on which the vehicle is traveling do not always coincide.If the object's position in real space is acquired under the aforementioned assumption, there is a problem that the accuracy of capturing the object's position may decrease.

[0006] Thus, the road surface on which the vehicle travels has the waviness of the real-world environment, and therefore a technology for estimating the waviness of the road surface is required. As an example of addressing the aforementioned problem, an automatic calibration device for an in-vehicle camera (see, for example, JP 2013-238497 A) is proposed, which is designed to calibrate the installation angle of the in-vehicle camera with respect to the road surface in real time while the vehicle is traveling, using motion information about an object existing on the road surface. However, the device is not designed to estimate the waviness of the road surface on which the vehicle is traveling.

[0007] From EP 2 051 206 B1 a driver assistance system is known which recognizes repeating road markings in an image taken by a camera and calculates the incline or gradient of the road section in front of the vehicle.

[0008] A system based on a similar approach is known from DE 10 2013 220 303 A1.

[0009] As described above, the road surface on which the vehicle drives has the waviness of the real environment, and therefore a technology to estimate the waviness of the road surface on which the vehicle drives is required. SUMMARY OF THE INVENTION

[0010] The present invention was made in view of the problem mentioned above and therefore has an objective of providing a road surface waviness estimating device and a road surface waviness estimating method that are capable of estimating the waviness of a road surface on which a vehicle is driving.

[0011] The problem is solved by a road surface waviness estimation device having the features of claim 1. Advantageous embodiments are described in the dependent claims. The problem is further solved by a road surface waviness estimation method according to claim 6.

[0012] According to the present invention, the configuration for detecting regularly occurring or appearing objects from the vehicle peripheral image captured by the camera is as follows: calculating the detection positions of the respective regularly occurring objects by using the camera installation information to transform the positions of the respective regularly occurring objects in the image into the positions in real space; calculating the detection intervals between the adjacent detection positions from the respective detection positions; and estimating, from the respective detection positions, the calculated detection intervals, and the installation interval of the regularly occurring objects, the inclinations of the actual road surface relative to the road surface obtained by the transformation calculation from the image to real space.As a result, the road surface waviness estimating device and the road surface waviness estimating method can be acquired, which are capable of estimating the waviness of the road surface on which the vehicle is driving. BRIEF DESCRIPTION OF THE DRAWINGS Fig. Figure 1 is a diagram illustrating a road surface waviness estimating device according to a first embodiment of the present invention. Fig. Figure 2 is a flowchart illustrating a series of operations of the road surface waviness estimator. Fig. 1. Fig. Figure 3 is an explanatory diagram illustrating an example of an operation of an object acquisition unit. Fig. 1 for capturing regularly occurring objects from a vehicle peripheral image. Fig. Figure 4 is an explanatory diagram illustrating a method for estimating the slope of a road surface, which is represented by a road surface estimation unit of Fig. 1 is executed. Fig. Figure 5 is an explanatory diagram illustrating a first calculation method for calculating a slope θ0 of the road surface using the road surface estimation unit of Fig. 1. Fig. Figure 6 is an explanatory diagram illustrating a second calculation method for calculating the slope θ0 of the road surface using the road surface estimation unit of Fig. 1. Fig. Figure 7 is an explanatory diagram illustrating a procedure for calculating a slope θ. n (where n≥1) of the road surface by the road surface estimation unit of Fig. 1. Fig. Figure 8 is an embodiment diagram illustrating a road surface waviness estimation device according to a second embodiment of the present invention. Fig. Figure 9 is a flowchart illustrating a series of operations of the road surface waviness estimator. Fig. 8. Fig. Figure 10 is an explanatory diagram illustrating an example of an operation of the object detection unit of Fig. 8 for capturing regularly occurring objects from a vehicle peripheral image. Fig. Figure 11 is an embodiment diagram illustrating a road surface waviness estimation device according to a third embodiment of the present invention. Fig. Figure 12 is a flowchart illustrating a series of operations of the road surface waviness estimator. Fig. 11. Fig. Figure 13 is an explanatory diagram illustrating a method for sequentially acquiring the times at which the respective regularly occurring objects appear at a reference position, from a temporal change in the vehicle peripheral image by the object detection unit of Fig. 11. Fig. Figure 14 is an explanatory diagram illustrating an example of an operation of the object acquisition unit of Fig. 11 for capturing regularly occurring objects from the vehicle peripheral image. DETAILED DESCRIPTION OF PREFERRED EXECUTION FORMS

[0013] Next, a road surface waviness estimating device and a road surface waviness estimating method according to exemplary embodiments of the present invention will be described with reference to the accompanying drawings. In the illustration of the drawings, the same or corresponding components are designated with the same reference numerals, and the overlapping description thereof is omitted here. First embodiment

[0014] Fig. Figure 1 is a diagram illustrating a road surface waviness estimation device 2 according to a first embodiment of the present invention. Fig. Figure 1 also illustrates a camera 1, which is designed to input a recording result or capture result to the road surface waviness estimating device 2. The road surface waviness estimating device 2 is designed to use objects (hereinafter referred to as regularly occurring objects) that are regularly arranged on a road surface at a constant installation interval to estimate the waviness of the road surface.

[0015] In Fig. In this case, camera 1 is installed on a vehicle driving on the road surface to capture or record the vehicle's periphery and is, for example, a monocular camera. Camera 1 is designed to capture or record an image of the vehicle's periphery and to output the resulting image as a vehicle periphery image to the road surface waviness estimation device 2.

[0016] The road surface waviness estimator 2 comprises a memory unit 21, an object detection unit 22, a position transformation unit 23, and a road surface estimation unit 24. The road surface waviness estimator 2 is implemented, for example, by a CPU for executing programs stored in memory and a processing circuit, for example, a system LSI.

[0017] The storage unit 21 is, for example, constructed by means of a memory, and the storage unit 21 is designed to store the installation information about the camera 1 and one or more installation intervals of the regularly occurring objects that are installed on the road surface.

[0018] The installation information (hereinafter referred to as camera installation information) for camera 1 includes the camera 1's installation angle relative to the road surface and its height relative to the road surface and is acquired in advance. The camera 1's installation angle is acquired in advance and represents the camera 1's position relative to the road surface, assuming the road surface is a horizontal plane. When camera 1 is calibrated prior to delivery at a location where the road surface is horizontal or has a constant slope, the camera 1's installation angle relative to the road surface is generally acquired in advance. Similarly, the camera 1's height relative to the road surface is acquired in advance.

[0019] In the first embodiment, regularly occurring objects, whose installation interval is regulated by a law or the like, are used as the objects. Examples of these regularly occurring objects include a segment of a lane line (hereinafter referred to as a broken lane line), whose line type is a broken or dashed line, and a guidepost. In this way, it is assumed that the installation interval of the regularly occurring objects is regulated by a law or the like, and thus the installation interval of the regularly occurring objects can be acquired in advance and is known.In other words, the road surface waviness estimating device 2 according to the first embodiment is designed to use the regularly occurring objects that are installed at the constant installation interval to estimate the waviness of the road surface.

[0020] The object detection unit 22 is designed to detect regularly occurring objects from the vehicle peripheral image captured by camera 1 and calculates the position of each of the detected regularly occurring objects in the image, more precisely, the position coordinates of each regularly occurring object in the image. Furthermore, the object detection unit 22 is designed to output a calculation result, namely the calculated positions of the respective regularly occurring objects in the image, to the position transformation unit 23 as position information.

[0021] The position transformation unit 23 is designed to apply a known technology to the position information input by the object detection unit 22 and the camera installation information acquired by the storage unit 21, in order to transform the positions of the respective regularly occurring objects in the image into positions (hereinafter referred to as detection positions) of the respective regularly occurring objects in real space. Furthermore, the position transformation unit 23 is designed to output the detection positions of the respective regularly occurring objects to the road surface estimation unit 24.

[0022] The road surface estimation unit 24 is designed to calculate intervals (hereinafter referred to as detection intervals) between adjacent detection positions from the detection positions of the regularly occurring objects that were entered by the position transformation unit 23. Furthermore, the road surface estimation unit 24 is designed to estimate, from the respective detection positions entered by the position transformation unit 23, the calculated detection intervals, and the installation interval of the regularly occurring objects that was entered by the storage unit 21, the waviness of the road surface, namely the slopes of the road surface, and to output an estimation result.

[0023] On this occasion, the inclination of the road surface means an inclination of an actual road surface with respect to the road surface obtained by the transformation calculation using the position transformation unit 23, more precisely an index that represents in the angle how much the actual road surface inclines with respect to the road surface obtained by the transformation calculation.

[0024] With reference to Fig. 2 and Fig. 3 now gives a description of a series of operations of the road surface waviness estimating device 2 according to the first embodiment. Fig. Figure 2 is a flowchart illustrating the series of operations of the road surface waviness estimator 2. Fig. 1. Fig. Figure 3 is an explanatory diagram illustrating an example of an operation of the object detection unit 22. Fig. 1 for capturing the respective regularly occurring objects from the vehicle peripheral image.

[0025] A processing of the flowchart of Fig. 2 is executed repeatedly, for example, according to a predefined processing cycle. Furthermore, it is assumed here that the regularly occurring object is a line segment 3 of a broken lane line. In this case, the object detection unit 22 is set to detect the position of an edge of one of the line segments 3, namely the position of an edge on a more distant side of line segment 3 in the vehicle periphery image than the position of line segment 3 in the image.

[0026] As in Fig. As illustrated in Figure 2, in step S101 the object detection unit 22 acquires the vehicle peripheral image from camera 1, and the processing proceeds to step S102.

[0027] In step S102, the object detection unit 22 captures the regularly occurring objects from the vehicle peripheral image captured by camera 1, and the processing proceeds to step S103.

[0028] In step S103, the object acquisition unit 22 determines whether the number k of detected regularly occurring objects is equal to or greater than a set value. The set value only needs to be pre-defined as equal to or greater than 2. In step S103, if the object acquisition unit 22 determines that the number k of regularly occurring objects is equal to or greater than the set value, processing proceeds to step S104. Conversely, if the object acquisition unit 22 determines that the number k of regularly occurring objects is less than the set value, processing returns to step S101.

[0029] In step S104, the road surface estimation unit 24 reads and acquires an installation interval b of regularly occurring objects, which is stored in the storage unit 21, and the processing proceeds to step S105. In step S105, the position acquisition unit 22 acquires positions Pi0, Pi1, ..., Pi k-1 of the respective regularly occurring objects detected in the image, and the processing proceeds to step S106.

[0030] In step S106, the position transformation unit 23 transforms the respective positions Pi0, Pi1, ..., Pi based on the camera installation information acquired by the storage unit 21. k-1 of the respective regularly occurring objects that have been entered by the object acquisition unit 22, in acquisition positions Pd0, Pd1, ..., Pd k-1 of the respective regularly occurring objects, and the processing proceeds to step S107.

[0031] In step S107, the road surface estimation unit 24 is calculated from the recording positions Pd0, Pd1, ..., Pd k-1 of the respective regularly occurring objects that were entered by the position transformation unit 23, recording intervals a0, a1, ..., a k-2 of the respective regularly occurring objects. Subsequently, the road surface estimation unit estimates 24 slopes θ0, θ 1, ... θ k-2 from the road surface or from the detection positions Pd0, Pd1, ..., Pd k-1 the respective regularly occurring objects that were entered by the position transformation unit 23, the calculated detection intervals a0, a1, ..., a k-2 the respective regularly occurring objects and the installation interval b of the regularly occurring objects, which has been acquired by storage unit 21. Then the processing sequence is terminated.

[0032] Now, consider a case in which the vehicle peripheral image captured by camera 1 shows a Fig. The illustrated state is presented. In this case, k=4, and in step S105, the object acquisition unit 22 acquires the positions Pi0 to Pi3 of the respective line segments 3 in the image. In step S106, the position transformation unit 23 transforms the positions Pi0 to Pi3 of the respective line segments 3 in the image into the acquisition positions Pd0 to Pd3 of the respective line segments 3. In step S107, the road surface estimation unit 24 estimates the slopes θ0 to θ2 of the road surface from the acquisition positions Pd0 to Pd3 of the respective line segments 3, the acquisition intervals a0 to a2 of the respective line segments 3, and the installation interval b.

[0033] With reference to Fig. Section 4 now describes a method for estimating the slopes of the road surface, which is carried out by the road surface estimation unit 24. Fig. Figure 4 is an explanatory diagram illustrating the procedure for estimating the slopes of the road surface, as described by the road surface estimation unit 24. Fig. 1 is executed. Note here that one variable is represented by n and satisfies a relationship of 0≤n≤k-2.

[0034] In Fig. Figure 4 illustrates a positional relationship between the detection positions Pd0 to Pd3 of the respective regularly occurring objects existing on the road surface, obtained by the transformation calculation using the position transformation unit 23, and the actual positions Pt0 to Pt3 of the respective regularly occurring objects existing on the actual road surface. It is assumed that the actual positions Pt0 to Pt3 exist on respective straight lines passing through a lens center O of camera 1 and the detection positions Pd0 to Pd3, and that the actual positions Pt0 to Pt3 are arranged on a straight line with the constant installation interval b.

[0035] The height H0 of the lens center O from the road surface, obtained through the transformation calculation, is contained in the camera installation information and is therefore known. Furthermore, angles between the respective lines passing through the lens center O and the detection positions Pd0 to Pd3, and the road surface, obtained through the transformation calculation, can be acquired from or based on positional relationships between the lens center O and the detection positions Pd0 to Pd3.

[0036] In the estimation method mentioned above, a section is calculated between the actual positions Pt. n to Pt n+1 approximates a straight line, and an angle θ n between the road surface obtained through the transformation calculation and the straight line Pt n Pt n+1 is called the inclination θ n calculated on the road surface.

[0037] With reference to Fig. 5 and Fig. Section 6 now describes a method for calculating the slope θ0 of the road surface. Fig. Figure 5 is an explanatory diagram illustrating a first calculation method for calculating the slope θ0 of the road surface using the road surface estimation unit 24. Fig. 1. Fig. Figure 6 is an explanatory diagram illustrating a second calculation method for calculating the slope θ0 of the road surface using the road surface estimation unit 24. Fig. 1.

[0038] As one assumption for calculating the slope θ0 of the road surface, the detection position Pd0 is the shortest detection position in terms of distance from the lens center O.

[0039] Therefore, it is assumed that the detection position Pd0 and the actual position Pt0 are approximately equal to each other, and thus, as in Fig. 5 and Fig. Figure 6 illustrates Pd0=Pt0.

[0040] In Fig. 5 illustrates the first calculation method for calculating the slope θ0 of the road surface, when a0 <b, d.h. die tatsächliche Straßenoberfläche neigt sich abwärts bezüglich der durch die Transformationsberechnung erhaltenen Straßenoberfläche. Darüber hinaus ist in Fig. 5, in a triangle Pt0Pt1Pd1, ∠Pd1Pt0Pt1, namely an angle between a line Pt0Pt1 and a line Pd0Pd1, denoted by θ0; ∠Pt0Pt1Pd1, A0; and ∠Pt0Pd1Pt1, B0.

[0041] In triangle Pt0Pt1Pd1, equation (1) is established according to the law of sines, and equation (1) is transformed into equation (2). Furthermore, equation (3) is established in triangle Pt0Pt1Pd1. b / sinB0=a0 / sinA0 sinA0=a0sinB0 / b θ0=π−(A0+B0)

[0042] In this case, the respective values ​​of a0 and b are known. Furthermore, ∠OPd1Pd0, namely an angle between the line from the lens center O to the detection position Pd1 and the road surface obtained by the transformation calculation, is known, and the value of B0 is thus determined. Furthermore, 0 <A0<π / 2 erfüllt. Deshalb ist basierend auf Gleichung (2) der Wert von A0 eindeutig bestimmt. Wenn der Wert von A0 auf diese Weise bestimmt ist, wird der Wert von θ0 basierend auf Gleichung (3) bestimmt.

[0043] In Fig. Figure 6 illustrates the second calculation method for calculating the slope θ0 of the road surface when a0>b, i.e. the actual road surface slopes upwards with respect to the road surface obtained by the transformation calculation.

[0044] In Fig. 6 are, as in Fig. 5, Equation (1) to Equation (3) established, the value of B0 is determined, and π / 2 <A0<π wird erfüllt. Wie in der oben gegebenen Beschreibung wird somit auf Grundlage von Gleichung (2) der Wert von A0 eindeutig bestimmt. Als ein Ergebnis wird basierend auf Gleichung (3) der Wert von θ0 bestimmt.

[0045] With reference to Fig. Section 7 will now describe a method for calculating the inclination θ. n (where n≥1) of the road surface is given. Fig. Figure 7 is an explanatory diagram illustrating a procedure for calculating the slope θ. n (where n≥1) of the road surface by the road surface estimation unit 24 of Fig. 1.

[0046] In Fig. 7 will be one with a straight line Pd n Pd n+1 parallel line from the actual position Pt n A point of intersection is drawn between this line and a line OPt. n+1 by Pd' n+1denoted, and the length of a straight line Pt n Pd' n+1 is through a' n denoted Pt in a triangle n Pt n+1 Pd' n+1 becomes ∠Pd' n+1 Pt n Pt n+1, namely an angle between the line Pt n Pt n+1 and a straight line Pt n Pd' n+1 , than the inclination θ n the road surface is considered. In addition, ∠Pd' n+1 Pt n+1 Pt n by A n designated, and ∠Pt n+1 Pd' n+1 Pt n is through B n denoted. The height of a triangle OPd n Pd n+1 is denoted by H0, and the height of a triangle OPd' n+1 Pt n is through H0+h n designated.

[0047] On this occasion, a relationship fulfills h n and h n-1 equation (4), and a relationship between a n and a' n satisfies equation (5). hn=hn−1+bsinθn−1 a'n:an=(H0+hn):H0

[0048] Furthermore, equation (6) can be derived from equations (4) and (5). a'n=an(H0+hn−1+bsinθn−1) / H0

[0049] In the triangle Pt n Pt n+1 Pd' n+1 will a' n determined on the basis of equation (6), and the value of b is known. Furthermore, ∠OPd n+1 Pd n , namely an angle between the line from the lens center O to the detection position Pd n+1 and the road surface obtained through the transformation calculation, known, and the value of B n is thus determined. Therefore, a calculation similar to that in Fig. 5 and Fig. 6 illustrated calculation methods for the value of θ n certainly.

[0050] In this way, the road surface estimation unit 24 can do the in Fig. 5 to Fig. 7 illustrated calculation methods for calculating the slopes θ0 to θ n use the road surface.

[0051] Thus, according to the first embodiment, the regularly occurring objects are detected from the vehicle periphery image captured by the camera, and the detection positions of the respective regularly occurring objects are calculated by using the camera installation information to transform the positions of the respective regularly occurring objects in the image into positions in real space. Furthermore, the detection intervals between the adjacent detection positions are calculated from the respective detection positions, and from the respective detection positions, the calculated detection intervals, and the installation interval of the regularly occurring objects, the slopes of the actual road surface relative to the road surface obtained by the transformation calculation from the image to real space are estimated.

[0052] This design allows the waviness of the road surface around the vehicle to be calculated from the image captured by the camera, without the need for any sensors other than the camera itself. As a result, the waviness of the road surface on which the vehicle is driving can be estimated.

[0053] Furthermore, in the first embodiment, regularly occurring objects installed at the known installation interval are used, and the installation interval of these regularly occurring objects is used to estimate the slope of the road surface. Compared to a case where the installation interval is calculated, as in the second and third embodiments of the present invention, which will be described later, the processing load imposed by calculating the road surface waviness can thus be reduced. Second embodiment

[0054] According to the second embodiment of the present invention, in contrast to the first embodiment, a description of the road surface estimating device 2 is given, which is configured to use regularly occurring objects, each of which displays a numerical value representing a distance, in order to calculate the installation interval of the regularly occurring objects. Note that, according to the second embodiment, a description for the same points as in the first embodiment is omitted, and mainly a description for points different from those in the first embodiment is given.

[0055] In the first embodiment, the regularly occurring object, whose installation interval is regulated by a law or the like, is used as the object, and the installation interval of the regularly occurring objects is known. In contrast, in the second embodiment, the regularly occurring object, on which a numerical value representing a distance is displayed, is used as the object, and the numerical value displayed on the regularly occurring object is used to calculate the installation interval of the regularly occurring objects.

[0056] Examples of the regularly occurring object on which the numerical value representing a distance is displayed include an inter-vehicle distance indicator installed at regular intervals for a driver to recognize a distance between their own vehicle and a vehicle ahead, and an indicator installed on a highway that displays a distance of the current location from a starting point.

[0057] Fig. Figure 8 is an embodiment diagram illustrating the road surface waviness estimation device 2 according to the second embodiment of the present invention. Fig. 8 contains the road surface waviness estimating device 2, the storage unit 21, the object detection unit 22, the position transformation unit 23, the road surface estimating unit 24 and an installation interval acquisition unit 25.

[0058] The installation interval acquisition unit 25 is configured to recognize the numerical values ​​displayed on the regularly occurring objects in the vehicle peripheral image input from camera 1, in order to calculate the installation interval of the regularly occurring objects. The installation interval acquisition unit 25 is configured to output the installation interval acquired by the calculation to the road surface estimation unit 24. The road surface estimation unit 24 uses the installation interval input from the installation interval acquisition unit 25 to estimate the slope of the road surface, which differs from the first embodiment.

[0059] With reference to Fig. 9 and Fig. 10 now gives a description of a series of operations of the road surface waviness estimating device 2 according to the second embodiment. Fig. Figure 9 is a flowchart illustrating the series of operations of the road surface waviness estimator 2. Fig. 8. Fig. Figure 10 is an explanatory diagram illustrating an example of an operation of the object detection unit 22. Fig. 8 for capturing the respective regularly occurring objects from the vehicle peripheral image.

[0060] The processing of the flowchart of Fig. 9 is executed repeatedly, for example, with a predefined processing cycle. Furthermore, it is assumed here that the regularly occurring object is an inter-vehicle distance display 4. In this case, the object detection unit 22 is set to detect a position where each inter-vehicle distance display 4 and the road surface are in contact, as the position of each inter-vehicle distance display 4 in the image.

[0061] As in Fig. Figure 9 illustrates that in step S201 the object detection unit 22 and the installation interval acquisition unit 25 acquire the vehicle peripheral image from camera 1 and the processing proceeds to step S202.

[0062] The object detection unit 22 performs steps S202 and S203, which have the same processing as steps S102 and S103 of Fig. There are 2.

[0063] In step S204, the installation interval acquisition unit 25 recognizes the numerical values ​​displayed on the regularly occurring objects from the vehicle peripheral image captured by camera 1, thereby calculating the installation interval b of the regularly occurring objects, and the processing proceeds to step S205.

[0064] The object acquisition unit 22 performs step S205, which is the same processing as step S105, and the processing proceeds to step S206.

[0065] Position transformation unit 23 performs step S206, which has the same processing as step S106 of Fig. 2 is, and processing proceeds to step S207.

[0066] In step S207, the road surface estimation unit 24 is calculated from the recording positions Pd0, Pd1, ..., Pd k-1 of the respective regularly occurring objects that were entered by the position transformation unit 23, the recording intervals a0, a1, ..., a k-2 of the respective regularly occurring objects. Subsequently, the road surface estimation unit 24 estimates the slopes θ0, θ 1, ..., θ k-2 from the road surface or from the detection positions Pd0, Pd1, ..., Pd k-1 the respective regularly occurring objects that were entered by the position transformation unit 23, the calculated detection intervals a0, a1, ..., a k-2the respective regularly occurring objects and the installation interval b acquired by the installation interval acquisition unit 25. Then the processing sequence is terminated. The road surface estimation unit 24 calculates the slopes θ0 to θ k-2 the road surface by the calculation method described in the first embodiment, i.e. the one described in Fig. 5 to Fig. 7 illustrated calculation methods.

[0067] Now, consider a case in which the vehicle peripheral image captured by camera 1 shows a Fig. The illustrated state is presented in Figure 10. In this case, k is 3, and in step S204, the installation interval acquisition unit 25 recognizes numerical values ​​from the vehicle periphery image captured by camera 1, which are displayed on the respective inter-vehicle distance indicators 4, namely 0 m, 50 m, and 100 m, thereby calculating the installation interval b and recognizing interval b as 50 m. In step S205, the object detection unit 22 acquires the positions Pi0 to Pi2 of the respective inter-vehicle distance indicators 4 in the image.

[0068] In step S206, the position transformation unit 23 transforms the positions Pi0 to Pi2 of the respective inter-vehicle distance display boards 4 in the image into the detection positions Pd0 to Pd2 of the respective inter-vehicle distance display boards 4. In step S207, the road surface estimation unit 24 estimates the slopes θ0 and θ1 of the road surface from the detection positions Pd0 to Pd2 of the respective inter-vehicle distance display boards 4, the detection intervals a0 and a1 of the respective inter-vehicle distance display boards 4, and the installation interval b (= 50 m).

[0069] Thus, according to the second embodiment, the regularly occurring objects, each displaying a numerical value indicating the distance, are used; the regularly occurring objects are detected from the vehicle periphery image captured by the camera; the numerical values ​​displayed on the respective detected regularly occurring objects are recognized, thereby calculating the installation interval; and the calculated installation interval is used to estimate the slopes of the road surface.

[0070] As a result, the same effects as those of the first embodiment are acquired, and even if the installation interval known in the first embodiment is not known, the waviness of the road surface can be estimated. Third embodiment

[0071] According to the third embodiment of the present invention, in contrast to the first embodiment, a description of the road surface estimation device 2 is given, which is designed to use regularly occurring objects whose installation interval is unknown and which have a recurring pattern, in order to calculate the installation interval of the regularly occurring objects. Note that according to the third embodiment, a description for the same points as in the first and second embodiments is omitted, and descriptions are mainly given for points different from those in the first and second embodiments.

[0072] In the second embodiment, the regularly occurring object on which the numerical value indicating the distance is displayed is used as the object, and the installation interval of the regularly occurring objects is calculated from the numerical values. In contrast, in the third embodiment, the regularly occurring objects whose installation interval is unknown and which have the characteristic of repeated occurrence are used as the object, and this characteristic is used to calculate the installation interval of the regularly occurring objects.

[0073] Examples of regularly occurring objects, whose installation interval is unknown and which have the characteristic of repeated occurrence, include a guardrail post, a windbreak wall post, and a sand barrier fence post.

[0074] Fig. Figure 11 is an embodiment diagram illustrating the road surface waviness estimation device 2 according to the third embodiment of the present invention. Fig. 11 contains the road surface waviness estimator 2, the storage unit 21, the object detection unit 22, the position transformation unit 23, the road surface estimator 24, and an installation interval estimator 26. In addition, motion information is input to the road surface waviness estimator 2 from a vehicle information acquisition unit 5.

[0075] The vehicle information acquisition unit 5 is designed to acquire motion information about the vehicle, such as vehicle speed and yaw rate, and to output the acquired motion information to the installation interval estimation unit 26. The vehicle information acquisition unit 5 is constructed, for example, by a vehicle speed sensor and a yaw rate sensor.

[0076] The object detection unit 22 is designed to detect regularly occurring objects with the characteristic of repeated occurrence from the vehicle peripheral image recorded by the camera 1, sequential acquisition, from the temporal change of the vehicle peripheral image recorded by the camera 1, of occurrence times at which the respective regularly occurring objects occur at a reference position, and output the occurrence times to the installation interval estimation unit 26.

[0077] The installation interval estimation unit 26 is configured to acquire motion information from the vehicle information acquisition unit 5. Furthermore, the installation interval estimation unit 26 is configured to use the respective occurrence times input by the object detection unit 22 and the motion information corresponding to those occurrence times to calculate the interval of regularly occurring objects. The installation interval estimation unit 26 is configured to output the calculated and estimated installation interval to the road surface estimation unit 24. The road surface estimation unit 24 is configured to use the installation interval input from the installation interval estimation unit 26 to estimate the slope of the road surface, which differs from the first embodiment.

[0078] With reference to Fig. 12 to Fig. 14 now gives a description of a series of operations of the road surface waviness estimating device 2 according to the third embodiment. Fig. Figure 12 is a flowchart illustrating the series of operations of the road surface waviness estimator 2. Fig. 11. Fig. Figure 13 is an explanatory diagram illustrating a method for sequentially acquiring occurrence times at which the respective regularly occurring objects appear at a reference position Pr, from a temporal change in the vehicle peripheral image by the object detection unit 22. Fig. 11. Fig. Figure 14 is an explanatory diagram illustrating an example of an operation of the object detection unit 22. Fig. 11 for capturing the respective regularly occurring objects from the vehicle peripheral image.

[0079] The processing of the flowchart of Fig. Procedure 12 is executed repeatedly, for example, with a predefined processing cycle. Furthermore, it is assumed here that the recurring object is a post 6 of a guardrail. In this case, the object detection unit 22 is set to detect the position where the post 6 and the road surface make contact, as the position of each of the respective posts 6 in the image.

[0080] As in Fig. As illustrated in Figure 12, the object detection unit performs 22 steps S301 to S303, which are the same processing as steps S101 to S103 of Fig. There are 2.

[0081] In step S304, the object detection unit 22 sequentially acquires from the temporal change in the vehicle peripheral image recorded by camera 1 the times at which the respective regularly occurring objects appear at the previously set reference position.

[0082] For example, a case is considered in which the temporal change of the vehicle peripheral image captured by camera 1 is as in Fig. 13 illustrated states are shown. For the purpose of easier understanding, in Fig. 13 two posts 6 designated by 6_1 and 6_2 in an ascending order, starting from the post 6 that is closest to the reference position Pr.

[0083] In Fig. 13. The object detection unit 22 acquires an occurrence time t0 from the temporal change in the vehicle periphery image captured by camera 1, i.e., when the post 6 first appears at the reference position Pr, i.e., when the first post 6_1 appears at the reference position Pr. Furthermore, the object detection unit 22 outputs the acquired occurrence time t0 to the installation interval estimation unit 26.

[0084] Next, the object detection unit 22 acquires an occurrence time t1 from the temporal change in the vehicle periphery image captured by camera 1, when the second post 6_2 occurs at the reference position Pr. Furthermore, the object detection unit 22 outputs the acquired occurrence time t1 to the installation interval estimation unit 26.

[0085] In this way, the object detection unit 22 sequentially acquires, from the temporal changes in the vehicle peripheral image captured by camera 1, the times at which the respective regularly occurring objects appear at the reference position. If the number of times at which the object detection unit 22 acquires is represented by j, j only needs to be preset to satisfy j ≥ 2.

[0086] With renewed reference to Fig. In step S305, 12 calculates the installation interval estimation unit 26 from the respective occurrence times entered by the object detection unit 22 and the movement information corresponding to the respective occurrence times, the installation interval b of the regularly occurring objects.

[0087] More precisely, the installation interval estimation unit calculates 26 time intervals Δt1 (=t1-t0), ..., Δt j (=t j-1 t j-2 ) each between occurrence times side by side in a time series of the respective occurrence times t0, t1, ..., t j-1 The installation interval estimation unit 26 calculates, from the motion information corresponding to the respective occurrence times t0 to t j-1 This corresponds to the distances entered by the object detection unit 22, over which the vehicle traveled in the respective time intervals Δt1 to Δt jdrives. Then the installation interval estimation unit 26 calculates from the distances corresponding to the respective calculated time intervals Δt1 to Δt j The installation interval b corresponds to this. For example, the average of the distances corresponding to the respective calculated time intervals Δt1 to Δt can be used. j correspond to the installation interval b being set.

[0088] The object acquisition unit 22 performs step S306, which is the same processing as step S105 of Fig. 2 is, and processing proceeds to step S307.

[0089] Position transformation unit 23 performs step S307, which has the same processing as step S106 of Fig. 2 is, and processing proceeds to step S308.

[0090] In step S308, the road surface estimation unit 24 is calculated from the detection positions Pd0, Pd1, ..., Pd k-1of the respective regularly occurring objects that were entered by the position transformation unit 23, the recording intervals a0, a1, ..., a k-2 of the respective regularly occurring objects. Subsequently, the road surface estimation unit 24 estimates the slopes θ0, θ 1, ... θ k-2 from the road surface or from the detection positions Pd0, Pd1, ..., Pd k-1 the respective regularly occurring objects that were entered by the position transformation unit 23, the calculated detection intervals a0, a1, ..., a k-2 the respective regularly occurring objects and the installation interval b acquired by the installation interval estimation unit 26. Then the processing sequence is terminated. The road surface estimation unit 24 calculates the slopes θ0 to θ k-2 the road surface by the calculation method described in the first embodiment, i.e. the one described in Fig. 5 to Fig. 7 illustrated calculation methods.

[0091] Now, consider a case in which the vehicle peripheral image captured by camera 1 shows a Fig. The illustrated state is presented in 14. In this case, k=9, and in step S306, the object acquisition unit acquires 22 positions Pi0 to Pi8 of the respective posts 6 in the image.

[0092] In step S307, the position transformation unit 23 transforms the positions Pi0 to Pi8 of the respective posts 6 in the image into detection positions Pd0 to Pd8 of the respective posts 6. In step S308, the road surface estimation unit 24 estimates slopes θ0 to θ7 of the road surface from the detection positions Pd0 to Pd8 of the respective posts 6, detection intervals a0 to a7 of the respective posts 6, and the installation interval b acquired by the installation interval estimation unit 26.

[0093] Thus, according to the third embodiment, the regularly occurring object is used, whose installation interval is unknown and which has the characteristic of repeated occurrence; the times of occurrence at which the respective regularly occurring objects appear at the reference position in the image are acquired sequentially from the temporal change in the vehicle peripheral image recorded by the camera, and the installation interval is calculated from the respective times of occurrence and the motion information corresponding to the respective times of occurrence.

[0094] As a result, the same effects as those of the first embodiment are acquired, and even if the installation interval known in the first embodiment is not known, the waviness of the road surface can be estimated.

[0095] Note that a description has been given individually for the first to third embodiments, but the embodiments disclosed in the first to third embodiments can be combined with one another as desired. Furthermore, the method for acquiring the installation interval of the regularly occurring objects is described in each of the first to third embodiments, but the method is not limited to these methods. The installation interval can be acquired by a different method than the method for acquiring the detection interval using the vehicle peripheral image.

[0096] Note that by applying the present invention to reduce the distance measurement performance of the camera for object detection, which may be caused by the waviness of the road surface in the real environment, to estimate the waviness of the road surface during vehicle travel, during which the waviness of the road surface is not always constant, and to use the estimation result for distance measurement, an inhibition of the reduction in performance can be expected.

[0097] Furthermore, the present invention enables the undulation of the road surface on which the vehicle travels to be detected in advance, and thus an application of the present invention to a control technology for active suspension can be expected. Moreover, the present invention enables the detection of a road rut (depression) that can cause spontaneous traffic congestion on a highway in advance, and thus an application of the present invention to, for example, a technology for directing the driver's attention to speed can be expected.

Claims

[1] Road surface waviness estimating device configured to input a vehicle peripheral image captured by a camera (1), wherein the camera (1) is installed on a vehicle traveling on a road surface to be capable of capturing an image of a periphery of the vehicle, the road surface waviness estimating device comprising: a storage unit (21) designed to store pre-acquired camera installation information; an object detection unit (22) designed to detect regularly occurring objects installed on the road surface at a constant installation interval from the vehicle peripheral image captured by the camera (1), calculate positions of the respective detected, regularly occurring objects in the vehicle peripheral image and output a calculation result as position information; a position transformation unit (23) configured to transform, based on the position information input by the object detection unit (22) and the camera installation information acquired by the storage unit (21), the positions of the respective regularly occurring objects in the vehicle peripheral image into positions of the respective regularly occurring objects in a real space and to output a transformation result as detection positions (Pd0 to Pd3) of the respective regularly occurring objects; and a road surface estimation unit (24) configured to calculate respective detection intervals (a0 to a2) between adjacent detection positions of the respective detection positions (Pd0 to Pd3) input by the position transformation unit (23), and to determine, based on the respective detection positions (Pd0 to Pd3), the respective calculated detection intervals (a0 to a2) and the constant installation interval of the regularly occurring objects, an angle of inclination (θn) between an actual road surface and a detected road surface obtained by the transformation calculation by the position transformation unit (23), wherein the actual road surface is obtained by connecting the actual positions (Pt0 to Pt3) in the installation interval of the regularly occurring objects, such that the actual positions (Pt0 to Pt3) each exist on straight lines,which pass through a lens center (O) of the camera (1) and the detection positions (Pd0 to Pd3). [2] Road surface waviness estimating device according to claim 1, wherein: the regularly occurring objects are installed at the constant installation interval, which is known; the storage unit (21) is designed to further store the constant installation interval; and The constant installation interval stored in the storage unit (21) is entered into the road surface estimation unit (24). [3] Road surface waviness estimating device according to claim 1, wherein: the regularly occurring object is designed to display a numerical value that represents a distance; The road surface waviness estimating device further comprises an installation interval acquisition unit (25) configured to detect the regularly occurring objects from the vehicle peripheral image captured by the camera (1) and to recognize the numerical values ​​displayed on the respective detected, regularly occurring objects in order to calculate the constant installation interval; and the constant installation interval calculated by the installation interval acquisition unit (25) is entered into the road surface estimation unit (24). [4] Road surface waviness estimating device according to claim 1, wherein: the constant installation interval of the regularly occurring objects is unknown, and the regularly occurring objects each have a characteristic of repeated occurrence; the object detection unit (22) is designed for sequential acquisition of temporal changes in the vehicle peripheral image captured by the camera (1), of times of occurrence at which the respective regularly occurring objects appear at a reference position in the vehicle peripheral image, and furthermore for outputting an acquisition result; The road surface waviness estimating device further comprises an installation interval estimating unit (26) configured to acquire movement information about the vehicle from a vehicle information acquisition unit (5), which is configured to acquire the movement information, and to calculate the constant installation interval from the respective occurrence times entered by the object detection unit (22) and the movement information corresponding to the respective occurrence times; and the constant installation interval calculated by the installation interval estimation unit (26) is entered into the road surface estimation unit (24). [5] Road surface waviness estimation device according to claim 4, wherein the installation interval estimation unit (26) is configured to calculate time intervals between occurrence times side by side in a time series of the respective occurrence times that have been entered by the object detection unit (22), calculate, from the motion information corresponding to the respective occurrence times, distances over which the vehicle travels in the respective calculated time intervals, and calculate the constant installation interval from the distances corresponding to the respective calculated time intervals. [6] Road surface waviness estimation method for estimating the waviness of a road surface by using a vehicle periphery image captured by a camera (1), wherein the camera (1) is installed on a vehicle traveling on the road surface in order to be able to capture an image of the periphery of the vehicle, the road surface waviness estimation method comprising: an object detection step for detecting regularly occurring objects that are installed on the road surface at a constant installation interval from the vehicle peripheral image captured by the camera (1), calculating positions of the respective detected, regularly occurring objects in the vehicle peripheral image and outputting a calculation result as position information; a position transformation step to transform, based on the position information output in the object detection step and previously acquired camera installation information, the positions of the respective regularly occurring objects in the vehicle peripheral image into positions of the respective regularly occurring objects in a real space and to output a transformation result as detection positions (Pd0 to Pd3) of the respective regularly occurring objects; and a road surface estimation step to calculate respective detection intervals (a0 to a2) between adjacent detection positions of the respective detection positions (Pd0 to Pd3) that were output in the position transformation step, and to determine, based on the respective detection positions (Pd0 to Pd3), the respective calculated detection intervals (a0 to a2) and the constant installation interval of the regularly occurring objects, an angle of inclination (θn) between an actual road surface and a detected road surface that was obtained by the transformation calculation in the position transformation step, wherein the actual road surface is obtained by connecting the actual positions (Pt0 to Pt3) in the installation interval of the regularly occurring objects, so that the actual positions (Pt0 to Pt3) each exist on straight lines that pass through a lens center (O) of the camera (1) and the detection positions (Pd0 to Pd3).

Images