Road surface drawing device and road surface drawing method

JPWO2024069682A5Active Publication Date: 2024-11-21STANLEY MOBILITY ELECTRIC CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2024548814
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2022-09-26
Filing Date
2022-09-26
Publication Date
2024-11-21
Estimated Expiration
2042-09-26

AI Technical Summary

Technical Problem

Existing road surface drawing devices distort images when projecting patterns onto sloped or uneven road surfaces, as they assume a horizontal surface and fail to correct for surface irregularities.

Method used

A road surface drawing device with an image acquisition unit and light projection control section that projects a reference pattern, calculates correction amounts based on the difference between the actual and reference patterns, and adjusts the light projection to draw desired patterns without distortion on any road surface slope.

Benefits of technology

Enables accurate and undistorted projection of patterns onto road surfaces regardless of slope or unevenness, ensuring clear visibility for drivers and pedestrians.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The purpose of the present disclosure is to use light projection from a vehicle onto a road surface to draw a desired drawing pattern without distortion regardless of the gradient of the road surface. A road surface drawing device (101) comprises: an image acquisition unit (11) that acquires captured-image information of a road surface in the surroundings of a vehicle (M); and a light projection control unit (12) that controls light projection onto the surrounding road surface by a light projection unit (22) which is mounted on the vehicle (M). The light projection control unit (12): causes the light projection unit (22) to project light having a first reference light projection pattern for drawing a first reference drawing pattern on a reference road surface; extracts, from the captured-image information, a first actual drawing pattern which has been drawn on the surrounding road surface by the light having the first reference light projection pattern; calculates a correction amount on the basis of the difference between the first actual drawing pattern and the first reference drawing pattern; corrects, by the correction amount, a second reference light projection pattern for drawing a second reference drawing pattern on the reference road surface, so as to prepare a corrected light projection pattern; and causes the light projection unit (22) to project light having the corrected light projection pattern.
Need to check novelty before this filing date? Find Prior Art

Description

Road surface drawing device and road surface drawing method

[0001] The present disclosure relates to a technology for drawing patterns such as figures or characters by irradiating light from a vehicle onto a road surface.

[0002] The road surface drawing device described in Patent Document 1 corrects and projects the desired image based on the positional relationship, such as the height or left-right spacing of the headlamps, in order to correctly draw the desired image on the road surface without distortion.

[0003] Japanese Patent Application Laid-Open No. 2022-86966

[0004] However, the road surface drawing device described in Patent Document 1 is based on the premise of drawing an image on a horizontal road surface, and therefore has the problem that the drawn image becomes distorted if the road surface is inclined or uneven.

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to project light from a vehicle onto a road surface to draw a desired drawing pattern without distortion, regardless of the slope or unevenness of the road surface.

[0006] The road surface drawing device disclosed herein includes an image acquisition unit that acquires captured image information, which is information about a captured image of a road surface surrounding a vehicle captured by at least one camera mounted on the vehicle, and a light-projection control unit that controls light projection onto the surrounding road surface by at least one light-projecting unit mounted on the vehicle. The light-projection control unit causes the at least one light-projecting unit to project light of a first reference light-projection pattern that draws a first reference drawing pattern on a reference road surface, extracts a first actual drawing pattern that has been drawn on the surrounding road surface by the light of the first reference light-projection pattern from the captured image information, calculates a correction amount based on a difference between the first actual drawing pattern and the first reference drawing pattern, draws a second reference drawing pattern on the reference road surface that is the same as or different from the first reference drawing pattern, corrects the second reference light-projection pattern that is the same as or different from the first reference light-projection pattern by the correction amount, and draws the second actual drawing pattern on the surrounding road surface by causing the at least one light-projecting unit to project light of the corrected light-projection pattern onto the surrounding road surface.

[0007] The road surface drawing device of the present disclosure can draw a desired drawing pattern without distortion by projecting light from a vehicle onto the road surface, regardless of the slope of the road surface. Objects, features, aspects, and advantages of the present disclosure will become more apparent from the following detailed description and the accompanying drawings.

[0008] 1 is a diagram showing how light is projected from a light-projecting unit of a vehicle onto a surrounding road surface. FIG. 2 is a diagram showing a reference drawing pattern drawn on a reference road surface. FIG. 3 is a diagram showing an actual drawing pattern drawn on a sloped road surface. FIG. 4 is a block diagram showing the configuration of a road surface drawing device and its peripheral devices according to embodiment 1. FIG. 5 is a diagram showing changes in the actual drawing pattern according to the slope of the road surface. FIG. 6 is a flowchart showing the operation of the road surface drawing device according to embodiment 1. FIG. 7 is a diagram showing the area of ​​the actual drawing pattern. FIG. 8 is a diagram showing the area of ​​each divided region of the actual drawing pattern. FIG. 9 is a diagram showing feature points of the actual drawing pattern. FIG. 10 is a diagram showing the dimensions of the actual drawing pattern. FIG. 11 is a diagram showing the dimensions of each divided region of the actual drawing pattern. FIG. 12 is a diagram showing a reference drawing pattern projected onto a flat road surface S0. FIG. 13 is a diagram showing an actual drawing pattern projected onto a non-flat road surface S1. FIG. 14 is a flowchart showing the operation of a road surface drawing device according to embodiment 4. FIG. 15 is a diagram showing how the azimuth angle of projection point P1 of the actual drawing pattern as viewed from viewpoint V is corrected to match the azimuth angle of projection point P0 of the reference drawing pattern. FIG. 16 is a diagram showing how the depression angle of projection point P1 of the actual drawing pattern as viewed from viewpoint V is corrected to match the depression angle of projection point P0 of the reference drawing pattern. 10 is a diagram showing a virtual rectangular parallelepiped viewed stereoscopically from a viewpoint V1. FIG. 11 is an x-z ​​plane view of the virtual rectangular parallelepiped viewed stereoscopically from the viewpoint V1. FIG. 12 is an x-y plane view of the virtual rectangular parallelepiped viewed stereoscopically from the viewpoint V1. FIG. 13 is an x-y plane view of a drawing pattern of the virtual rectangular parallelepiped viewed stereoscopically from the viewpoint V1. FIG. 14 is a diagram showing a virtual rectangular parallelepiped viewed stereoscopically from a viewpoint V2. FIG. 15 is an x-z ​​plane view of the virtual rectangular parallelepiped viewed stereoscopically from the viewpoint V2. FIG. 16 is an x-y plane view of the virtual rectangular parallelepiped viewed stereoscopically from the viewpoint V2. FIG. 17 is an x-y plane view of the drawing pattern of the virtual rectangular parallelepiped viewed stereoscopically from the viewpoint V2. FIG. 18 is a block diagram showing the configuration of a road surface drawing device and its peripheral devices according to a seventh embodiment. FIG. 19 is a flowchart showing the operation of the road surface drawing device according to the seventh embodiment. FIG. 19 is a diagram showing an alignment pattern projected from two light-projecting units in the same direction. FIG. 19 is a diagram showing a grid-shaped alignment pattern. FIG. 19 is a diagram showing the hardware configuration of a road surface drawing device according to each embodiment. FIG. 19 is a diagram showing the hardware configuration of a road surface drawing device according to each embodiment.

[0009] <A. First Embodiment> Fig. 1 shows how light is projected from light-projecting units 22L and 22R of a vehicle M onto a surrounding road surface 3. The light-projecting units 22L and 22R are, for example, headlights of the vehicle M. In Fig. 1, the surrounding road surface 3 is the road surface ahead of the vehicle M. In Fig. 1, the illumination range of the light-projecting units 22L and 22R is indicated by the reference numeral 6.

[0010] The road surface drawing device in the first embodiment controls the light projection of the light projectors 22L, 22R so that a predetermined pattern is drawn on the surrounding road surface 3. The pattern drawn on the surrounding road surface 3 by the light projection of the light projectors 22L, 22R (hereinafter referred to as the "drawing pattern") is directed, for example, at the driver 1 of the vehicle M. An example of the drawing pattern directed at the driver 1 is an arrow indicating the desired traveling direction of the vehicle M. The driver 1 can drive the vehicle M along the traveling direction indicated in the drawing pattern on the surrounding road surface 3.

[0011] The drawing pattern may be directed to a traffic participant such as a pedestrian 7 around the vehicle M. An example of a drawing pattern directed to the pedestrian 7 is an arrow indicating the traveling direction of the vehicle M. The pedestrian 7 can understand the traveling direction of the vehicle M by looking at the drawing pattern on the surrounding road surface 3 and take action to avoid the vehicle M.

[0012] In this way, the road surface drawing device controls the pattern of light projected from the light-projecting units 22L, 22R (hereinafter referred to as the "light-projection pattern") so that a predetermined drawing pattern such as an arrow is drawn on the surrounding road surface 3. The drawing pattern may be drawn by light projected from either one of the light-projecting units 22L, 22R.

[0013] 1 shows a case where the gradient of the surrounding road surface 3 is a horizontal road surface that does not change between the traveling position of the vehicle M and the projection position of the light projecting unit 22. Such a horizontal road surface is also called a reference road surface.

[0014] In contrast, Figure 2 shows a case where the gradient of the surrounding road surface 3 changes between the traveling position of the vehicle M and the projection position of the light-projecting unit 22. In Figure 2, the gradient of the surrounding road surface 3 is horizontal at the traveling position of the vehicle M, but becomes downward in front of the vehicle M. Even if a light projection pattern similar to that for the reference road surface is projected onto the surrounding road surface 3, which is not the reference road surface, the desired drawing pattern cannot be obtained.

[0015] 2, the surrounding road surface 3, which is assumed to be horizontal, is shown by a dashed line. If the surrounding road surface 3 is horizontal, a circular drawing pattern 8 is drawn on the surrounding road surface 3. This drawing pattern 8 is the desired drawing pattern.

[0016] Figure 3 is a view of the surrounding road surface 3 in Figure 2 as seen from above the vehicle M. The positions of the solid line A and the dashed lines B and C in Figure 3 correspond to the positions of the solid line A and the dashed lines B and C in Figure 2. As shown in Figure 3, the drawing pattern 9 on the surrounding road surface 3 with a downward slope ends up being a pattern that extends in the direction of travel of the vehicle M. In this way, even if the same pattern is projected from the light-projecting unit 22, the pattern that is actually drawn on the surrounding road surface 3 will differ depending on the gradient of the surrounding road surface 3.

[0017] Therefore, the road surface drawing device of embodiment 1 corrects the light projection pattern of the light-projecting unit 22 according to the gradient of the surrounding road surface 3 of the vehicle M, so that the desired pattern can be drawn on the surrounding road surface 3 of any gradient.

[0018] 4 is a block diagram showing the configuration of a road surface drawing device 101 and its peripheral devices according to embodiment 1. The road surface drawing device 101 corrects the light projection pattern of the light projector 22 mounted on the vehicle M in accordance with the gradient of the surrounding road surface 3, thereby suppressing distortion of a pattern (hereinafter referred to as an "actual drawing pattern") that is actually drawn on the surrounding road surface 3 by the light projected by the light projector 22. Note that there may be one or more light projectors 22, and in FIGS. 1 to 3, two light projectors 22 provided on the front left and front right sides of the vehicle M are shown as light projectors 22R and 22L.

[0019] The road surface drawing device 101 is configured to include an image acquisition unit 11 and a light projection control unit 12 .

[0020] The image acquisition unit 11 acquires an image captured by the front camera 21 of the vehicle M from the front camera 21. The front camera 21 is an example of a camera that captures an image of the surrounding road surface 3 of the vehicle M, including the light projection range of the light projection unit 22. The front camera 21 captures an image in front of the vehicle M. The front camera 21 outputs captured image information, which is information on the captured image of the surrounding road surface 3, to the image acquisition unit 11. When the light projection unit 22 projects light onto the surrounding road surface 3, the captured image of the front camera 21 includes an actual drawing pattern of the surrounding road surface 3 due to the light projection of the light projection unit 22.

[0021] The front camera 21 may be one or more. FIG. 1 illustrates one front camera 21A provided above the head of the driver 1. That is, the front camera 21 may be a single front camera 21A. Also, FIGS. 2 and 3 illustrate two front cameras 21B provided near two light-projecting units 22L, 22R of the vehicle M. That is, the front camera 21 may be a plurality of front cameras 21B provided at multiple locations on the vehicle M corresponding to the multiple light-projecting units 22L, 22R.

[0022] In order to know the amount of correction of the light projection pattern according to the surrounding road surface 3, the light projection control unit 12 first creates a first reference light projection pattern and causes the light projection unit 22 to project this. Here, the first reference light projection pattern is a light projection pattern of the light projection unit 22 that can be projected onto the reference road surface to draw the first reference drawing pattern on the reference road surface. The first reference drawing pattern is a pattern that is compared with an actual drawing pattern, which will be described later, and is used to calculate the amount of correction according to the gradient of the surrounding road surface 3. The first reference drawing pattern is, for example, a desired pattern that the road surface drawing device 101 wants to draw on the surrounding road surface 3. Alternatively, the first reference drawing pattern may be a simpler pattern.

[0023] When the light-projecting unit 22 projects the first reference light-projection pattern onto the surrounding road surface 3, a pattern is drawn on the surrounding road surface 3. The actual drawing pattern drawn on the surrounding road surface 3 by the first reference light-projection pattern is referred to as the actual drawing pattern. An image of the actual drawing pattern is captured by the front camera 21 and acquired by the image acquisition unit 11. The light-projection control unit 12 extracts the actual drawing pattern from the captured image information acquired from the image acquisition unit 11, and calculates the amount of correction by comparing the actual drawing pattern with the first reference drawing pattern.

[0024] The light-projection control unit 12 corrects the second reference light-projection pattern using the calculated correction amount, thereby creating a corrected light-projection pattern, and causes the light-projection unit 22 to project this corrected light-projection pattern. The second reference light-projection pattern is a light-projection pattern that enables the road surface drawing device 101 to draw the desired pattern on the reference road surface. Therefore, when the desired pattern that the road surface drawing device 101 wants to draw on the surrounding road surface 3 is used as the first reference drawing pattern, the second reference light-projection pattern is the same pattern as the first reference light-projection pattern. On the other hand, when a simpler pattern is used as the first reference drawing pattern, the second reference light-projection pattern is a pattern different from the first reference light-projection pattern.

[0025] The actual drawing pattern drawn on the surrounding road surface 3 by the first reference light projection pattern is also referred to as the first actual drawing pattern, and the actual drawing pattern drawn on the surrounding road surface 3 by the corrected light projection pattern is also referred to as the second actual drawing pattern. When the first reference light projection pattern and the second reference light projection pattern are the same pattern, the deviation between the second actual drawing pattern and the reference drawing pattern is smaller than the deviation between the first actual drawing pattern and the reference drawing pattern. In this way, by correcting the light projection pattern of the light-projecting unit 22, it is possible to draw the desired pattern regardless of the gradient of the surrounding road surface 3.

[0026] In the following description, the first reference drawing pattern is the desired pattern that the road surface drawing device 101 wishes to draw on the surrounding road surface 3, and the first reference light projection pattern and the second reference light projection pattern are the same pattern.

[0027] In FIG. 2 , the reference drawing pattern is a circular drawing pattern 8. However, the shape of the reference drawing pattern is not limited to a circle. FIG. 5 shows reference drawing patterns of various shapes and their corresponding actual drawing patterns. In FIG. 5 , the dashed lines indicate the reference drawing patterns, and the matte-hatched areas indicate the actual drawing patterns. The reference drawing patterns are patterns such as circles, straight arrows, or right-turn arrows. When the surrounding road surface 3 is a flat reference road surface, the actual drawing pattern matches the reference drawing pattern. However, when the surrounding road surface 3 has a downward slope, the actual drawing pattern is a pattern obtained by extending the reference drawing pattern in the traveling direction of the vehicle M. Furthermore, when the surrounding road surface 3 has an upward slope, the actual drawing pattern is a pattern obtained by shortening the reference drawing pattern in the traveling direction of the vehicle M.

[0028] 6 is a flowchart showing the operation of the road surface drawing device 101. The operation of the road surface drawing device 101 will be described below with reference to the flow in FIG.

[0029] First, in step S101, the light-projection control unit 12 creates a reference light-projection pattern. The light-projection control unit 12 creates the reference light-projection pattern based on a reference drawing pattern to be drawn on the surrounding road surface 3.

[0030] Next, in step S102, the light-projection control unit 12 outputs the reference light-projection pattern to the light-projection unit 22 and instructs the light-projection unit 22 to project the reference light-projection pattern onto the surrounding road surface 3. As a result, the light-projection unit 22 projects the reference light-projection pattern onto the surrounding road surface 3, and an actual drawing pattern is drawn. This actual drawing pattern is photographed by the front camera 21.

[0031] Thereafter, in step S103 , the image acquisition unit 11 acquires a photographed image of the actual drawing pattern from the front camera 21 .

[0032] Next, in step S104, the light projection control unit 12 compares the actual drawing pattern with the reference drawing pattern and determines whether or not correction of the reference light projection pattern is necessary. If correction is not necessary in step S104, the processing of the road surface drawing device 101 returns to step S103. If correction is necessary in step S104, the amount of correction of the reference light projection pattern is calculated in step S105.

[0033] After step S105, in step S106, the light-projection control unit 12 corrects the reference light-projection pattern using the correction amount calculated in step S105, and creates a corrected light-projection pattern.

[0034] After step S106, in step S107, the light-projection control unit 12 outputs a corrected light-projection pattern to the light-projection unit 22 and instructs the light-projection unit 22 to project the corrected light-projection pattern onto the surrounding road surface 3. This causes the light-projection unit 22 to project the corrected light-projection pattern onto the surrounding road surface 3. Then, the processing of the road surface drawing device 101 returns to step S103.

[0035] A specific example of the processing of steps S104-106 is shown below. In this specific example, the light-projection control unit 12 compares the area of ​​the reference drawing pattern with the area of ​​the actual drawing pattern, and creates a corrected light-projection pattern if the difference between the two areas is equal to or greater than a predetermined threshold. In the following description, the relative positions of the eyes of the driver 1 of the vehicle M, the front camera 21, and the light-projection unit 22 are fixed. Also, the reference drawing pattern, the image of the reference drawing pattern captured by the front camera 21, and the reference drawing pattern viewed by the driver 1 are assumed to be similar in shape.

[0036] 7 shows an arrow-shaped actual drawing pattern. The light projection control unit 12 calculates the area A of the actual drawing pattern and the area Ao of the reference drawing pattern, calculates the area ratio between the two, A / Ao=AK, and sets the reciprocal of the area ratio, 1 / AK, as the correction amount for the reference light projection pattern.

[0037] The light-projection control unit 12 also compares the area ratio AK of the actual drawing pattern and the reference drawing pattern with a predetermined threshold AKth, and determines that correction is not necessary if AK<AKth, and that correction is necessary if AK≧AKth. In other words, the light-projection control unit 12 creates a corrected light-projection pattern when the difference in area between the actual drawing pattern and the reference drawing pattern is equal to or greater than a predetermined threshold. If correction is necessary, the light-projection control unit 12 creates a corrected light-projection pattern by enlarging or reducing the area of ​​the reference light-projection pattern by 1 / AK.

[0038] In FIG. 7 , the area of ​​the entire region of the actual drawing pattern is compared with the area of ​​the entire region of the reference drawing pattern. However, the light projection control unit 12 may divide the actual drawing pattern into a plurality of regions as shown by the dashed lines in FIG. 8 and compare the areas of each divided region. In FIG. 8 , the actual drawing pattern is divided into six regions. The areas of each divided region of the actual drawing pattern are designated A1 to A6. The light projection control unit 12 divides the reference drawing pattern into six regions corresponding to each divided region of the actual drawing pattern. The areas of each divided region of the reference drawing pattern are designated Ao1 to Ao6. The light projection control unit 12 calculates the area ratio Ai / Aoi=AKi (i is a natural number from 1 to 6) between the actual drawing pattern and the reference drawing pattern for each divided region, and determines the reciprocal 1 / AKi as the correction amount for the reference light projection pattern.

[0039] Next, the light-projection control unit 12 compares the area ratio AKi of the actual drawing pattern and the reference drawing pattern for each divided region with a predetermined threshold AKit, and determines that correction is not necessary if AKi<AKit for all divided regions, but determines that correction is necessary if there is even one divided region where Aki≧Akith. If correction is necessary, the light-projection control unit 12 creates a corrected light-projection pattern by enlarging or reducing, by 1 / Aki, the area of ​​the region of the reference light-projection pattern that corresponds to the divided region of the reference drawing pattern where Aki≧Akith.

[0040] The road surface drawing device 101 according to the first embodiment includes an image acquisition unit 11 and a light projection control unit 12. The image acquisition unit 11 acquires captured image information, which is information on a captured image of a road surface 3 around the vehicle M captured by at least one camera mounted on the vehicle M. The light projection control unit 12 controls the projection of light onto the surrounding road surface 3 by at least one light projection unit 22 mounted on the vehicle M. More specifically, the light-projection control unit 12 causes at least one light-projection unit 22 to project light of a first reference light-projection pattern that draws a first reference drawing pattern on the reference road surface, extracts from the captured image information a first actual drawing pattern that is drawn on the surrounding road surface 3 by the light of the first reference light-projection pattern, calculates a correction amount based on the difference between the second actual drawing pattern and the first reference drawing pattern, draws a second reference drawing pattern that is the same as or different from the first reference drawing pattern on the reference road surface, corrects the second reference light-projection pattern that is the same as or different from the first reference light-projection pattern with the correction amount to create a corrected light-projection pattern, and causes at least one light-projection unit 22 to project light of the corrected light-projection pattern, thereby drawing the second actual drawing pattern on the surrounding road surface 3. In this way, the desired pattern that is to be drawn on the surrounding road surface 3 is drawn regardless of the gradient of the surrounding road surface 3.

[0041] <B. Second Embodiment> As shown in Fig. 4, a road surface drawing device 102 according to a second embodiment has the same configuration as the road surface drawing device 101 according to the first embodiment. In the second embodiment, the specific processing of steps S104-106 in Fig. 6 differs from that of the first embodiment as follows.

[0042] In the first embodiment, the light projection control unit 12 calculates the correction amount by comparing the areas of the reference drawing pattern and the actual drawing pattern. On the other hand, in the second embodiment, the light projection control unit 12 calculates the correction amount by comparing the distances between the feature points of the reference drawing pattern and the actual drawing pattern.

[0043] 9 shows characteristic points a, b, d, e, f, and g of an arrow-shaped actual drawing pattern. The light projection control unit 12 can extract these characteristic points a, b, d, e, f, and g by blinking the image of the actual drawing pattern, pattern matching, or using a detection algorithm such as SURF (Speed-Up Robust Features) or SIFT (Scale-Invariant Feature Transform).

[0044] 10, the light-projection control unit 12 calculates the distance L1 between the feature points b and e, the distance L2 between the feature points a and g or a and f, and the distance L3 between the feature points f and g. Similarly, the light-projection control unit 12 extracts the feature points ao, bo, do, eo, fo, and go of the reference drawing pattern that correspond to the feature points a, b, d, e, f, and g of the actual drawing pattern (not shown), and calculates the distance Lo1 between the feature points ao and eo, the distance Lo2 between the feature points ao and go or ao and fo, and the distance Lo3 between the feature points fo and go.

[0045] In step S104, the light projection control unit 12 compares the ratio LKi of the distances between the characteristic points with a predetermined threshold LKth, and determines that correction is not necessary if LKi<LKth, and that correction is necessary if LKi≧LKth.

[0046] If correction is required in step S104, in step S105 the light projection control unit 12 calculates the ratio Li / Loi = LKi (i = 1, 2, 3) of the distances between the feature points of the actual drawing pattern and the reference drawing pattern, and uses the reciprocal 1 / LKi as the correction amount for the reference light projection pattern.

[0047] After step S105, in step S106, the light projection control unit 12 enlarges or reduces the reference light projection pattern by multiplying the distance between the feature points of the reference light projection pattern corresponding to the feature points of the reference drawing pattern by 1 / LKi, thereby creating a corrected light projection pattern.

[0048] The distance comparison between the feature points of the actual drawing pattern and the reference drawing pattern may be performed for each divided region, as in the area comparison. In the example of Fig. 11, the actual drawing pattern is divided into four regions, left and right and top and bottom, and the distances between the feature points in each divided region are calculated as L11, L12, L21, L22, L23, L24, L31, and L32.

[0049] In the road surface drawing device 102 according to the second embodiment, the captured image information includes information on the distances between feature points in the captured image. The light-projection control unit 12 creates a corrected light-projection pattern based on a comparison between the distances between the feature points of the first reference drawing pattern and the distances between the feature points of the actual drawing pattern corresponding to the feature points of the first reference drawing pattern, which are acquired from the captured image information.

[0050] <C. Third Embodiment> As shown in Fig. 4, a road surface drawing device 103 according to a third embodiment has the same configuration as the road surface drawing devices 101 and 102 according to the first and second embodiments. In the third embodiment, the specific processing of steps S104-106 in Fig. 6 differs from that of the first and second embodiments as follows.

[0051] 3, when the front camera 21 is configured with two front cameras 21B, i.e., a stereo camera, the light-projection control unit 12 can measure the distance between each feature point of the actual drawing pattern and the front camera 21B from images of the actual drawing pattern captured by the two front cameras 21B. Therefore, the light-projection control unit 12 can correct the reference light-projection pattern so that the distance between each feature point of the actual drawing pattern and the front camera 21B matches the distance between each feature point of the reference drawing pattern and the front camera 21B.

[0052] 3 shows distances D1 and D2 between each feature point of the drawing pattern 9 (actual drawing pattern) and the front camera 21B. Similarly, the light projection control unit 12 calculates distances Do1 and Do2 between each feature point of the reference drawing pattern and the front camera 21B.

[0053] In step S104, the light projection control unit 12 compares the distance ratio DKi with a predetermined threshold value Dth, and determines that correction is not necessary if DKi<DKth, and determines that correction is necessary if DKi≧DKth.

[0054] If correction is required in step S104, in step S105 the light projection control unit 12 calculates the ratio Di / Doi = DKi (i = 1, 2) between the distance from each feature point in the actual drawing pattern to the front camera 21B and the distance from each feature point in the reference drawing pattern to the front camera 21B, and uses the reciprocal 1 / DKi as the correction amount for the reference light projection pattern.

[0055] After step S105, in step S106, the light-projection control unit 12 corrects the reference light-projection pattern to obtain a corrected light-projection pattern by multiplying the distance between each feature point of the reference drawing pattern and the front camera 21B by 1 / DKi. This method allows the pattern to be drawn at a constant distance from the vehicle M even if the gradient of the surrounding road surface 3 changes.

[0056] D. Fourth Embodiment A road surface drawing device 104 according to a fourth embodiment has the same configuration as that according to the first to third embodiments, as shown in FIG. 4. In the fourth embodiment, the specific processing of steps S104-106 in FIG. 6 differs from that according to the first to third embodiments. The road surface drawing device 104 creates a corrected light projection pattern by correcting the reference light projection pattern so that the azimuth angle and depression angle at which each feature point of the second actual drawing pattern is viewed from a specific viewpoint coincide with the azimuth angle and depression angle at which each feature point of the first reference drawing pattern corresponding to each feature point of the second actual drawing pattern is viewed from the specific viewpoint.

[0057] FIG. 12 shows a state in which a reference light projection pattern is projected from light source O toward a flat road surface S0. The x, y, and z coordinates have their origin on the body of vehicle M. The x-axis direction is the forward direction of the vehicle body, the y-axis direction is the rightward direction, and the z-axis is the height direction. A perpendicular line drawn from light source O to the frame surface f of light projectors 22L and 22R or front camera 21 intersects with the frame surface at the center of f. The position within the frame is indicated by the angle with respect to the origin or the distance in the coordinate axis direction. These are called frame coordinates. Once the projection surface is determined, the frame coordinates can be converted into coordinates of the projection position using a formula. When an arrow-shaped drawing pattern having an outer frame ABCD is drawn on the frame coordinates of light projectors 22L and 22R, a coordinate-converted figure is drawn on road surface S0. The rectangle on the frame coordinates is projected as a trapezoid onto road surface S0. Road surface S0 is the reference road surface, and the pattern drawn on road surface S0 is the reference drawing pattern. The projection point constituting the reference drawing pattern on the road surface S0 is defined as P0 (x0, y0, z0).

[0058] Figure 13 shows a state in which the same reference light projection pattern as in Figure 12 is projected from a light source O toward a road surface S1. Because the road surface S1 is not flat, the actual drawing pattern drawn on the road surface S1 differs from the reference drawing pattern in Figure 12 and is distorted. The outer frame ABCD in the reference drawing pattern in Figure 12 is transformed into an outer frame A'B'C'D' in the actual drawing pattern in Figure 13. The arrows within the frame are also transformed in the same way as the outer frame. The projection points constituting the actual drawing pattern on the road surface S1 are designated as P1 (x1, y1, z1).

[0059] Now, consider correcting the distortion of the actual drawing pattern. However, because road surface S0 and road surface S1 are located at different heights (in the z-axis direction in FIGS. 12 and 13 ), even if the reference light projection pattern is corrected, the actual drawing pattern cannot be drawn at the same height as road surface S0. The magnitude and direction of the deviation between feature points on the reference drawing pattern and the corresponding feature points on the actual drawing pattern will differ depending on the viewpoint from which the actual drawing pattern is viewed.

[0060] Therefore, the road surface drawing device 104 creates a corrected light-projection pattern by correcting the reference light-projection pattern that is projected onto the road surface S1 so that the direction (azimuth angle and depression angle) in which each feature point of the actual drawing pattern on the road surface S1 is viewed from an arbitrary viewpoint V matches the direction (azimuth angle and depression angle) in which each feature point of the reference drawing pattern on the road surface S0 is viewed from the same viewpoint V. In this way, the road surface drawing device 104 corrects for differences in the appearance of the drawing pattern depending on the road surface.

[0061] 14 is a flowchart of the light projection pattern correction process performed by the road surface drawing device 104. The light projection pattern correction process performed by the road surface drawing device 104 will be described below with reference to the flow of FIG.

[0062] First, in step S201, the light projection control unit 12 acquires a projection image to be drawn on the road surface S1. The projection image may be a simple two-dimensional figure, a three-dimensional figure, or a plurality of figures that change over time.

[0063] Next, in step S202, the light projection control unit 12 acquires the position of viewpoint V (hereinafter referred to as viewpoint position) (Xv, Yv, Zv). The viewpoint position may be a predetermined value, or may be a value acquired using a three-dimensional distance sensor such as a Lidar (Light Detection and Ranging) or a camera.

[0064] Then, in step S203, the light projection control unit 12 creates a reference drawing pattern by correcting the projection image based on the viewpoint position. When the reference drawing pattern drawn on the reference road surface is viewed from the viewpoint position, it appears as a projection image. The reference drawing pattern is determined based on the projection image, the position of the light projection unit 22, and the viewpoint position.

[0065] Next, in step S204, the light-projection control unit 12 creates a reference light-projection pattern based on the reference drawing pattern. The reference light-projection pattern is a pattern set in the light-projection unit 22, and is created by back-calculating from the reference drawing pattern. This step corresponds to step S101 in FIG. 6.

[0066] Thereafter, in step S205, the light projection control unit 12 extracts characteristic points of the reference light projection pattern.

[0067] The positions of the feature points of the reference drawing pattern corresponding to the feature points of the reference light projection pattern change depending on the depth of the projection surface, i.e., the road surface S1. Therefore, in step S206 after step S205, the light projection control unit 12 calculates the trajectory of the feature points of the reference drawing pattern when the depth of the projection surface (z-axis position in FIG. 13) is changed. The trajectory of the feature points may be represented by a table or a formula.

[0068] In parallel with steps S205 and S206, in step S207, the light-projection control unit 12 controls the projection of the reference light-projection pattern. This step corresponds to step S102 in Fig. 6. As a result, the reference light-projection pattern is projected from the light-projection unit 22 onto the road surface S1, and the reference drawing pattern is drawn on the road surface S1.

[0069] After step S207, the front camera 21 captures an image of the actual drawing pattern drawn on the road surface S1. Then, in step S208, the image acquisition unit 11 acquires captured image information including the actual drawing pattern from the front camera 21. This step corresponds to step S103 in FIG. 6 .

[0070] After step S208, in step S209, the light projection control unit 12 extracts feature points of the actual drawing pattern by blinking the image of the actual drawing pattern, pattern matching, or a detection algorithm such as SURF (Speed-Up Robust Features) or SIFT (Scale-Invariant Feature Transform).

[0071] After steps S206 and S209, in step S210, the light-projection control unit 12 compares the reference drawing pattern with the actual drawing pattern, and calculates or modifies the correction amount of the reference light-projection pattern by changing the positions of the feature points of the actual drawing pattern on the trajectory calculated in step S206. Specifically, the light-projection control unit 12 moves the feature points of the actual drawing pattern on the trajectory calculated in step S206 so that the azimuth angle and depression angle of the actual drawing pattern as seen from viewpoint V approach the azimuth angle and depression angle of the reference drawing pattern as seen from viewpoint V. The correction amount of the reference light-projection pattern is determined based on the movement direction and movement distance of the feature points of the actual drawing pattern.

[0072] Next, in step S211, the light-projection control unit 12 creates a corrected light-projection pattern by correcting the reference light-projection pattern based on the correction amount calculated in step S210.

[0073] Next, in step S212, the light projection control unit 12 determines whether there has been a change in the projected image. If there has been a change in the projected image, the processing of the road surface drawing device 104 returns to step S201. If there has been no change in the projected image, in step S213, the light projection control unit 12 determines whether there has been a change in the viewpoint V.

[0074] If there is a change in viewpoint V, the processing of the road surface drawing device 104 returns to step S202. If there is no change in viewpoint V, the processing of the road surface drawing device 104 returns to step S207. In step S207, the light-projection control unit 12 performs light-projection control using the corrected light-projection pattern created in step S211 as a new reference light-projection pattern.

[0075] That is, the light projection control unit 12 repeats the correction amount calculation process in step S210 while there are no changes to either the projected image or the viewpoint V. The second and subsequent calculations of the correction amount are performed based on the previous correction amount, the positions of the feature points of the actual drawing pattern moved in the previous correction amount calculation process, and the positions of the feature points of the current actual drawing pattern. As a result, the azimuth angle and depression angle of the actual drawing pattern as viewed from the viewpoint V gradually approach the azimuth angle and depression angle of the reference drawing pattern as viewed from the viewpoint V.

[0076] That is, in this example, the first reference light-projection pattern and the second reference light-projection pattern are the same. Then, the light-projection control unit 12 creates a new corrected light-projection pattern by using the most recent corrected light-projection pattern as a new second reference light-projection pattern and the surrounding road surface at the time of creating the most recent corrected light-projection pattern as a new reference road surface.

[0077] The light-projection control unit 12 controls the azimuth angle and depression angle of the reference light-projection pattern projected from the light-projection unit 22 so that the azimuth angle and depression angle of the projection point P1 onto the distorted road surface S1 as seen from an arbitrary viewpoint V coincide with the azimuth angle and depression angle of the projection point P0 onto the flat road surface S0. The specific procedure is as follows.

[0078] Step 1: The light projecting unit 22 projects the reference light projection pattern at an azimuth angle θ 0 , depression angle φ 0 Project in the direction of.

[0079] Step 2: Calculate the amount of correction for the projection azimuth angle of the light-projecting unit 22 in the xy view. Figure 15 shows the projection state in the xy view. P1 is set as the feature point of the actual drawing pattern. The amount of correction for the projection azimuth angle is calculated so that P1 moves to P1' on the extension of the line V-P0.

[0080] Step 3: Calculate the correction amount for the projection depression angle of the light projector 22 in the xz direction. Figure 16 shows the projection state in the xy direction. The correction amount for the projection depression angle is calculated so that P1 moves to P1', which is an extension of the line V-P0.

[0081] Step 4: The light projection azimuth angle and light projection depression angle of the light projector 22 are corrected using the correction amount calculated in step 3.

[0082] Step 5: Repeat steps 2, 3, and 4.

[0083] While typical projection mapping uses a fixed surface, such as a building wall, as the projection surface, in vehicle applications the shape of the projection surface changes as the vehicle travels. Therefore, if steps 2 and 3 are performed only once and correction is stopped, the road surface shape will change as the vehicle travels, causing the image projected onto the road surface to fluctuate. However, by repeating steps 2, 3, and 4 in step 5, corrections can be made continuously, reducing the fluctuation of the image.

[0084] 15 and 16, specific examples of steps 2 and 3 will be described below. First, the parameters used in the specific examples of steps 2 and 3 will be described below.

[0085] O(0,0,0), C(Xc,Yc,Zc), and V(Xv,Yv,Zv) are fixed points, which respectively represent the positions of the light projector 22, the front camera 21, and the viewpoint. The direction vector of O, i.e., the light projection direction vector of the light projector 22, is defined as ω 0 (θ 0 , φ 0 The direction vector of C, that is, the direction vector as seen from the front camera 21, is defined as ω c (θ c , φ c ) The direction vector seen from V is ω V (θ V , φ V )

[0086] S0 is a flat road surface that serves as the reference road surface. S0 is parallel to the xy plane, and the z-axis coordinate of S0 is -h. S1 is a non-flat surrounding road surface, which is the road surface onto which the light-projecting unit 22 actually projects light. The shape of S1 is unknown.

[0087] P represents the projection point of the light-projecting unit 22. P0 (x0, y0, z0) represents the projection point onto the road surface S0 by the reference light-projection pattern. P1 (x1, y1, z1) represents the projection point onto the road surface S1 by the reference light-projection pattern. P1' (x1', y1', z1') represents the projection point onto the road surface S1 by the corrected light-projection pattern. P1' is determined so that the direction in which P1' is viewed from viewpoint V matches the direction in which P0 is viewed from viewpoint V. In other words, P1' is on an extension of the straight line V-P0.

[0088] θ is the azimuth angle, which is the angle between the straight line and the positive y-axis direction in the xy view. O is the projection azimuth angle of the light projector 22. c is the azimuth angle at which the projection point is viewed from the front camera 21. v is the azimuth angle from viewpoint V to the projection point. o,0 is the projection azimuth angle from the light projector 22 to P0. o,1´ is the projection azimuth angle from the light projector 22 to P1'. c,1is the azimuth angle from the front camera 21 to P1. V,0 is the azimuth angle when viewing P0 from viewpoint V. V,1´ is the azimuth angle when viewing P1' from viewpoint V.

[0089] φ is the depression angle, which is the angle with the xy plane. x-z is the depression angle in the xz view. x-z In FIG. 16, the counterclockwise direction is +. O,x-z,0 is the depression angle of the light projected from O to P0 in the xz view. O,x-z,1´ is the depression angle of the light projected from O to P1' in the xz view. C,x-z,1 is the depression angle when looking at P1 from C in the xz view. V,x-z,0 is the depression angle when looking at P0 from V in the xz view.

[0090] The following are calculated values: X1_Azimuth is θ O,1´ Z1_Elevation is the x-direction position of P1' used in the calculation of φ O,1´ is the z-direction position of P1' used in the calculation of

[0091] Next, a specific example of step 2 will be described with reference to FIG. 15. The light projection control unit 12 operates the light projection azimuth angle of the projection point P1 in the xy view to place it on the extension line of V-P0. In the xy view, the position in the z direction can be ignored. The azimuth angle θ when an arbitrary projection point P(x, y, z) is viewed from V, C, O is V , θ C , θ O is expressed by the following formula:

[0092]

[0093] First, the light projecting unit 22 is O,1 (θ O,1 , φ O,1 ) direction and projects a point P1. The point P1 is located at a distance of ω C,1 (θ C,1 , φ C,1 ) direction. O,1 , θ C,1 is expressed by the following equation from FIG.

[0094]

[0095] By transforming equations (4) and (5), the following equations are obtained:

[0096]

[0097] From equations (6) and (7), the x-coordinate x1 of the projection point P1 is expressed by the following equation.

[0098]

[0099] Projection azimuth angle θ O Even if θ is changed, the x-coordinate of the projection point remains unchanged as x1 = X1_Azimuth. O The y coordinate of the projection point when is changed is expressed by the following equation.

[0100]

[0101] From equations (1), (8), and (9), the following equation is obtained.

[0102]

[0103] The position of the corrected projection point P1' is determined so that the azimuth angle at which the corrected projection point P1' is viewed from the viewpoint V is the same as the azimuth angle at which the projection point P0 on the road surface S0 is viewed from the viewpoint V. Therefore, the y-coordinate y1' of the corrected projection point P1' is expressed as follows:

[0104]

[0105] By substituting equation (8) into this, y1' is expressed as follows:

[0106]

[0107] The projection azimuth angle θ from the position O of the light projecting unit 22 to the corrected projection point P1′ O,1´ is expressed by the following formula:

[0108]

[0109] According to the equations (8), (12), and (13), the projection azimuth angle θ to the projection point P1′ after correction is O,1´is determined. P1' is located on a plane that satisfies x = X1_Azimuth and on the extension of V-P0. However, since the road surface S1 is unknown, the actually projected point P1' often approaches the extension of V-P0, but is not necessarily located exactly on the extension of V-P0.

[0110] Next, a specific example of step 3 will be described with reference to Figure 16. The light projection control unit 12 operates the projection depression angle of the projection point P1 in the xz view to place it on the extension line of V-P0. The depression angle φ in the xz view when an arbitrary point P(x, y, z) is viewed from the viewpoint V, the position C of the front camera 21, and the position O of the light projection unit 22 is V,x-z , φ C,x-z , φ O,x-z is expressed by the following formula:

[0111]

[0112] Consider the projection point P1' after the first correction. First, the light projecting unit 22 moves from O to ω O,1 (θ O,1 , φ O,1 ) direction and projects a point P1. The point P1 is located at a distance of ω C,1 (θ C,1 , φ C,1 ) direction. The depression angle φ when looking at point P1 from O and C in the x-z view O,x-z,1 , φ C,x-z,1 is expressed by the following equation from FIG.

[0113]

[0114] By modifying equations (17) and (18), the following equations are obtained:

[0115]

[0116] From equations (19) and (20), the x-coordinate x1 of the projection point P1 is expressed by the following equation.

[0117]

[0118] By modifying equation (20), the z coordinate z1 of the projection point P1 is expressed by the following equation.

[0119]

[0120] Light projection depression angle φO,x-z Even if φ is changed, the z coordinate of the projection point remains unchanged as z1 = Z1_Elevation. O,x-z The x-coordinate of the projection point when is changed is expressed by the following equation.

[0121]

[0122] Depression angle φ from viewpoint V to P0 v,x-z,0 and the depression angle φ from the viewpoint V to the corrected projection point P1′. v,x-z,1´ are expressed by the following formulas, respectively.

[0123]

[0124] φ v,x-z,0 and φ v,x-z,1´ Make sure that is the same.

[0125] Since Z1'=Z1_Elevation, equation (25) is transformed into equation (26).

[0126]

[0127] The position of the corrected projection point P1' is determined so that the depression angle when viewing the corrected projection point P1' from viewpoint V is the same as the depression angle when viewing the projection point P0 on the road surface S0 from viewpoint V. Therefore, from equations (24) and (26), the x-coordinate x1' of the corrected projection point P1' is expressed by the following equation:

[0128]

[0129] The depression angle φ on the xz plane from the position O of the light projecting unit 22 to the corrected projection point P1′ O,x-z,1´ is expressed by the following formula:

[0130]

[0131] The light projection azimuth angle θ from the position O of the light projecting unit 22 O,1´ is as shown in equation (13). Therefore, the depression angle φ from the position O of the light projecting unit 22 to the corrected projection point P1′ is O,1´ is calculated by using equation (28) as the projection azimuth angle θ O,1´ and is expressed by the following formula:

[0132]

[0133] According to the equations (8), (12), and (13), the projection azimuth angle θ to the projection point P1′ after correction is O,1´ The projection depression angle φ at the projection point P1′ after correction is determined by the equations (12), (13), (27), and (29). O,1´ The light projection control unit 12 determines the light projection azimuth angle θ O,1´ and projection depression angle φ O,1´ As a result, when P1' is projected onto a plane that satisfies z=Z1_Elevation, P1' is located on an extension of V-P0.

[0134] However, since S1 is an unknown road surface, the projection point P1' does not necessarily satisfy z = Z1_Elevation. Therefore, in many cases, P1' approaches the extension of V-P0, but is not necessarily located exactly on the extension. Even so, by repeating the correction process using the obtained P1' as the new P1, P1' will come close enough to the extension of V-P0.

[0135] In the road surface drawing device 104 of embodiment 4, the light projection control unit 12 creates a corrected light projection pattern so that the azimuth angle and depression angle when viewing each feature point of the second actual drawing pattern from a specific viewpoint match the azimuth angle and depression angle when viewing each feature point of the first reference drawing pattern corresponding to each feature point of the second actual drawing pattern from a specific viewpoint.

[0136] E. Fifth Embodiment A road surface drawing device 105 according to a fifth embodiment has the same configuration as those according to the first to fourth embodiments, as shown in Fig. 4. In this embodiment, the second reference drawing pattern is a pattern that is viewed stereoscopically from a specific viewpoint.

[0137] 17 to 20 show examples of creating a second reference drawing pattern that is stereoscopically viewed from a specific viewpoint, ie, viewpoint V1.

[0138] FIG. 17 is a stereoscopic diagram showing a rectangular parallelepiped placed on the xy plane at z=0 in the xyz coordinate system and viewed stereoscopically from viewpoint V1. The light projection control unit 12 projects each of the vertices a, b, c, d, e, f, g, and h of the rectangular parallelepiped onto the xy plane at z=0, starting from viewpoint V1, to obtain projection points a1, b1, c1, d1, e1, f1, g1, and h1. Projection point a1 is the intersection point where an extension of the straight line connecting viewpoint V1 and vertex a intersects with the xy plane at z=0. Other projection points are determined in a similar manner. Note that FIG. 17 only shows projection points b1, c1, and d1, which are visible and not hidden by the rectangular parallelepiped.

[0139] FIG. 18 is a view of FIG. 17 as viewed from the y-axis direction.

[0140] FIG. 19 is a view of FIG. 17 as seen from the z-axis direction.

[0141] 20 shows a figure drawn on the xy plane (z=0) with projection points a1, b1, c1, d1, e1, and f1 as feature points. This figure is a reference drawing pattern that appears as a rectangular parallelepiped from viewpoint V1 when drawn on the reference road surface.

[0142] 21 to 24 show examples of creating a second reference drawing pattern that is stereoscopically viewed from a specific viewpoint V2. As the specific viewpoint changes from V1 to V2, the position of the projection point onto the xy plane at z=0 changes.

[0143] The light projection control unit 12 projects each of the vertices a, b, c, d, e, f, g, and h of the rectangular parallelepiped onto the xy plane where z = 0, starting from viewpoint V1, to obtain projection points a2, b2, c2, d2, e2, f2, g2, and h2. Note that only projection points a2, c2, and d2 that are not hidden by the rectangular parallelepiped and can be seen with the naked eye are shown in Fig. 21.

[0144] FIG. 22 is a view of FIG. 21 as seen from the y-axis direction.

[0145] FIG. 23 is a view of FIG. 21 as seen from the z-axis direction.

[0146] 24 shows a figure drawn on the xy plane (z = 0) with projection points a2, b2, c2, d2, e2, f2, g2, and h2 as feature points. This figure is a second reference drawing pattern that appears as a rectangular parallelepiped from viewpoint V2 when drawn on the reference road surface.

[0147] That is, when the reference drawing pattern is transformed from the pattern shown in Fig. 20 to the pattern shown in Fig. 24, the same rectangular parallelepiped is visually recognized even when the viewpoint moves from V1 to V2. The light projection control unit 12 uses the pattern shown in Fig. 20 or 24 as the reference drawing pattern for stereoscopic vision depending on the viewpoint.

[0148] Although a rectangular parallelepiped has been used in the above description, the three-dimensional object to be viewed stereoscopically does not have to be a rectangular parallelepiped. Also, the three-dimensional object may have a pattern or letters drawn on its surface.

[0149] <F. Sixth Embodiment> A road surface drawing device 106 according to a sixth embodiment has the same configuration as those according to the first to fifth embodiments, as shown in FIG.

[0150] In embodiment 1-5, the first reference light projection pattern, which is the reference light projection pattern for calculating the correction amount of the light projection pattern according to the surrounding road surface 3, and the second reference light projection pattern, which is the reference light projection pattern to which the correction amount is applied, are described as being the same.

[0151] However, in the sixth embodiment, the first reference light-projection pattern and the second reference light-projection pattern are different patterns.

[0152] Specifically, the second reference drawing pattern is a desired pattern that the road surface drawing device 106 wants to draw on the surrounding road surface 3. The second reference light projection pattern is a detailed pattern that is set by working backward from the desired second reference drawing pattern.

[0153] In contrast, the first reference light projection pattern and the first reference drawing pattern are simple patterns with a small number of feature points, such as circles, rectangles, triangles, etc. By using a simple first reference light projection pattern, calculation processes such as area comparison, distance comparison between feature points, and viewpoint extraction between the first actual drawing pattern and the first reference drawing pattern are simplified.

[0154] Furthermore, since the first reference light projection pattern is used solely for calculating the correction amount, it does not need to be visible to the driver 1, etc. Therefore, the first reference light projection pattern may be an invisible light pattern such as infrared light or ultraviolet light. In other words, the light of the first reference light projection pattern may be invisible light, and the light of the second reference light projection pattern and the correction light projection pattern may be visible light.

[0155] <G. Seventh Embodiment> Fig. 25 is a block diagram showing the configuration of a road surface drawing device 107 and its peripheral devices according to a seventh embodiment. The road surface drawing device 107 is configured by adding a warning control unit 13 to the configuration of the road surface drawing device 101 according to the first embodiment. In this embodiment, the vehicle M is equipped with an output device 23 in addition to a front camera 21 and a light projecting unit 22.

[0156] The output device 23 is an LED light, a display, or a speaker mounted on the vehicle M.

[0157] Fig. 26 shows an actual drawing pattern with defects. An actual drawing pattern with defects is an actual drawing pattern that is missing feature points compared to the reference projection pattern. The actual drawing pattern in Fig. 26 is an arrow-shaped actual drawing pattern that is originally composed of feature points a, b, c, d, e, f, and g as shown in Fig. 9, but is missing feature points a, d, and e.

[0158] For example, if snow or mud adheres to the front of the headlight, which is the light projecting unit 22, a defect such as that shown in FIG. 26 will occur in the actual drawing pattern. Alternatively, if snow or mud adheres to the front of the front camera 21, no defect will occur in the actual drawing pattern drawn on the surrounding road surface 3, but a defect such as that shown in FIG. 26 will occur in the actual drawing pattern acquired by the image acquiring unit 11. If there is a defect in the actual drawing pattern acquired by the image acquiring unit 11, the area determination method of embodiment 1 will produce an area where the area cannot be calculated. Furthermore, the distance determination method of embodiment 2 will produce an area where the distance cannot be extracted. Furthermore, the method of embodiment 4 will also produce abnormalities such as a lack of feature points.

[0159] 27 is a flowchart showing the operation of the road surface drawing device 107. The flow in Fig. 27 is obtained by adding step S103A and step S108 to the flow of the road surface drawing device 101 shown in Fig. 6.

[0160] When the image acquisition unit 11 acquires the actual drawing pattern in step S103, the light projection control unit 12 compares the actual drawing pattern with the reference drawing pattern in step S103A and determines whether the number of feature points in the actual drawing pattern is insufficient.

[0161] If the number of feature points in the actual drawing pattern is insufficient in step S103A, the alarm control unit 13 determines that there is a defect in the actual drawing pattern, and performs alarm control in step S108. In this step, the alarm control unit 13 causes the output device 23 to output a lighting abnormality alarm. Upon receiving this alarm, the driver 1 can stop the vehicle M, visually inspect the light-projecting unit 22, and clean it if any snow, mud, or other deposits are found on the light-projecting unit 22. Furthermore, if there is no deposit on the light-projecting unit 22, the driver 1 can determine that there is an internal malfunction of the light-projecting unit 22 and take appropriate measures.

[0162] At the same time as outputting the light abnormality alarm, the alarm control unit 13 may operate a wiper (not shown) attached to the front surface of the light projecting unit 22 to clean the front surface of the light projecting unit 22 .

[0163] The road surface drawing device 107 of the seventh embodiment includes an alarm control unit 13 that causes an output device mounted on the vehicle to output an alarm when the difference between the number of feature points in the first actual drawing pattern and the number of feature points in the first reference drawing pattern is equal to or greater than a predetermined threshold value. This allows the driver 1 of the vehicle M to notice a malfunction in the light-projecting unit 22 or the front camera 21 and take necessary measures.

[0164] <H. Eighth Embodiment> A road surface drawing device 108 according to an eighth embodiment has the same configuration as those according to the first to sixth embodiments, as shown in FIG.

[0165] When the surrounding road surface 3 on which the actual drawing pattern is drawn is inclined downward, the specular reflection light component of the light projected by the light projecting unit 22 onto the surrounding road surface 3 becomes larger and the reflection light component toward the vehicle M becomes smaller, compared to when the road surface is horizontal. Therefore, the actual drawing pattern appears dark to the driver 1 or a pedestrian who is not in the direction of travel of the vehicle M, and the actual drawing pattern appears excessively bright to a pedestrian 7 who is in the direction of travel of the vehicle M.

[0166] Therefore, when the person viewing the actual drawing pattern is the driver 1 or a pedestrian who is not in the traveling direction of the vehicle M, the light-projection control unit 12 increases the brightness of the corrected light-projection pattern. On the other hand, when the person viewing the drawing pattern is a pedestrian 7 who is in the traveling direction of the vehicle M, the light-projection control unit 12 decreases the brightness of the corrected light-projection pattern.

[0167] The light-projection control unit 12 can calculate the gradient of the surrounding road surface 3 by comparing the actual drawing pattern with the reference drawing pattern. Furthermore, when the actual drawing pattern is divided into multiple regions and compared with the reference drawing pattern as shown in Figure 8 or Figure 11, if the correction amount of the divided region ahead in the traveling direction of the vehicle M is greater than the correction amount of the published divided region, the gradient of the surrounding road surface 3 changes midway through the projection range of the light-projection unit 22. Therefore, the light-projection control unit 12 increases the brightness of the divided region ahead in the traveling direction of the corrected light-projection pattern compared to the brightness of the divided region behind it.

[0168] In the road surface drawing device 108 of embodiment 8, the light projection control unit 12 increases the brightness of the light of the corrected light projection pattern when the downward gradient value of the surrounding road surface 3 exceeds a predetermined threshold value compared to when it does not exceed the threshold value.

[0169] <I. Ninth Embodiment> A road surface drawing device 109 according to a ninth embodiment has the same configuration as those according to the first, sixth, and eighth embodiments, as shown in FIG.

[0170] 12, 15, and 16 have their origins at the body of vehicle M, more specifically, the light sources of light-projecting units 22L and 22R. Other coordinate systems include an xc-yc-zc coordinate system recognized by the camera image of front camera 21, which has its origin at front camera 21; an xl-yl-zl coordinate system of light-projecting unit 22L; and an xr-yr-zr coordinate system of light-projecting unit 22R. The xl-yl-zl and xr-yr-zr coordinate systems are determined by the projection angles of light-projecting units 22L and 22R. Because each coordinate system is independent, even if coordinate values ​​indicating a specific position in one coordinate system are input into another coordinate system, if the position or rotation of the origin of each coordinate system is unknown, the input cannot be projected to the same specific position in the other coordinate system.

[0171] Fig. 28 shows light projection points P1L and P1R, which are alignment patterns projected in the same direction from both light projection unit 22L and light projection unit 22R so that they project light onto the same point. Fig. 28 shows the xc-yc positions of light projection points P1L and P1R in an image captured by front camera 21. Light projection point P1L by light projection unit 22L, represented by a + mark, does not overlap with light projection point P1R by light projection unit 22R, represented by an x ​​mark.

[0172] The light-projection control unit 12 of the road surface drawing device 109 uses the light-projection points P1L and P1R as an alignment pattern and corrects the xl-yl-zl coordinate system of the light-projection unit 22L and the xr-yr-zr coordinate system of the light-projection unit 22R based on the misalignment between them. The correction is performed by determining the position of the origin of each coordinate system and rotating it around each coordinate axis. In this case, the coordinates have six adjustment parameters: the x-position, y-position, and z-position of the origin, as well as x-, y-, and z-rotation. It is not necessary to identify all adjustment parameters; only those likely to have large errors or that are easy to identify may be identified. The direction of the z-rotation is the same as θ, which is expressed as the azimuth angle of the light projection direction of the light-projection units 22L and 22R in the fourth embodiment. The direction of the y-rotation is the same as φ, which is expressed as the depression angle of the light projection direction of the light-projection units 22L and 22R in the fourth embodiment.

[0173] 12 and the y direction of the origin O from the deviation of P1L and P1R. In the xc-yc-zc coordinate system, the position coordinates of P1L and P1R in the x and y directions in the frame coordinate system can be determined from the image captured by the front camera 21. From these position coordinates, the direction angles and depression angles of P1L and P1R in the xc-yc-zc coordinate system can be calculated.

[0174] Both P1L and P1R are projected onto the S0 plane. The azimuth angle and zx visual depression angle of P1L as viewed from the front camera 21 are defined as θ C (1, r) and φ C,x-z The azimuth angle and zx viewing depression angle of P1R as viewed from the front camera 21 are defined as θ C (1, l) and φ C,x-z Let (1, l).

[0175] The xc coordinate and the yc coordinate are expressed by the following equations.

[0176]

[0177] Therefore, the xc coordinate xc(1,l) and yc coordinate yc(1,l) of P1L, and the xc coordinate xc(1,r) and xc coordinate yc(1,r) of P1R are expressed by the following equations: In many cases, P1R is located near P1L, so there is no major contradiction in assuming that P1L and P1R are on the same plane.

[0178]

[0179] First, the process of adjusting the deviation in the yc direction will be described. In this case, the deviation Δyc, (1) in the yc direction between P1L and P1R can be calculated by the following formula from formulas (33) and (35).

[0180]

[0181] Even if the rotation of the coordinate system in the y direction is changed, the projection position in the y direction does not change. Therefore, the y direction position of light-projecting unit 22R, which is the origin position of the coordinate system xr-yr-zr, is moved by -Δyc, (1). This causes the yc direction positions of P1R and P1L to match.

[0182] Next, the process of aligning the rotation direction of the y-axis will be described. The deviation Δxc, (1) between P1L and P1R in the xc direction is expressed by the following equation.

[0183]

[0184] The xr coordinate in the xr-yr-zr coordinate system of the light projecting unit 22R is expressed by the following equation.

[0185]

[0186] The amount of rotation when the light projecting unit 22R is rotated around the yc axis by the amount required to correct the deviation Δxc (1) between P1L and P1R in the xc direction, i.e., the correction amount, is Δφ r (1), the following equation is obtained:

[0187]

[0188] Correction amount Δφ r (1) is determined by the following formula:

[0189]

[0190] When correction is made using the position hr on the yr-axis instead of correction around the yr-axis, equation (41) including the differentiation of xr with respect to hr is used instead of equation (40).

[0191]

[0192] In the above example, the adjustment of the positions of the light projectors 22L and 22R in the xc and yc directions has been described. That is, the identified parameters are the xc coordinate and the yc coordinate. However, more parameters may be identified. Because the image captured by the front camera 21 contains information in the x and y directions, two parameters can be simultaneously identified using one point on the captured image. The number of parameters that can be simultaneously identified is twice the number of points used in the alignment pattern. If the alignment pattern has two points, four parameters can be simultaneously identified, and if it has three points, six parameters can be simultaneously identified.

[0193] To further improve accuracy, the light projection control unit 12 may perform the above correction once and then repeat the same correction again. In this way, correction is possible even on roads other than the flat and level road surface S0. Correction is also possible even when the road surface shape changes during driving.

[0194] 29 shows grid-like alignment patterns P1L, P2L, P3L, P4L, P5L, P6L, P7L, P8L, P9L, P10L, P11L, P12L and P1R, P2R, P3R, P4R, P5R, P6R, P7R, P8R, 9R, P10R, P11R, P12R. By using multiple alignment patterns in this way, it is possible to identify adjustment parameters at each location in the xc-yc coordinate system and change the adjustment parameter values ​​depending on the projection position.

[0195] The front camera 21, which uses an image element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), captures an image one frame at a time at each frame rate. While projecting a drawing pattern onto the driver 1, the light projecting unit 22 may momentarily project an alignment pattern in synchronization with the timing at which the front camera 21 captures the image. For example, the light projecting unit 22L may project an alignment pattern in one frame, and the light projecting unit 22R may project an alignment pattern in another frame.

[0196] Furthermore, the light projection control unit 12 may embed the alignment pattern in the drawing pattern for the driver 1, etc. In this case, the light projecting unit 22L and the light projecting unit 22R may be made distinguishable from each other by projecting light at different frame timings.

[0197] In the above description, the two light-projecting units, the light-projecting unit 22L and the light-projecting unit 22R, are used, but the light-projecting unit 22 may be composed of three or more light-projecting units.

[0198] 28 and 29 show alignment patterns with simple shapes, but the alignment patterns may be images such as figures or photographs. It is possible to correct not only the positions of identical points on the alignment patterns, but also the positions of points that have a specific relationship, such as parallel, rotated, or enlarged. The irradiation direction or irradiation position of the light projector 22 may be mechanically changed and adjusted.

[0199] J. Tenth Embodiment A road surface drawing device 110 according to a tenth embodiment has the same configuration as those according to the first, sixth, eighth and ninth embodiments, as shown in FIG.

[0200] In the ninth embodiment, the description has been given of correcting the deviation of the coordinate system caused by the different positions of the light projectors 22L and 22R. In the tenth embodiment, the light-projection control unit 12 corrects the deviation of the coordinate system caused by the different positions of the plurality of front cameras 21B. The correction method is the same as that in the ninth embodiment.

[0201] 3, two front cameras 21B are installed at the same positions as the light projecting units 22L and 22R. These two front cameras 21B capture images of the same object on the surrounding road surface. The light projecting control unit 12 determines the coordinates of the object in a coordinate system viewed from the origin of each front camera 21B from the camera frame images of each front camera 21B, and corrects at least one of the parameters representing the positional or rotational relationship between the two front cameras 21B.

[0202] Here, the object on the surrounding road surface used for correction does not have to be a pattern drawn on the surrounding road surface by the light projector 22. Instead of the front camera 21B, a position detection sensor such as a lidar or a radar may be used.

[0203] K. Eleventh Embodiment A road surface drawing device 111 according to an eleventh embodiment has the same configuration as those of the first to sixth and eighth to tenth embodiments, as shown in FIG.

[0204] In FIG. 3 , two front cameras 21B are installed at the same positions as light projectors 22L and 22R. Light projector 22L projects light in a certain direction and at a certain depression angle, drawing a pattern on the road surface. At this time, light-projection control unit 12 can determine the coordinates of the light-projection point on the road surface in a coordinate system whose origin is left front camera 21B, which is installed at the same position as light projector 22L. Next, right front camera 21B captures an image of the pattern drawn on the road surface. Then, light-projection control unit 12 converts the frame coordinates of the image captured by right front camera 21B into coordinates of the projection position, thereby determining the coordinates of the light-projection point in the coordinate system whose origin is right front camera 21B. Based on the thus-obtained deviation between the coordinates on the coordinate system whose origin is left front camera 21B and the coordinates on the coordinate system whose origin is front camera 21B, light-projection control unit 12 can correct the deviation between the coordinate systems in the same manner as in embodiment 9.

[0205] The tenth and eleventh embodiments may be combined. The light-projection control unit 12 may simultaneously perform the correction described in the tenth embodiment and the correction described in the eleventh embodiment to adjust the coordinate system of a sensor such as a lidar or radar that measures the distance to an object and the coordinate system of the light-projecting unit 22. That is, the light-projection control unit 12 may correct at least one of the positional relationship and the rotational relationship between the light-projecting unit 22 and the front camera 21B.

[0206] <L. Hardware Configuration> The image acquisition unit 11, light projection control unit 12, and warning control unit 13 in the road surface drawing devices 101-111 described above are realized by a processing circuit 81 shown in FIG. 30. That is, the processing circuit 81 includes the image acquisition unit 11, light projection control unit 12, and warning control unit 13 (hereinafter referred to as the "image acquisition unit 11, etc."). Dedicated hardware may be applied to the processing circuit 81, or a processor that executes a program stored in memory may be applied. Examples of the processor include a central processing unit, processing unit, arithmetic unit, microprocessor, microcomputer, DSP (Digital Signal Processor), etc.

[0207] When the processing circuitry 81 is dedicated hardware, the processing circuitry 81 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof. The functions of each unit, such as the image acquisition unit 11, may be realized by multiple processing circuits 81, or the functions of each unit may be realized together by a single processing circuit.

[0208] When the processing circuitry 81 is a processor, the functions of the image acquisition unit 11 and the like are realized by a combination of software, etc. (software, firmware, or software and firmware). The software, etc. is written as a program and stored in memory. As shown in Figure 31, a processor 82 applied to the processing circuitry 81 realizes the functions of each unit by reading and executing a program stored in memory 83. That is, the road surface drawing device 101-108 has a memory 83 for storing a program that, when executed by the processing circuit 81, results in the execution of the following steps: acquiring photographed image information of the surrounding road surface; causing the light-projecting unit 22 to project light of a first reference light-projection pattern that draws a first reference drawing pattern on the reference road surface; extracting from the photographed image information the actual drawing pattern drawn on the surrounding road surface by the light of the first reference light-projection pattern; calculating a correction amount based on the difference between the first drawing pattern and the first reference drawing pattern; creating a corrected light-projection pattern by correcting, with the correction amount, a second reference light-projection pattern that draws a second reference drawing pattern that is the same as or different from the first reference drawing pattern on the reference road surface; and causing the light-projecting unit 22 to project light of the corrected light-projection pattern.

[0209] In other words, this program causes a computer to execute the procedures or methods of the image acquisition unit 11, etc. Here, the memory 83 may be, for example, a non-volatile or volatile semiconductor memory such as a random access memory (RAM), a read-only memory (ROM), a flash memory, an erasable programmable read-only memory (EPROM), or an electrically erasable programmable read-only memory (EEPROM), a hard disk drive (HDD), a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, a digital versatile disk (DVD) and its drive device, or any storage medium that will be used in the future.

[0210] The above describes a configuration in which each function of the image acquisition unit 11 and the like is realized either by hardware or software, etc. However, the present invention is not limited to this, and a configuration in which part of the image acquisition unit 11 and the like is realized by dedicated hardware and another part is realized by software, etc. For example, the function of the light projection control unit 12 can be realized by a processing circuit as dedicated hardware, and the other functions can be realized by the processing circuit 81 as the processor 82 reading and executing programs stored in the memory 83.

[0211] As described above, the processing circuit can realize each of the above-described functions by hardware, software, or a combination of these.

[0212] Furthermore, although the road surface drawing devices 101-108 have been described above as in-vehicle devices, they can also be applied to a system constructed by appropriately combining a PND (Portable Navigation Device), a communication terminal (for example, a mobile terminal such as a mobile phone, smartphone, or tablet), the functions of applications installed on these, a server, etc. In this case, the functions or components of the road surface drawing devices 101-108 described above may be distributed among the devices that construct the system, or may be centrally located in one of the devices.

[0213] It should be noted that the embodiments can be freely combined, and each embodiment can be modified or omitted as appropriate. The above description is an example in all respects. It is understood that countless variations not illustrated can be envisioned.

[0214] The origin of the coordinate system used for specifying the projection pattern position or for calculating the coordinates does not have to be the center in the width direction of the vehicle body.

[0215] 3 Surrounding road surface, 11 Image acquisition unit, 12 Light projection control unit, 13 Warning control unit, 21, 21A, 21B Front camera, 22, 22L, 22R Light projection unit, 23 Output device, 81 Processing circuit, 82 Processor, 83 Memory, 101-111 Road surface drawing device.

Claims

1. an image acquisition unit that acquires captured image information, which is information on a captured image of a road surface around the vehicle captured by at least one camera mounted on the vehicle; a light projection control unit that controls projection of light onto the surrounding road surface by at least one light projection unit mounted on the vehicle; The light projection control unit includes: A light of a first reference light projection pattern for drawing a first reference drawing pattern on a reference road surface is projected from the at least one light projection unit; A first actual drawing pattern drawn on the surrounding road surface by the light of the first reference light projection pattern is extracted from the captured image information; calculating a correction amount based on a difference between the first actual drawing pattern and the first reference drawing pattern; A second reference drawing pattern, which may be the same as or different from the first reference drawing pattern, is drawn on the reference road surface; and a corrected light projection pattern is created by correcting the second reference light projection pattern, which may be the same as or different from the first reference light projection pattern, with the correction amount; by causing the at least one light-projecting unit to project light of the corrected light-projection pattern onto the surrounding road surface, a second actual drawing pattern is drawn on the surrounding road surface; the light projection control unit creates the corrected light projection pattern when a difference in area between the first actual drawing pattern and the first reference drawing pattern is equal to or greater than a predetermined threshold value, or creates the corrected light projection pattern based on a comparison between the distances between the feature points of the first actual drawing pattern corresponding to the feature points of the first reference drawing pattern, which are acquired from the captured image information, when the captured image information includes information on distances between feature points in the captured image. Road surface drawing device.

2. the at least one light-projecting unit is a plurality of light-projecting units provided at a plurality of locations on the vehicle, The at least one camera is a plurality of cameras provided at a plurality of locations on the vehicle corresponding to the plurality of light projecting units.

2. The road surface drawing device according to claim 1.

3. the light projection control unit creates the corrected light projection pattern so that an azimuth angle and a depression angle when each feature point of the second actual drawing pattern is viewed from a specific viewpoint match an azimuth angle and a depression angle when each feature point of the first reference drawing pattern corresponding to each feature point of the second actual drawing pattern is viewed from the specific viewpoint.

2. The road surface drawing device according to claim 1.

4. The second reference drawing pattern is an image that is stereoscopically viewed from a specific viewpoint.

2. The road surface drawing device according to claim 1.

5. the light of the first reference light projection pattern is invisible light, the light of the second reference light projection pattern and the correction light projection pattern is visible light; 2. The road surface drawing device according to claim 1.

6. the first reference light projection pattern and the second reference light projection pattern are identical, the light-projection control unit sets the most recent corrected light-projection pattern as a new second reference light-projection pattern, and sets the surrounding road surface at the time of creating the most recent corrected light-projection pattern as a new reference road surface to create the new corrected light-projection pattern.

2. The road surface drawing device according to claim 1.

7. and a warning control unit that causes an output device mounted on the vehicle to output a warning when a difference between the number of feature points of the first actual drawing pattern and the number of feature points of the first reference drawing pattern is equal to or greater than a predetermined threshold.

2. The road surface drawing device according to claim 1.

8. the light projection control unit increases the luminance of the light of the corrected light projection pattern when the downward gradient value of the surrounding road surface exceeds a predetermined threshold value compared to when the downward gradient value does not exceed a predetermined threshold value.

2. The road surface drawing device according to claim 1.

9. the at least one light projecting unit is a plurality of light projecting units, The image acquisition unit acquires information about an image captured by the at least one camera of an alignment pattern that is drawn by projecting light from the multiple light projecting units onto the surrounding road surface, and The light projection control unit corrects at least one of a positional relationship or a rotational relationship between the plurality of light projection units based on information of a captured image of the alignment pattern.

2. The road surface drawing device according to claim 1.

10. the at least one camera is a plurality of cameras; The image acquisition unit acquires information on images of the same object on the surrounding road surface captured by the multiple cameras, The light projection control unit detects a value relating to a position of the same object from information of images of the same object on the surrounding road surface captured by the multiple cameras, and corrects at least one of a positional relationship or a rotational relationship between the multiple cameras based on the detected value.

2. The road surface drawing device according to claim 1.

11. The light projection control unit detects a value relating to a position of light projected by the at least one light projection unit onto the surrounding road surface from information of an image of the surrounding road surface captured by the at least one camera, and corrects at least one of a position or a rotation between the at least one light projection unit and the at least one camera based on the detected value relating to the position of the light.

2. The road surface drawing device according to claim 1.

12. An image acquisition unit acquires captured image information, which is information on a captured image of a road surface around the vehicle captured by at least one camera mounted on the vehicle; a light projection control unit that controls projection of light onto the surrounding road surface by at least one light projection unit mounted on the vehicle; the light projection control unit causes the at least one light projection unit to project light of a first reference light projection pattern for drawing a first reference drawing pattern on a reference road surface; The light projection control unit extracts a first actual drawing pattern drawn on the surrounding road surface by the light of the first reference light projection pattern from the captured image information, the light projection control unit calculates a correction amount based on a difference between the first actual drawing pattern and the first reference drawing pattern; the light projection control unit draws a second reference drawing pattern that is the same as or different from the first reference drawing pattern on the reference road surface, and creates a corrected light projection pattern by correcting the second reference light projection pattern that is the same as or different from the first reference light projection pattern with the correction amount; the light projection control unit causes the at least one light projection unit to project light of the corrected light projection pattern onto the surrounding road surface, thereby drawing a second actual drawing pattern on the surrounding road surface; the light projection control unit creates the corrected light projection pattern when a difference in area between the first actual drawing pattern and the first reference drawing pattern is equal to or greater than a predetermined threshold value, or creates the corrected light projection pattern based on a comparison between the distances between the feature points of the first actual drawing pattern corresponding to the feature points of the first reference drawing pattern, which are acquired from the captured image information, when the captured image information includes information on distances between feature points in the captured image. Road surface drawing method.