Road surface drawing device and road surface drawing method

Abstract

The purpose of the present disclosure is to use light projection from a vehicle onto a road surface to draw a desired drawing pattern without distortion regardless of the gradient of the road surface. A road surface drawing device (101) comprises: an image acquisition unit (11) that acquires captured-image information of a road surface in the surroundings of a vehicle (M); and a light projection control unit (12) that controls light projection onto the surrounding road surface by a light projection unit (22) which is mounted on the vehicle (M). The light projection control unit (12): causes the light projection unit (22) to project light having a first reference light projection pattern for drawing a first reference drawing pattern on a reference road surface; extracts, from the captured-image information, a first actual drawing pattern which has been drawn on the surrounding road surface by the light having the first reference light projection pattern; calculates a correction amount on the basis of the difference between the first actual drawing pattern and the first reference drawing pattern; corrects, by the correction amount, a second reference light projection pattern for drawing a second reference drawing pattern on the reference road surface, so as to prepare a corrected light projection pattern; and causes the light projection unit (22) to project light having the corrected light projection pattern.

Description

[Technical field]

[0001] The present disclosure relates to a technique for drawing patterns such as figures or characters by irradiating light from a vehicle onto a road surface. [Background technology]

[0002] The road surface drawing device described in Patent Document 1 corrects the desired image and projects it based on the positional relationship, such as the height of the headlights or the distance between them, in order to correctly draw the desired image on the road surface without distortion. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-86966 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the road surface drawing device described in Patent Document 1 is based on the premise of drawing an image on a horizontal road surface, and therefore has the problem that the drawn image becomes distorted if the road surface is inclined or uneven.

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to project light from a vehicle onto a road surface to draw a desired drawing pattern without distortion, regardless of the slope or unevenness of the road surface. [Means for solving the problem]

[0006] The road surface drawing device disclosed herein includes an image acquisition unit that acquires captured image information, which is information on a captured image of a road surface surrounding a vehicle captured by at least one camera mounted on the vehicle, and a light projection control unit that controls light projection onto the surrounding road surface by at least one light projector mounted on the vehicle. The light projection control unit causes at least one light projector to project light of a first reference light projection pattern that draws a first reference drawing pattern on a reference road surface, extracts a first actual drawing pattern drawn on the surrounding road surface by the light of the first reference light projection pattern from the captured image information, calculates a correction amount based on a difference between the first actual drawing pattern and the first reference drawing pattern, draws a second reference drawing pattern identical or different from the first reference drawing pattern on the reference road surface, creates a corrected light projection pattern by correcting the second reference light projection pattern identical or different from the first reference light projection pattern with the correction amount, and causes at least one light projector to project light of the corrected light projection pattern onto the surrounding road surface to draw the second actual drawing pattern on the surrounding road surface. The light projection control unit creates a corrected light projection pattern when the difference in area between the first actual drawing pattern and the first reference drawing pattern is equal to or greater than a predetermined threshold value, or creates a corrected light projection pattern based on a comparison between the feature points of the first actual drawing pattern corresponding to the feature points of the first reference drawing pattern, which are acquired from the captured image information, when the captured image information includes information on the distance between the feature points in the captured image. do. Effect of the Invention

[0007] According to the road surface drawing device of the present disclosure, a desired drawing pattern can be drawn without distortion by projecting light from a vehicle onto the road surface, regardless of the gradient of the road surface. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram showing a state in which light is projected from a light-projecting unit of a vehicle onto a surrounding road surface. FIG. [Diagram 2] FIG. 2 is a diagram showing a reference drawing pattern drawn on a reference road surface. [Diagram 3] FIG. 13 is a diagram showing an actual drawing pattern that is drawn on a road surface with a slope. [Figure 4] 1 is a block diagram showing the configuration of a road surface drawing device and its peripheral devices according to a first embodiment. [Diagram 5] 11A and 11B are diagrams showing changes in the actual drawing pattern according to the gradient of the road surface. [Figure 6]4 is a flowchart showing the operation of the road surface drawing device according to the first embodiment. [Figure 7] FIG. 13 is a diagram showing the area of ​​an actual drawing pattern. [Figure 8] FIG. 13 is a diagram showing the area of ​​each divided region of the actual writing pattern. [Figure 9] FIG. 13 is a diagram showing feature points of an actual drawing pattern. [Figure 10] FIG. 13 is a diagram showing dimensions of an actual drawing pattern. [Figure 11] FIG. 13 is a diagram showing dimensions of each divided area of ​​an actual writing pattern. [Figure 12] 13 is a diagram showing a reference drawing pattern projected onto a flat road surface S0. FIG. [Figure 13] FIG. 13 is a diagram showing an actual drawing pattern projected onto an uneven road surface S1. [Figure 14] 13 is a flowchart showing the operation of the road surface drawing device according to the fourth embodiment. [Figure 15] 13 is a diagram showing how the azimuth angle of a projection point P1 of an actual drawing pattern as viewed from a viewpoint V is corrected to match the azimuth angle of a projection point P0 of a reference drawing pattern. FIG. [Figure 16] 13 is a diagram showing how the depression angle of a projection point P1 of an actual drawing pattern as viewed from a viewpoint V is corrected to match the depression angle of a projection point P0 of a reference drawing pattern. FIG. [Figure 17] 1 is a diagram showing a virtual rectangular parallelepiped viewed stereoscopically from a viewpoint V1. [Figure 18] This is an xz plane view of a virtual rectangular parallelepiped viewed stereoscopically from viewpoint V1. [Figure 19] This is an xy plan view of a virtual rectangular parallelepiped viewed stereoscopically from viewpoint V1. [Figure 20] This is an xy plan view of a virtual rectangular parallelepiped drawing pattern viewed stereoscopically from viewpoint V1. [Figure 21] 13 is a diagram showing a virtual rectangular parallelepiped stereoscopically viewed from viewpoint V2. [Figure 22] This is an xz plane view of a virtual rectangular parallelepiped viewed stereoscopically from viewpoint V2. [Figure 23] This is an xy plan view of a virtual rectangular parallelepiped viewed stereoscopically from viewpoint V2. [Figure 24] It is a plan view of the xy plane of a virtual rectangular parallelepiped drawing pattern stereoscopically viewed from the viewpoint V2. [Diagram 25] It is a block diagram showing the configuration of a road surface drawing device and its peripheral devices according to Embodiment 7. [Figure 26] It is a diagram showing a missing actual drawing pattern. [Figure 27] It is a flowchart showing the operation of the road surface drawing device according to Embodiment 7. [Figure 28] It is a diagram showing an alignment pattern projected in the same direction from two light projecting parts. [Figure 29] It is a diagram showing a grid-like alignment pattern. [Diagram 30] It is a diagram showing the hardware configuration of the road surface drawing device of each embodiment. [Diagram 31] It is a diagram showing the hardware configuration of the road surface drawing device of each embodiment.

Mode for Carrying Out the Invention

[0009] <A. Embodiment 1> FIG. 1 shows a state where light is projected from the light projecting parts 22L and 22R of the vehicle M onto the peripheral road surface 3. The light projecting parts 22L and 22R are, for example, the headlights of the vehicle M. In FIG. 1, the peripheral road surface 3 is the road surface in front of the vehicle M. In FIG. 1, the irradiation ranges of the light projecting parts 22L and 22R are indicated by reference numeral 6.

[0010] The road surface drawing device in Embodiment 1 controls the light projection of the light projecting parts 22L and 22R so that a predetermined pattern is drawn on the peripheral road surface 3. The pattern (hereinafter referred to as the "drawing pattern") drawn on the peripheral road surface 3 by the light projection of the light projecting parts 22L and 22R is, for example, directed toward the driver 1 of the vehicle M. An example of the drawing pattern for the driver 1 is an arrow indicating the desired traveling direction of the vehicle M. The driver 1 can drive the vehicle M along the traveling direction indicated by the drawing pattern on the peripheral road surface 3.

[0011] The drawing pattern may be directed to a traffic participant such as a pedestrian 7 around the vehicle M. An example of a drawing pattern for the pedestrian 7 is an arrow indicating the traveling direction of the vehicle M. The pedestrian 7 can grasp the traveling direction of the vehicle M by looking at the drawing pattern on the surrounding road surface 3 and take action to avoid the vehicle M.

[0012] In this way, the road surface drawing device controls the pattern of light projected from the light projectors 22L, 22R (hereinafter, "light projection pattern") so that a predetermined drawing pattern such as an arrow is drawn on the surrounding road surface 3. The drawing pattern may be drawn by projecting light from either one of the light projectors 22L, 22R.

[0013] 1 shows a case where the gradient of the surrounding road surface 3 is a horizontal road surface that does not change between the traveling point of the vehicle M and the projection position of the light projector 22. Such a horizontal road surface is also called a reference road surface.

[0014] In contrast, Fig. 2 shows a case where the gradient of the surrounding road surface 3 changes between the traveling point of the vehicle M and the projection position of the light-projecting unit 22. In Fig. 2, the gradient of the surrounding road surface 3 is horizontal at the traveling point of the vehicle M, but becomes downward in front of the vehicle M. Even if a light projection pattern similar to that for the reference road surface is projected onto the surrounding road surface 3, which is not the reference road surface, the desired drawing pattern cannot be obtained.

[0015] 2, the surrounding road surface 3, which is assumed to be horizontal, is shown by a dashed line. If the surrounding road surface 3 is horizontal, a circular drawing pattern 8 is drawn on the surrounding road surface 3. This drawing pattern 8 is the desired drawing pattern.

[0016] Fig. 3 is a view of the surrounding road surface 3 in Fig. 2 as viewed from above the vehicle M. The positions of the solid line A and the dashed lines B and C in Fig. 3 correspond to the positions of the solid line A and the dashed lines B and C in Fig. 2. As shown in Fig. 3, the drawing pattern 9 on the surrounding road surface 3 with a downward slope ends up being a pattern that extends in the traveling direction of the vehicle M. In this way, even if the same pattern is projected from the light-projecting unit 22, the pattern that is actually drawn on the surrounding road surface 3 differs depending on the gradient of the surrounding road surface 3.

[0017] Therefore, the road surface drawing device of embodiment 1 corrects the light projection pattern of the light-projecting unit 22 in accordance with the gradient of the surrounding road surface 3 of the vehicle M, so that the desired pattern is drawn on the surrounding road surface 3 of any gradient.

[0018] 4 is a block diagram showing the configuration of a road surface drawing device 101 and its peripheral devices according to the first embodiment. The road surface drawing device 101 corrects the light projection pattern of the light projector 22 mounted on the vehicle M in accordance with the gradient of the surrounding road surface 3, thereby suppressing distortion of a pattern (hereinafter referred to as an "actual drawing pattern") that is actually drawn on the surrounding road surface 3 by the light projection of the light projector 22. Note that the number of light projectors 22 may be one or more, and in Figs. 1 to 3, two light projectors 22 provided on the left front and right front of the vehicle M are shown as light projectors 22R and 22L.

[0019] The road surface drawing device 101 includes an image acquisition unit 11 and a light projection control unit 12.

[0020] The image acquisition unit 11 acquires an image captured by the front camera 21 of the vehicle M from the front camera 21. The front camera 21 is an example of a camera that captures an image of the surrounding road surface 3 of the vehicle M including the light projection range of the light projection unit 22. The front camera 21 captures an image in front of the vehicle M. The front camera 21 outputs captured image information, which is information on the captured image of the surrounding road surface 3, to the image acquisition unit 11. When the light projection unit 22 projects light onto the surrounding road surface 3, the captured image of the front camera 21 includes an actual drawing pattern of the surrounding road surface 3 by the light projection of the light projection unit 22.

[0021] The front camera 21 may be one or more. FIG. 1 illustrates one front camera 21A provided above the head of the driver 1. That is, the front camera 21 may be one front camera 21A. Also, FIG. 2 and FIG. 3 illustrate two front cameras 21B provided near two light projecting units 22L, 22R of the vehicle M. That is, the front camera 21 may be a plurality of front cameras 21B provided at a plurality of locations of the vehicle M corresponding to the plurality of light projecting units 22L, 22R.

[0022] In order to know the correction amount of the light projection pattern according to the surrounding road surface 3, the light projection control unit 12 first creates a first reference light projection pattern and causes the light projection unit 22 to project this. Here, the first reference light projection pattern is a light projection pattern of the light projection unit 22 that is projected onto the reference road surface to draw the first reference drawing pattern on the reference road surface. The first reference drawing pattern is a pattern that is compared with an actual drawing pattern described later and is used to calculate a correction amount according to the gradient of the surrounding road surface 3. The first reference drawing pattern is, for example, a desired pattern that the road surface drawing device 101 wants to draw on the surrounding road surface 3. Alternatively, the first reference drawing pattern may be a simpler pattern.

[0023] When the first reference light projection pattern is projected from the light projector 22 onto the surrounding road surface 3, a pattern is drawn on the surrounding road surface 3. The actual drawing pattern drawn on the surrounding road surface 3 by the first reference light projection pattern is referred to as the actual drawing pattern. An image of the actual drawing pattern is captured by the front camera 21 and acquired by the image acquisition unit 11. The light projection control unit 12 extracts the actual drawing pattern from the captured image information acquired from the image acquisition unit 11, and calculates the amount of correction by comparing the actual drawing pattern with the first reference drawing pattern.

[0024] The light-projection control unit 12 creates a corrected light-projection pattern by correcting the second reference light-projection pattern using the calculated correction amount, and causes the light-projection unit 22 to project this corrected light-projection pattern. The second reference light-projection pattern is a light-projection pattern that enables the road surface drawing device 101 to draw an intended pattern on the reference road surface. Therefore, when the intended pattern that the road surface drawing device 101 wants to draw on the surrounding road surface 3 is used as the first reference drawing pattern, the second reference light-projection pattern is the same pattern as the first reference light-projection pattern. Also, when a simpler pattern is used as the first reference drawing pattern, the second reference light-projection pattern is a pattern different from the first reference light-projection pattern.

[0025] The actual drawing pattern drawn on the surrounding road surface 3 by the first reference light projection pattern is also referred to as the first actual drawing pattern, and the actual drawing pattern drawn on the surrounding road surface 3 by the corrected light projection pattern is also referred to as the second actual drawing pattern. When the first reference light projection pattern and the second reference light projection pattern are the same pattern, the deviation between the second actual drawing pattern and the reference drawing pattern is smaller than the deviation between the first actual drawing pattern and the reference drawing pattern. In this way, by correcting the light projection pattern of the light-projecting unit 22, it is possible to draw the desired pattern regardless of the gradient of the surrounding road surface 3.

[0026] In the following description, the first reference drawing pattern is assumed to be a desired pattern that the road surface drawing device 101 wishes to draw on the surrounding road surface 3, and the first reference light projection pattern and the second reference light projection pattern are assumed to be the same pattern.

[0027] In FIG. 2, the circular drawing pattern 8 was the reference drawing pattern. However, the shape of the reference drawing pattern is not limited to a circle. FIG. 5 shows reference drawing patterns of various shapes and corresponding actual drawing patterns. In FIG. 5, the dashed lines indicate the reference drawing patterns, and the matte-hatched areas indicate the actual drawing patterns. The reference drawing patterns are patterns such as circles, straight arrows, or right-turn arrows. When the surrounding road surface 3 is a flat reference road surface, the actual drawing pattern matches the reference drawing pattern. However, when the surrounding road surface 3 has a downward slope, the actual drawing pattern is a pattern in which the reference drawing pattern is extended in the traveling direction of the vehicle M. Also, when the surrounding road surface 3 has an upward slope, the actual drawing pattern is a pattern in which the reference drawing pattern is shortened in the traveling direction of the vehicle M.

[0028] 6 is a flowchart showing the operation of the road surface drawing device 101. The operation of the road surface drawing device 101 will be described below with reference to the flow of FIG.

[0029] First, in step S101, the light-projection control unit 12 creates a reference light-projection pattern. The light-projection control unit 12 creates the reference light-projection pattern based on a reference drawing pattern to be drawn on the surrounding road surface 3.

[0030] Next, in step S102, the light-projection control unit 12 outputs the reference light-projection pattern to the light-projection unit 22, and instructs the light-projection unit 22 to project the reference light-projection pattern onto the surrounding road surface 3. As a result, the light-projection unit 22 projects the reference light-projection pattern onto the surrounding road surface 3, and an actual drawing pattern is drawn. This actual drawing pattern is photographed by the front camera 21.

[0031] Thereafter, in step S103, the image acquisition unit 11 acquires a photographed image of the actual drawing pattern from the front camera 21.

[0032] Next, in step S104, the light projection control unit 12 compares the actual drawing pattern with the reference drawing pattern to determine whether or not the reference light projection pattern needs to be corrected. If correction is not required in step S104, the process of the road surface drawing device 101 returns to step S103. If correction is required in step S104, the amount of correction of the reference light projection pattern is calculated in step S105.

[0033] After step S105, in step S106, the light-projection control unit 12 corrects the reference light-projection pattern using the correction amount calculated in step S105, and creates a corrected light-projection pattern.

[0034] After step S106, in step S107, the light-projection control unit 12 outputs a corrected light-projection pattern to the light-projection unit 22, and instructs the light-projection unit 22 to project the corrected light-projection pattern onto the surrounding road surface 3. As a result, the light-projection unit 22 projects the corrected light-projection pattern onto the surrounding road surface 3. Then, the processing of the road surface drawing device 101 returns to step S103.

[0035] A specific example of the processing of steps S104-106 will be described below. In this specific example, the light projection control unit 12 compares the area of ​​the reference drawing pattern with the area of ​​the actual drawing pattern, and creates a corrected light projection pattern if the difference between the areas is equal to or greater than a predetermined threshold. In the following description, the relative positions of the eyes of the driver 1 of the vehicle M, the front camera 21, and the light projection unit 22 are fixed. Also, the reference drawing pattern, the image captured by the front camera 21 of the reference drawing pattern, and the reference drawing pattern visually recognized by the driver 1 are assumed to be similar in shape.

[0036] 7 shows an arrow-shaped actual drawing pattern. The light projection control unit 12 calculates the area A of the actual drawing pattern and the area Ao of the reference drawing pattern, calculates the area ratio between them, A / Ao=AK, and sets the reciprocal of the area ratio, 1 / AK, as the correction amount of the reference light projection pattern.

[0037] Further, the light projection control unit 12 compares the area ratio AK of the actual drawing pattern and the reference drawing pattern with a preset threshold AKth, and determines that correction is not required when AK < AKth, and that correction is required when AK ≧ AKth. In other words, the light projection control unit 12 creates a corrected light projection pattern when the difference in area between the actual drawing pattern and the reference drawing pattern is equal to or greater than a preset threshold. In the case where correction is required, the light projection control unit 12 creates a corrected light projection pattern by expanding or reducing the area of the reference light projection pattern by 1 / AK times.

[0038] In FIG. 7, the area of the entire region of the actual drawing pattern is compared with the area of the entire region of the reference drawing pattern. However, the light projection control unit 12 may divide the actual drawing pattern into a plurality of regions as shown by the broken line in FIG. 8, and perform area comparison for each divided region. In FIG. 8, the actual drawing pattern is divided into six regions. Let the areas of the divided regions of the actual drawing pattern be A1 - A6. The light projection control unit 12 divides the reference drawing pattern into six regions corresponding to each divided region of the actual drawing pattern. Let the areas of the divided regions of the reference drawing pattern be Ao1 - Ao6. The light projection control unit 12 calculates the area ratio Ai / Aoi = AKi (where i is a natural number from 1 to 6) of the actual drawing pattern and the reference drawing pattern for each divided region, and uses the reciprocal 1 / AKi as the correction amount of the reference light projection pattern.

[0039] Next, the light projection control unit 12 compares the area ratio AKi of the actual drawing pattern and the reference drawing pattern for each divided region with a preset threshold AKith, and determines that correction is not required when AKi < AKith in all divided regions, and determines that correction is required if there is at least one divided region where Aki ≧ Akith. In the case where correction is required, the light projection control unit 12 creates a corrected light projection pattern by expanding or reducing the area of the region of the reference light projection pattern corresponding to the divided region of the reference drawing pattern where Aki ≧ Akith by 1 / Aki times.

[0040] The road surface drawing device 101 according to Embodiment 1 includes an image acquisition unit 11 and a light projection control unit 12. The image acquisition unit 11 acquires captured image information, which is information on a captured image of the surrounding road surface 3 of the vehicle M captured by at least one camera mounted on the vehicle M. The light projection control unit 12 controls the light projection onto the surrounding road surface 3 by at least one light projection unit 22 mounted on the vehicle M. More specifically, the light projection control unit 12 causes at least one light projection unit 22 to project light of a first reference light projection pattern for drawing a first reference drawing pattern on the reference road surface, extracts a first actual drawing pattern drawn on the surrounding road surface 3 by the light of the first reference light projection pattern from the captured image information, calculates a correction amount based on the difference between the first reference drawing pattern and the second actual drawing pattern, creates a corrected light projection pattern by correcting a second reference light projection pattern, which is the same as or different from the first reference light projection pattern, with the correction amount for drawing a second reference drawing pattern, which is the same as or different from the first reference drawing pattern, on the reference road surface, and causes at least one light projection unit 22 to project the light of the corrected light projection pattern, thereby drawing a second actual drawing pattern on the surrounding road surface 3. Thereby, regardless of the gradient of the surrounding road surface 3, a desired target pattern to be drawn on the surrounding road surface 3 is drawn.

[0041] <B. Embodiment 2> As shown in FIG. 4, the road surface drawing device 102 of Embodiment 2 has the same configuration as the road surface drawing device 101 of Embodiment 1. In Embodiment 2, the specific processing in steps S104 - 106 of FIG. 6 is different from that of Embodiment 1 as follows.

[0042] In Embodiment 1, the light projection control unit 12 calculates the correction amount by comparing the areas of the reference drawing pattern and the actual drawing pattern. On the other hand, in Embodiment 2, the light projection control unit 12 calculates the correction amount by comparing the distances between the feature points of the reference drawing pattern and the actual drawing pattern.

[0043] FIG. 9 shows the feature points a, b, d, e, f, g of the actual drawing pattern in the shape of an arrow. The light projection control unit 12 can extract these feature points a, b, d, e, f, g by using blinking of the image of the actual drawing pattern, pattern matching, or detection algorithms such as SURF (Speeded-Up Robust Features) or SIFT (Scale-Invariant Feature Transform).

[0044] Then, as shown in FIG. 10, the light projection control unit 12 calculates the distance L1 between the feature points b and e, the distance L2 between the feature points a and g or a and f, and the distance L3 between the feature points f and g. Similarly, the light projection control unit 12 extracts the feature points ao, bo, do, eo, fo, go of the reference drawing pattern corresponding to the feature points a, b, d, e, f, g of the actual drawing pattern (not shown), and calculates the distance Lo1 between the feature points ao and eo, the distance Lo2 between the feature points ao and go or ao and fo, and the distance Lo3 between the feature points fo and go.

[0045] In step S104, the light projection control unit 12 compares the ratio LKi of the distances between the feature points with a predetermined threshold value LKth, determines that correction is not required when LKi < LKth, and determines that correction is required when LKi ≧ LKth.

[0046] When correction is required in step S104, in step S105, the light projection control unit 12 calculates the ratio Li / Loi = LKi (i = 1, 2, 3) of the distances between the feature points of the actual drawing pattern and the reference drawing pattern, and uses the reciprocal 1 / LKi as the correction amount of the reference light projection pattern.

[0047] After step S105, in step S106, the light projection control unit 12 multiplies the distance between the feature points of the reference light projection pattern corresponding to the feature points of the reference drawing pattern by 1 / LKi to expand or contract the reference light projection pattern and create a corrected light projection pattern.

[0048] Regarding the comparison of the distances between the feature points of the actual drawing pattern and the reference drawing pattern, similar to the area comparison, it may be performed for each divided region. In the example of FIG. 11, the actual drawing pattern is divided into a total of four regions on the left and right and vertically and horizontally, and the distances between the feature points in each divided region are calculated as L11, L12, L21, L22, L23, L24, L31, L32.

[0049] In the road surface drawing device 102 according to the second embodiment, the photographed image information includes information on the distances between the feature points in the photographed image. The light projection control unit 12 creates a corrected light projection pattern based on a comparison between the distances between the respective feature points of the first reference drawing pattern and the distances between the respective feature points of the actual drawing pattern corresponding to the respective feature points of the first reference drawing pattern obtained from the photographed image information.

[0050] <C. Third Embodiment> The road surface drawing device 103 of the third embodiment has the same configuration as the road surface drawing devices 101 and 102 of the first and second embodiments as shown in FIG. 4. In the third embodiment, the specific processing in steps S104 - 106 of FIG. 6 is different from those of the first and second embodiments as follows.

[0051] As shown in FIG. 3, when the front camera 21 is composed of two front cameras 21B, that is, a stereo camera, the light projection control unit 12 can measure the distances between the respective feature points of the actual drawing pattern and the front camera 21B from the images of the actual drawing pattern photographed by the two front cameras 21B. Therefore, the light projection control unit 12 can correct the reference light projection pattern so that the distances between the respective feature points of the actual drawing pattern and the front camera 21B match the distances between the respective feature points of the reference drawing pattern and the front camera 21B.

[0052] FIG. 3 shows the distances D1 and D2 between the respective feature points of the drawing pattern 9 (actual drawing pattern) and the front camera 21B. Similarly, the light projection control unit 12 calculates the distances Do1 and Do2 between the respective feature points of the reference drawing pattern and the front camera 21B.

[0053] In step S104, the light projection control unit 12 compares the ratio of distances DKi with a predetermined threshold value Dth, determines that correction is not required when DKi < Dth, and determines that correction is required when DKi ≥ Dth.

[0054] In the case where correction is required in step S104, in step S105, the light projection control unit 12 calculates the ratio Di / Doi = DKi (i = 1, 2) between the distance of each feature point in the actual drawing pattern from the front camera 21B and the distance of each feature point in the reference drawing pattern from the front camera 21B, and sets the reciprocal 1 / DKi as the correction amount of the reference light projection pattern.

[0055] After step S105, in step S106, the light projection control unit 12 corrects the reference light projection pattern to obtain a corrected light projection pattern so that a drawing pattern obtained by multiplying the distance between each feature point of the reference drawing pattern and the front camera 21B by 1 / DKi can be drawn. According to this method, even if the gradient of the surrounding road surface 3 changes, a pattern can always be drawn at a constant distance from the vehicle M.

[0056] <D. Embodiment 4> The road surface drawing device 104 of Embodiment 4 has the same configuration as that of Embodiments 1-3 as shown in FIG. 4. In Embodiment 4, the specific processing of steps S104-106 in FIG. 6 is different from that of Embodiments 1-3. The road surface drawing device 104 corrects the reference light projection pattern to create a corrected light projection pattern so that the azimuth angle and elevation angle of viewing each feature point of the second actual drawing pattern from a specific viewpoint match the azimuth angle and elevation angle of viewing each feature point of the first reference drawing pattern corresponding to each feature point of the second actual drawing pattern from the specific viewpoint.

[0057] FIG. 12 shows a state where a reference light projection pattern is projected from a light source O toward a flat road surface S0. The xyz coordinate system has an origin on the body of the vehicle M. The x-axis direction is the forward direction of the vehicle body, the y-axis direction is the rightward direction, and the z-axis is the height direction. A perpendicular line drawn from the light source O to the frame surface f of the light projecting units 22L and 22R or the front camera 21 intersects with the frame surface at the center of the frame surface f. The position within the frame is indicated by the angle with respect to the origin or the distance in the coordinate axis direction. This is called the frame coordinate system. Once the projection surface is determined, it is possible to convert the frame coordinate system to the coordinate system of the projection position by a formula. When an arrow-shaped drawing pattern having an outer frame ABCD is drawn on the frame coordinate system of the light projecting units 22L and 22R, a figure with the coordinate conversion is drawn on the road surface S0. The rectangle on the frame coordinate system is projected into a trapezoid shape onto the road surface S0. The road surface S0 is the reference road surface, and the pattern drawn on the road surface S0 is the reference drawing pattern. The projection point constituting the reference drawing pattern on the road surface S0 is defined as P0 (x0, y0, z0).

[0058] FIG. 13 shows a state in which the same reference light projection pattern as in FIG. 12 is projected from a light source O toward a road surface S1. Because the road surface S1 is not flat, the actual drawing pattern drawn on the road surface S1 differs from the reference drawing pattern in FIG. 12 and is distorted. The outer frame ABCD in the reference drawing pattern in FIG. 12 is transformed into an outer frame A'B'C'D' in the actual drawing pattern in FIG. 13. As with the outer frame, the arrows within the frame are also transformed in the same way. The projection point constituting the actual drawing pattern on the road surface S1 is defined as P1(x1, y1, z1).

[0059] Now consider correcting the distortion of the actual drawing pattern. However, because road surface S0 and road surface S1 are at different positions in the height direction (z-axis direction in Figures 12 and 13), the actual drawing pattern cannot be drawn at the same height as road surface S0 even if the reference light projection pattern is corrected. The magnitude and direction of the deviation between feature points on the reference drawing pattern and the corresponding feature points on the actual drawing pattern differ depending on the viewpoint from which the actual drawing pattern is viewed.

[0060] Therefore, the road surface drawing device 104 creates a corrected light projection pattern by correcting the reference light projection pattern to be projected onto the road surface S1 so that the direction (azimuth angle and depression angle) in which each feature point of the actual drawing pattern on the road surface S1 is viewed from an arbitrary viewpoint V matches the direction (azimuth angle and depression angle) in which each feature point of the reference drawing pattern on the road surface S0 is viewed from the same viewpoint V. In this way, the road surface drawing device 104 corrects differences in the appearance of the drawing pattern due to the road surface.

[0061] 14 is a flowchart of the light projection pattern correction process by the road surface drawing device 104. The light projection pattern correction process by the road surface drawing device 104 will be described below with reference to the flow of FIG.

[0062] First, in step S201, the light projection control unit 12 acquires a projection image to be drawn on the road surface S1. The projection image may be a simple two-dimensional figure, a three-dimensional figure, or a plurality of figures that change in time series.

[0063] Next, in step S202, the light projection control unit 12 acquires the position of the viewpoint V (hereinafter, viewpoint position) (Xv, Yv, Zv). The viewpoint position may be a predetermined value, or may be a value acquired using a three-dimensional distance sensor such as Lidar (Light Detection And Ranging) or a camera.

[0064] Then, in step S203, the light projection control unit 12 creates a reference drawing pattern by correcting the projection image at the viewpoint position. When the reference drawing pattern drawn on the reference road surface is viewed from the viewpoint position, it appears as a projection image. The reference drawing pattern is determined based on the projection image, the position of the light projection unit 22, and the viewpoint position.

[0065] Next, in step S204, the light projection control unit 12 creates a reference light projection pattern based on the reference drawing pattern. The reference light projection pattern is a pattern set in the light projection unit 22, and is created by back-calculating from the reference drawing pattern. This step corresponds to step S101 in FIG. 6.

[0066] After that, in step S205, the light projection control unit 12 extracts feature points of the reference light projection pattern.

[0067] The positions of the feature points of the reference drawing pattern corresponding to the feature points of the reference light projection pattern change depending on the depth of the projection surface, i.e., the road surface S1. Therefore, in step S206 after step S205, the light projection control unit 12 calculates the trajectory of the feature points of the reference drawing pattern when the depth of the projection surface (z-axis position in FIG. 13) is changed. The trajectory of the feature points may be represented by a table or a formula.

[0068] In parallel with steps S205 and S206, in step S207, the light-projection control unit 12 controls the projection of the reference light-projection pattern. This step corresponds to step S102 in Fig. 6. As a result, the reference light-projection pattern is projected from the light-projection unit 22 onto the road surface S1, and the reference drawing pattern is drawn on the road surface S1.

[0069] After step S207, the front camera 21 captures the actual drawing pattern drawn on the road surface S1. Then, in step S208, the image acquisition unit 11 acquires captured image information including the actual drawing pattern from the front camera 21. This step corresponds to step S103 in FIG.

[0070] After step S208, in step S209, the light projection control unit 12 extracts feature points of the actual drawing pattern. Here, the feature points are extracted by blinking the image of the actual drawing pattern, pattern matching, or a detection algorithm such as SURF (Speed-Up Robust Features) or SIFT (Scale-Invariant Feature Transform).

[0071] After steps S206 and S209, in step S210, the light projection control unit 12 compares the reference drawing pattern with the actual drawing pattern, and calculates or modifies the correction amount of the reference light projection pattern by changing the position of the feature point of the actual drawing pattern on the trajectory calculated in step S206. Specifically, the light projection control unit 12 moves the feature point of the actual drawing pattern on the trajectory calculated in step S206 so that the azimuth angle and depression angle of the actual drawing pattern as seen from the viewpoint V approach the azimuth angle and depression angle of the reference drawing pattern as seen from the viewpoint V. The correction amount of the reference light projection pattern is determined based on the movement direction and movement distance of the feature point of the actual drawing pattern.

[0072] Next, in step S211, the light-projection control unit 12 creates a corrected light-projection pattern by correcting the reference light-projection pattern based on the amount of correction calculated in step S210.

[0073] Next, in step S212, the light projection control unit 12 determines whether there is a change in the projection image. If there is a change in the projection image, the process of the road surface drawing device 104 returns to step S201. If there is no change in the projection image, in step S213, the light projection control unit 12 determines whether there is a change in the viewpoint V.

[0074] If there is a change in the viewpoint V, the process of the road surface drawing device 104 returns to step S202. If there is no change in the viewpoint V, the process of the road surface drawing device 104 returns to step S207. In step S207, the light projection control unit 12 performs light projection control using the corrected light projection pattern created in step S211 as a new reference light projection pattern.

[0075] That is, while there is no change in either the projection image or the viewpoint V, the light projection control unit 12 repeats the calculation process of the correction amount in step S210. The second and subsequent calculations of the correction amount are performed based on the previous correction amount, the position of the feature point of the actual drawing pattern moved in the previous correction amount calculation process, and the position of the feature point of the current actual drawing pattern. As a result, the azimuth angle and depression angle of the actual drawing pattern as viewed from the viewpoint V gradually approach the azimuth angle and depression angle of the reference drawing pattern as viewed from the viewpoint V.

[0076] That is, in this example, the first reference light-projection pattern and the second reference light-projection pattern are the same. Then, the light-projection control unit 12 creates a new corrected light-projection pattern by setting the most recent corrected light-projection pattern as a new second reference light-projection pattern and setting the surrounding road surface at the time of creating the most recent corrected light-projection pattern as a new reference road surface.

[0077] The light-projection control unit 12 controls the azimuth angle and depression angle of the reference light-projection pattern projected from the light-projection unit 22 so that the azimuth angle and depression angle of the projection point P1 onto the distorted road surface S1 as viewed from an arbitrary viewpoint V coincide with the azimuth angle and depression angle of the projection point P0 onto the flat road surface S0. The specific procedure is as follows.

[0078] Step 1: The light projecting unit 22 projects a reference light projection pattern at an azimuth angle θ 0 , depression angle φ 0 The projection is in the direction of

[0079] Step 2: Calculate the amount of correction for the projection azimuth angle of the light projector 22 in the xy view. Fig. 15 shows the projection state in the xy view. P1 is set as the feature point of the actual drawing pattern. The amount of correction for the projection azimuth angle is calculated so that P1 moves to P1' on the extension of the straight line V-P0.

[0080] Step 3: Calculate the correction amount of the projection depression angle of the light projector 22 in the xz view. Figure 16 shows the projection state in the xy view. The correction amount of the projection depression angle is calculated so that P1 moves to P1' on the extension line of the straight line V-P0.

[0081] Step 4: Using the correction amount calculated in step 3, the light projection azimuth angle and the light projection depression angle of the light projector 22 are corrected.

[0082] Step 5: Repeat steps 2, 3, and 4.

[0083] While general projection mapping uses a fixed surface such as a building wall as the projection surface, in vehicle applications the shape of the projection surface changes as the vehicle travels. Therefore, if steps 2 and 3 are performed once and correction is stopped, the road surface shape will change as the vehicle travels thereafter, causing the image projected onto the road surface to shake. However, by repeating steps 2, 3, and 4 in step 5, corrections can be made continuously and the shaking of the image can be reduced.

[0084] A specific example of steps 2 and 3 will be described below with reference to Figures 15 and 16. First, parameters used in the specific example of steps 2 and 3 will be described below.

[0085] O(0,0,0), C(Xc,Yc,Zc), and V(Xv,Yv,Zv) are fixed points, and respectively represent the positions of the light projector 22, the front camera 21, and the viewpoint. The direction vector of O, that is, the light projection direction vector of the light projector 22, is expressed as ω 0 (θ 0 ,φ 0 ) The direction vector of C, that is, the direction vector seen from the front camera 21, is ω c (θ c ,φ c ) Let the direction vector seen from V be ω V (θ V ,φ V )

[0086] S0 is a flat road surface that serves as a reference road surface. S0 is parallel to the xy plane, and the z-axis coordinate of S0 is -h. S1 is a non-flat surrounding road surface, which is the road surface onto which the light projecting unit 22 actually projects light. The shape of S1 is unknown.

[0087] P represents the projection point of the light projector 22. P0(x0, y0, z0) represents the projection point onto the road surface S0 by the reference light projection pattern. P1(x1, y1, z1) represents the projection point onto the road surface S1 by the reference light projection pattern. P1'(x1', y1', z1') represents the projection point onto the road surface S1 by the corrected light projection pattern. P1' is determined so that the direction in which P1' is viewed from viewpoint V matches the direction in which P0 is viewed from viewpoint V. In other words, P1' is on an extension of the straight line V-P0.

[0088] θ is the azimuth angle, which is the angle between the straight line and the positive y-axis direction in the xy view. θ O is the projection azimuth angle of the light projector 22. c is the azimuth angle at which the projection point is viewed from the front camera 21. v is the azimuth angle from viewpoint V to the projection point. θ o,0 is the projection azimuth angle from the light projector 22 to P0. o,1´ is the projection azimuth angle from the light projector 22 to P1'. c,1 is the azimuth angle from the front camera 21 to P1. V,0 is the azimuth angle from viewpoint V to P0. V,1´ is the azimuth angle of P1' viewed from viewpoint V.

[0089] φ is the depression angle, which is the angle with the xy plane. φ x-z is the depression angle in the xz direction. φ x-z In FIG. 16, the counterclockwise direction is +. O,x-z,0 is the depression angle of the light projected from O to P0 in the xz view. -φ O,x-z,1´ is the depression angle of the light projected from O to P1' in the xz view. -φ C,x-z,1 is the depression angle when looking at P1 from C in the xz view. -φ V,x-z,0 is the depression angle when looking at P0 from V in xz view.

[0090] The following are calculated values. X1_Azimuth is θ O,1´ is the x-direction position of P1' used in the calculation of Z1_Elevation. O,1´ is the z-position of P1' used in the calculation of

[0091] Next, a specific example of step 2 will be described with reference to FIG. 15. The light projection control unit 12 manipulates the light projection azimuth of the projection point P1 in the xy view to place it on the extension line of V-P0. In the xy view, the position in the z direction can be ignored. The azimuth angle θ when an arbitrary projection point P(x,y,z) is viewed from V, C, O is V ,θ C ,θ O is expressed by the following formula:

[0092]

number

[0093] First, the light projecting unit 22 is O,1 (θ O,1 ,φ O,1 ) direction and projects a point P1. Point P1 is located at a distance of ω C,1 (θ C,1 ,φ C,1 ) direction. O,1 ,θ C,1 From FIG. 15, it can be expressed by the following equation.

[0094]

number

[0095] By transforming equations (4) and (5), we obtain the following equation.

[0096]

number

[0097] From equations (6) and (7), the x-coordinate x1 of the projection point P1 is expressed by the following equation.

[0098]

number

[0099] Projection azimuth angle θ O Even if you change θ, the x coordinate of the projection point will remain the same as x1=X1_Azimuth. O The y coordinate of the projection point when is changed is expressed by the following formula.

[0100]

number

[0101] From equations (1), (8), and (9), the following equation is obtained.

[0102]

number

[0103] The position of the corrected projection point P1' is determined so that the azimuth angle at which the corrected projection point P1' is viewed from the viewpoint V is the same as the azimuth angle at which the projection point P0 on the road surface S0 is viewed from the viewpoint V. Therefore, the y-coordinate y1' of the corrected projection point P1' is expressed as follows.

[0104]

number

[0105] By substituting equation (8) into this, y1' is expressed as follows.

[0106]

number

[0107] The projection azimuth angle θ from the position O of the light projecting unit 22 to the corrected projection point P1′ O,1´ is expressed by the following formula:

[0108]

number

[0109] According to the formulas (8), (12), and (13), the projection azimuth angle θ O,1´ is determined. P1' is located on a plane that satisfies x=X1_Azimuth and on the extension of V-P0. However, since the road surface S1 is unknown, the actually projected point P1' often approaches the extension of V-P0, but is not necessarily located exactly on the extension of V-P0.

[0110] Next, a specific example of step 3 will be described with reference to Fig. 16. The light projection control unit 12 operates the projection depression angle of the projection point P1 in the xz view to place it on the extension line of V-P0. The depression angle φ in the xz view when an arbitrary point P(x, y, z) is viewed from the viewpoint V, the position C of the front camera 21, and the position O of the light projection unit 22 is V,x-z ,φ C,x-z ,φ O,x-z is expressed by the following formula:

[0111]

number

[0112] Consider the projection point P1' after the first correction. First, the light projecting unit 22 projects from O to ω O,1 (θ O,1 ,φ O,1 ) direction and projects a point P1. Point P1 is located at a distance of ω C,1 (θ C,1 ,φ C,1 ) direction. The depression angle φ when looking at point P1 from O, C in xz view O,x-z,1 ,φ C,x-z,1 is expressed by the following equation from FIG.

[0113]

number

[0114] By transforming equations (17) and (18), the following equation is obtained.

[0115]

number

[0116] From equations (19) and (20), the x-coordinate x1 of the projection point P1 is expressed by the following equation.

[0117]

number

[0118] By transforming equation (20), the z coordinate z1 of the projection point P1 is expressed by the following equation.

[0119]

number

[0120] Light projection depression angle φ O,x-z Even if the z coordinate of the projection point is changed, z1 = Z1_Elevation remains unchanged. φ O,x-z The x-coordinate of the projection point when is changed is expressed by the following formula.

[0121]

number

[0122] Depression angle φ from viewpoint V to P0 v,x-z,0 and the depression angle φ from the viewpoint V to the corrected projection point P1' v,x-z,1´ are expressed by the following formulas, respectively.

[0123]

number

[0124] φ v,x-z,0 and φ v,x-z,1´ Make sure that is the same.

[0125] Since Z1´=Z1_Elevation, equation (25) transforms into equation (26).

[0126]

number

[0127] The position of the corrected projection point P1' is determined so that the depression angle when the corrected projection point P1' is viewed from the viewpoint V is the same as the depression angle when the projection point P0 on the road surface S0 is viewed from the viewpoint V. Therefore, from equations (24) and (26), the x-coordinate x1' of the corrected projection point P1' is expressed by the following equation.

[0128]

number

[0129] The depression angle φ on the xz plane from the position O of the light projecting unit 22 to the corrected projection point P1' O,x-z,1´ is expressed by the following formula:

[0130]

number

[0131] The light projection azimuth angle θ from the position O of the light projector 22 O,1´ is as shown in the formula (13). Therefore, the depression angle φ from the position O of the light projecting unit 22 to the corrected projection point P1' is O,1´ (28) is applied to the projection azimuth angle θ O,1´ This is corrected and expressed by the following formula:

[0132]

number

[0133] According to the formulas (8), (12), and (13), the projection azimuth angle θ O,1´ The projection depression angle φ to the projection point P1' after correction is determined by the equations (12), (13), (27), and (29). O,1´ The light projection control unit 12 determines the light projection azimuth angle θ O,1´ and projection depression angle φ O,1´ As a result, when P1' is projected onto a plane that satisfies z=Z1_Elevation, P1' is located on the extension line of V-P0.

[0134] However, since S1 is an unknown road surface, the projection point P1' does not necessarily satisfy z=Z1_Elevation. Therefore, in many cases, P1' approaches the extension line of V-P0, but is not necessarily located exactly on the extension line. Nevertheless, by repeating the correction process with the obtained P1' as the new P1, P1' will come close enough to the extension line of V-P0.

[0135] In the road surface drawing device 104 according to the fourth embodiment, the light projection control unit 12 creates a corrected light projection pattern such that the azimuth angle and elevation angle for viewing each feature point of the second actual drawing pattern from a specific viewpoint match the azimuth angle and elevation angle for viewing each feature point of the first reference drawing pattern corresponding to each feature point of the second actual drawing pattern from the specific viewpoint.

[0136] <E. Fifth Embodiment> The road surface drawing device 105 according to the fifth embodiment has the same configuration as that of the first to fourth embodiments as shown in FIG. 4. In this embodiment, the second reference drawing pattern is a pattern that is stereoscopically viewed from a specific viewpoint.

[0137] FIGS. 17 to 20 show examples of creating the second reference drawing pattern that is stereoscopically viewed from the viewpoint V1 which is a specific viewpoint.

[0138] FIG. 17 is a perspective view showing a rectangular parallelepiped that is arranged on the xy plane where z = 0 in the xyz coordinates and is stereoscopically viewed from the viewpoint V1. The light projection control unit 12 projects each vertex a, b, c, d, e, f, g, h of the rectangular parallelepiped onto the xy plane where z = 0 starting from the viewpoint V1 to obtain projection points a1, b1, c1, d1, e1, f1, g1, h1. The projection point a1 is the intersection point where the extension line of the straight line connecting the viewpoint V1 and the vertex a overlaps with the xy plane where z = 0. The other projection points are defined in the same manner. Note that in FIG. 17, only the projection points b1, c1, d1 that are not hidden by the rectangular parallelepiped and can be visually observed are shown.

[0139] FIG. 18 is a view of FIG. 17 seen from the y-axis direction.

[0140] FIG. 19 is a view of FIG. 17 seen from the z-axis direction.

[0141] FIG. 20 shows a figure drawn on the xy plane (z = 0) with the projection points a1, b1, c1, d1, e1, f1 as feature points. This figure is a reference drawing pattern that appears as a rectangular parallelepiped when viewed from the viewpoint V1 when drawn on the reference road surface.

[0142] Figures 21 to 24 show examples of creating a second reference drawing pattern that is stereoscopically viewed from the viewing point V2, which is a specific viewing point. When the specific viewing point changes from V1 to V2, the position of the projection point on the xy plane where z = 0 changes.

[0143] The light projection control unit 12 projects each vertex a, b, c, d, e, f, g, h of the rectangular parallelepiped onto the xy plane where z = 0 starting from the viewing point V1, and obtains the projection points a2, b2, c2, d2, e2, f2, g2, h2. In FIG. 21, only the projection points a2, c2, d2 that are not hidden by the rectangular parallelepiped and can be visually observed are shown.

[0144] FIG. 22 is a view of FIG. 21 seen from the y-axis direction.

[0145] FIG. 23 is a view of FIG. 21 seen from the z-axis direction.

[0146] FIG. 24 shows a figure drawn on the xy plane (z = 0) with the projection points a2, b2, c2, d2, e2, f2, g2, h2 as feature points. This figure is the second reference drawing pattern that appears as a rectangular parallelepiped when viewed from the viewing point V2 when drawn on the reference road surface.

[0147] That is, when the reference drawing pattern is deformed from the pattern shown in FIG. 20 to the pattern shown in FIG. 24, the same rectangular parallelepiped is visually recognized even when the viewing point moves from V1 to V2. The light projection control unit 12 uses the pattern shown in FIG. 20 or FIG. 24 as the reference drawing pattern for stereoscopic viewing according to the viewing point.

[0148] Note that a rectangular parallelepiped is used in the above description, but the three-dimensional object to be stereoscopically viewed does not have to be a rectangular parallelepiped. Also, a pattern or characters may be drawn on the surface of the three-dimensional object.

[0149] <F. Embodiment 6> The road surface drawing device 106 of Embodiment 6 has the same configuration as that of Embodiments 1-5 as shown in FIG. 4.

[0150] In Embodiments 1 to 5, the description has been given on the assumption that the first reference light projection pattern, which is a reference light projection pattern for calculating the correction amount of the light projection pattern according to the peripheral road surface 3, and the second reference light projection pattern, which is a reference light projection pattern to which the correction amount is applied, are the same.

[0151] However, in Embodiment 6, the first reference light projection pattern and the second reference light projection pattern are different patterns.

[0152] Specifically, the second reference drawing pattern is a desired pattern that the road surface drawing device 106 wants to draw on the peripheral road surface 3. And the second reference light projection pattern is a detailed pattern set by inverse calculation from the second reference drawing pattern which is the desired pattern.

[0153] On the other hand, the first reference light projection pattern and the first reference drawing pattern are simple patterns with few feature points such as a circle, a rectangle, and a triangle. By making the first reference light projection pattern a simple pattern, arithmetic processing such as area comparison, distance comparison between feature points, and viewpoint extraction becomes easy between the first actual drawing pattern and the first reference drawing pattern.

[0154] Also, since the first reference light projection pattern is only for calculating the correction amount, it does not need to be visually recognized by the driver 1 or the like. Therefore, the first reference light projection pattern may be a pattern of invisible light such as infrared light or ultraviolet light. That is, the light of the first reference light projection pattern may be invisible light, and the light of the second reference light projection pattern and the correction light projection pattern may be visible light.

[0155] <G. Embodiment 7> FIG. 25 is a block diagram showing the configuration of the road surface drawing device 107 of Embodiment 7 and its peripheral devices. The road surface drawing device 107 is obtained by adding an alarm control unit 13 to the configuration of the road surface drawing device 101 of Embodiment 1. Also, in the present embodiment, the vehicle M is provided with a front camera 21, a light projection unit 22, and an output device 23 in addition to these.

[0156] The output device 23 is an LED light, a display, or a speaker mounted on the vehicle M.

[0157] Figure 26 shows an actual drawing pattern with defects. An actual drawing pattern with defects is an actual drawing pattern that is missing feature points compared to the reference projection pattern. The actual drawing pattern in Figure 26 is an arrow-shaped actual drawing pattern that is originally composed of feature points a, b, c, d, e, f, and g as shown in Figure 9, but is missing feature points a, d, and e.

[0158] For example, when snow or mud adheres to the front of the headlight, which is the light projecting unit 22, a defect as shown in FIG. 26 occurs in the actual drawing pattern. Alternatively, when snow or mud adheres to the front of the front camera 21, no defect occurs in the actual drawing pattern drawn on the surrounding road surface 3, but a defect as shown in FIG. 26 occurs in the actual drawing pattern acquired by the image acquiring unit 11. When there is a defect in the actual drawing pattern acquired by the image acquiring unit 11, an area where the area cannot be calculated occurs in the area determination method of the first embodiment. Also, an area where the distance cannot be extracted occurs in the distance determination method of the second embodiment. Also, an abnormality such as a lack of feature points occurs in the method of the fourth embodiment.

[0159] Fig. 27 is a flowchart showing the operation of the road surface drawing device 107. The flow in Fig. 27 is obtained by adding step S103A and step S108 to the flow of the road surface drawing device 101 shown in Fig. 6.

[0160] When the image acquisition unit 11 acquires the actual drawing pattern in step S103, the light projection control unit 12 compares the actual drawing pattern with the reference drawing pattern in step S103A and determines whether the number of feature points in the actual drawing pattern is insufficient.

[0161] If the number of feature points of the actual drawing pattern is insufficient in step S103A, the warning control unit 13 determines that there is a defect in the actual drawing pattern and performs warning control in step S108. In this step, the warning control unit 13 causes the output device 23 to output a lighting abnormality warning. In response to this warning, the driver 1 stops the vehicle M, visually inspects the light projecting unit 22, and can clean it if there are attachments such as snow or mud. Also, if there are no attachments on the light projecting unit 22, the driver 1 can determine that there is an internal failure in the light projecting unit 22 and take appropriate measures.

[0162] Note that simultaneously with the output of the lighting abnormality warning, the warning control unit 13 may operate a wiper (not shown) attached to the front surface of the light projecting unit 22 to clean the front surface of the light projecting unit 22.

[0163] The road surface drawing device 107 of the seventh embodiment includes a warning control unit 13 that causes an alarm to be output to an output device mounted on a vehicle when the difference between the number of feature points of the first actual drawing pattern and the number of feature points of the first reference drawing pattern is equal to or greater than a predetermined threshold value. Thereby, the driver 1 of the vehicle M can notice a malfunction of the light projecting unit 22 or the front camera 21 and take necessary measures.

[0164] <H. Eighth Embodiment> The road surface drawing device 108 of the eighth embodiment has the same configuration as that of the first to sixth embodiments as shown in FIG. 4.

[0165] When the peripheral road surface 3 on which the actual drawing pattern is drawn slopes downward, compared with the case where it is horizontal, the specular reflection light component of the projection light of the light projecting unit 22 with respect to the peripheral road surface 3 increases, and the reflected light component to the vehicle M decreases. Therefore, the actual drawing pattern looks dark to the driver 1 or a pedestrian not in the traveling direction of the vehicle M, and the actual drawing pattern looks excessively bright to the pedestrian 7 in the traveling direction of the vehicle M.

[0166] Therefore, when the viewer of the actual drawing pattern is a pedestrian who is not the driver 1 or in the traveling direction of the vehicle M, the light projection control unit 12 increases the luminance of the corrected light projection pattern. Further, when the viewer of the drawing pattern is the pedestrian 7 in the traveling direction of the vehicle M, the light projection control unit 12 decreases the luminance of the corrected light projection pattern.

[0167] The light projection control unit 12 can calculate the gradient of the peripheral road surface 3 by comparing the actual drawing pattern with the reference drawing pattern. Further, when the actual drawing pattern is divided into a plurality of regions and compared with the reference drawing pattern as shown in FIG. 8 or FIG. 11, if the correction amount of the front divided region in the traveling direction of the vehicle M is larger than the correction amount of the published divided region, the gradient of the peripheral road surface 3 changes in the middle of the light projection range of the light projection unit 22. Therefore, the light projection control unit 12 increases the luminance of the front divided region in the traveling direction of the corrected light projection pattern compared to the luminance of the rear divided region.

[0168] In the road surface drawing device 108 of the eighth embodiment, when the downward gradient value of the peripheral road surface 3 exceeds a predetermined threshold value, the light projection control unit 12 increases the luminance of the light of the corrected light projection pattern more than when it does not exceed.

[0169] <I. Eighth Embodiment> The road surface drawing device 109 of the ninth embodiment has the same configuration as that of the first to sixth and eighth embodiments as shown in FIG. 4.

[0170] The x-y-z coordinate systems shown in FIGS. 12, 15, and 16 are centered on the vehicle body of the vehicle M, more specifically, the light sources of the light projection units 22L and 22R. In addition, there are a coordinate system xc-yc-zc recognized by the camera image of the front camera 21 centered on the front camera 21, a coordinate system xl-yl-zl of the light projection unit 22L, and a coordinate system xr-yr-zr of the light projection unit 22R. The coordinate systems xl-yl-zl and xr-yr-zr are determined by the light projection angles of the light projection units 22L and 22R. Since each coordinate system is independent, even if the coordinate values indicating a specific position in a certain coordinate system are input to another coordinate system, if the position or rotation of the origin of each coordinate system is unknown, it cannot be projected to the same specific position in another coordinate system.

[0171] Fig. 28 shows light projection points P1L and P1R, which are alignment patterns projected in the same direction from both light projector 22L and light projector 22R to project light onto the same point. Fig. 28 shows the positions on the xc-yc plane of light projection points P1L and P1R in an image captured by front camera 21. Light projection point P1L by light projector 22L, represented by a + mark, does not overlap with light projection point P1R by light projector 22R, represented by an x ​​mark.

[0172] The light projection control unit 12 of the road surface drawing device 109 uses the light projection points P1L and P1R as alignment patterns, and corrects the coordinate system xl-yl-zl of the light projection unit 22L and the coordinate system xr-yr-zr of the light projection unit 22R based on the deviation between the two. The correction is performed by the position of the origin of each coordinate system and rotation around each coordinate axis. In this case, the coordinate has six adjustment parameters, the x position, the y position, and the z position of the origin, and the x rotation, the y rotation, and the z rotation. It is not necessary to identify all the adjustment parameters, and only those that are likely to have a large error or are easy to identify may be identified. The direction of the z rotation is the same as θ expressed as the azimuth angle of the light projection direction of the light projection units 22L and 22R in the fourth embodiment. The direction of the y rotation is the same as φ expressed as the depression angle of the light projection direction of the light projection units 22L and 22R in the fourth embodiment.

[0173] A case will be described where the rotation of the depression angle direction of light projectors 22L, 22R in Fig. 12 and the y direction of origin O are corrected based on the deviation of P1L, P1R. In the xc-yc-zc coordinate system, the position coordinates of P1L, P1R in the x and y directions in the frame coordinate system can be known from the image captured by front camera 21. From these position coordinates, the direction angle and depression angle of P1L, P1R in the xc-yc-zc coordinate system can be calculated.

[0174] Both P1L and P1R are projected onto the S0 plane. The azimuth angle and zx visual depression angle of P1L as viewed from the front camera 21 are expressed as θ C (1,r) and φ C,x-z (1, r). The azimuth angle and zx viewing angle of P1R as seen from the front camera 21 are expressed as θC (1,l) and φ C,x-z Let's say it is (1,l).

[0175] The xc and yc coordinates are expressed by the following equations.

[0176]

number

[0177] Therefore, the xc coordinate xc(1,l) and yc coordinate yc(1,l) of P1L, and the xc coordinate xc(1,r) and xc coordinate yc(1,r) of P1R are expressed by the following equations. In many cases, P1R is located near P1L, so there is no major contradiction in assuming that P1L and P1R are on the same plane.

[0178]

number

[0179] First, the process of adjusting the deviation in the yc direction will be described. In this case, the deviation Δyc,(1) in the yc direction between P1L and P1R can be calculated by the following formula based on formulas (33) and (35).

[0180]

number

[0181] Even if the rotation of the coordinate system in the y direction is changed, the y direction projection position does not change. Therefore, the y direction position of the light projector 22R, which is the origin position of the coordinate system xr-yr-zr, is moved by -Δyc,(1). This causes the yc direction positions of P1R and P1L to match.

[0182] Next, we will explain the process of aligning the rotation direction of the y-axis. The deviation Δxc,(1) of P1L and P1R in the xc direction is expressed by the following formula.

[0183]

number

[0184] The xr coordinate in the coordinate system xr-yr-zr of the light projector 22R is expressed by the following equation.

[0185]

number

[0186] The amount of rotation when the light projecting unit 22R is rotated around the yc axis by the amount of correction for the deviation Δxc, (1) between P1L and P1R in the xc direction, i.e., the correction amount, is Δφ r (1), we obtain the following equation.

[0187]

number

[0188] Correction amount Δφ r (1) is determined by the following formula:

[0189]

number

[0190] When correction is made using the position hr on the yr axis instead of correction about the yr axis, equation (41) including the differentiation of xr with respect to hr is used instead of equation (40).

[0191]

number

[0192] In the above example, the adjustment of the positions of the light projectors 22L and 22R in the xc and yc directions has been described. That is, the identified parameters are two, the xc coordinate and the yc coordinate. However, the number of identified parameters may be greater. Since the image captured by the front camera 21 has information in the x and y directions, two parameters can be identified simultaneously using one point on the captured image. The number of parameters that can be identified simultaneously is twice the number of points used in the alignment pattern. If the alignment pattern has two points, four parameters can be identified simultaneously, and if it has three points, six parameters can be identified simultaneously.

[0193] To further improve accuracy, the light projection control unit 12 may perform the above correction once and then repeat the same correction again. In this way, correction is possible even if the road surface is not flat and level like S0. Correction is also possible even if the road surface shape changes during driving.

[0194] 29 shows lattice-shaped alignment patterns P1L, P2L, P3L, P4L, P5L, P6L, P7L, P8L, P9L, P10L, P11L, P12L and P1R, P2R, P3R, P4R, P5R, P6R, P7R, P8R, 9R, P10R, P11R, P12R. By using multiple alignment patterns in this way, it is possible to identify adjustment parameters at each location in the xc-yc coordinates and change the adjustment parameter values ​​depending on the projection position.

[0195] The front camera 21, which uses an image element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), captures an image one frame at a time for each frame rate. While projecting a drawing pattern onto the driver 1, the light projecting unit 22 may project an alignment pattern instantaneously in accordance with the timing at which the front camera 21 captures an image. For example, the light projecting unit 22L may project an alignment pattern in one frame, and the light projecting unit 22R may project an alignment pattern in another frame.

[0196] Alternatively, the light projection control unit 12 may embed the alignment pattern in the drawing pattern for the driver 1 or the like. In this case, the light projection units 22L and 22R may project light at different frame timings so that they can be distinguished from each other.

[0197] In the above description, the case where two light projection units, namely the light projection unit 22L and the light projection unit 22R, are used has been described. However, the light projection unit 22 may be composed of three or more light projection units.

[0198] In FIGS. 28 and 29, alignment patterns of simple shapes are shown, but the alignment pattern may be an image such as a figure or a photograph. Not only the position of the same point in the alignment pattern may be corrected, but also the positions of points having a specific relationship such as parallelism, rotation, and enlargement may be corrected. The irradiation direction or irradiation position of the light projection unit 22 may be mechanically changed and adjusted.

[0199] <J. Embodiment 10> The road surface drawing device 110 of Embodiment 10 has the same configuration as that of Embodiments 1-6, 8, and 9 as shown in FIG. 4.

[0200] In Embodiment 9, the correction of the coordinate system deviation due to the different positions of the light projection units 22L and 22R has been described. In Embodiment 10, the light projection control unit 12 corrects the coordinate system deviation due to the different positions of the plurality of front cameras 21B. The correction method is the same as that in Embodiment 9.

[0201] In FIG. 3, it is assumed that two front cameras 21B are installed at the same positions as the light projection units 22L and 22R. These two front cameras 21B photograph the same object on the surrounding road surface. The light projection control unit 12 determines the coordinates of the object in the coordinate system viewed from the origin of each front camera 21B from the camera frame images of the front cameras 21B, and corrects at least one of the parameters representing the positional or rotational relationship between the two front cameras 21B.

[0202] Here, the object on the peripheral road surface used for correction does not have to be the drawing pattern on the peripheral road surface by the light projecting unit 22. Instead of the front camera 21B, a position detection sensor such as Lidar or radar may be used.

[0203] <K. Embodiment 11> The road surface drawing device 111 of Embodiment 11 has the same configuration as that of Embodiments 1-6, 8-10 as shown in FIG. 4.

[0204] In FIG. 3, it is assumed that two front cameras 21B are installed at the same positions as the light projecting units 22L and 22R. Light is projected from the light projecting unit 22L at a certain azimuth and elevation angle, and a pattern is drawn on the road surface. At this time, the light projection control unit 12 can grasp the coordinates on the coordinate system with the left front camera 21B installed at the same position as the light projecting unit 22L for the light projection point on the road surface. Next, the right front camera 21B captures the pattern drawn on the road surface. Then, the light projection control unit 12 can grasp the coordinates of the light projection point in the coordinate system with the right front camera 21B as the origin by converting the frame coordinates of the captured image of the right front camera 21B into the coordinates of the projection position. The light projection control unit 12 can correct the deviation between the coordinate systems in the same manner as in Embodiment 9 based on the deviation between the coordinates on the coordinate system with the left front camera 21B as the origin and the coordinates on the coordinate system with the front camera 21B as the origin thus obtained.

[0205] Embodiments 10 and 11 may be combined. The light projection control unit 12 may simultaneously perform the correction described in Embodiment 10 and the correction described in Embodiment 11, and adjust the coordinate system of a sensor such as Lidar or radar that measures the distance to the object and the coordinate system of the light projecting unit 22. That is, the light projection control unit 12 may correct at least one of the positional relationship or the rotational relationship between the light projecting unit 22 and the front camera 21B.

[0206] <L. Hardware Configuration> The image acquisition unit 11, the light projection control unit 12, and the warning control unit 13 in the above-mentioned road surface drawing devices 101-111 are realized by a processing circuit 81 shown in Fig. 30. That is, the processing circuit 81 includes the image acquisition unit 11, the light projection control unit 12, and the warning control unit 13 (hereinafter referred to as "image acquisition unit 11, etc."). Dedicated hardware may be applied to the processing circuit 81, or a processor that executes a program stored in a memory may be applied. The processor is, for example, a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, a DSP (Digital Signal Processor), etc.

[0207] When the processing circuit 81 is a dedicated hardware, the processing circuit 81 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination of these. Each function of the image acquisition unit 11 and the like may be realized by a plurality of processing circuits 81, or the functions of each unit may be realized by a single processing circuit.

[0208] When the processing circuit 81 is a processor, the functions of the image acquisition unit 11 and the like are realized by a combination of software, etc. (software, firmware, or software and firmware). The software, etc. is written as a program and stored in a memory. As shown in FIG. 31, a processor 82 applied to the processing circuit 81 realizes the functions of each unit by reading and executing a program stored in a memory 83. That is, the road surface drawing device 101-108 has a memory 83 for storing a program which, when executed by the processing circuit 81, results in the execution of the following steps: acquiring photographed image information of the surrounding road surface; causing the light-projecting unit 22 to project light of a first reference light-projection pattern that draws a first reference drawing pattern on the reference road surface; extracting from the photographed image information the actual drawing pattern drawn on the surrounding road surface by the light of the first reference light-projection pattern; calculating a correction amount based on the difference between the first drawing pattern and the first reference drawing pattern; creating a corrected light-projection pattern by correcting, with the correction amount, the second reference light-projection pattern that draws a second reference drawing pattern that is the same as or different from the first reference drawing pattern on the reference road surface; and causing the light-projecting unit 22 to project the light of the corrected light-projection pattern.

[0209] In other words, this program can be said to cause a computer to execute the procedure or method of the image acquisition unit 11, etc. Here, the memory 83 may be, for example, a non-volatile or volatile semiconductor memory such as a random access memory (RAM), a read only memory (ROM), a flash memory, an erasable programmable read only memory (EPROM), or an electrically erasable programmable read only memory (EEPROM), a hard disk drive (HDD), a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, a digital versatile disk (DVD) and a drive device thereof, or any storage medium to be used in the future.

[0210] The above describes a configuration in which the functions of the image acquisition unit 11, etc. are realized either in hardware or software, etc. However, the present invention is not limited to this, and a configuration in which part of the image acquisition unit 11, etc. is realized by dedicated hardware and another part is realized by software, etc. For example, the function of the light projection control unit 12 can be realized by a processing circuit as dedicated hardware, and the other functions can be realized by the processing circuit 81 as the processor 82 reading and executing a program stored in the memory 83.

[0211] As described above, the processing circuitry can realize the above-mentioned functions by hardware, software, or a combination of these.

[0212] Furthermore, although the road surface drawing devices 101-108 have been described above as in-vehicle devices, they can also be applied to a system constructed as a system by appropriately combining a PND (Portable Navigation Device), a communication terminal (e.g., a mobile terminal such as a mobile phone, a smartphone, or a tablet), the functions of applications installed thereon, a server, etc. In this case, each function or each component of the road surface drawing devices 101-108 described above may be distributed and disposed in each device that constructs the system, or may be centralized and disposed in one of the devices.

[0213] In addition, each embodiment can be freely combined, and each embodiment can be appropriately modified or omitted. The above description is an example in all respects. It is understood that countless modified examples not illustrated can be assumed.

[0214] The origin of the coordinate system used for specifying the projection pattern position or for calculating the coordinates does not have to be the center in the width direction of the vehicle body. [Explanation of symbols]

[0215] 3 surrounding road surface, 11 image acquisition unit, 12 light projection control unit, 13 alarm control unit, 21, 21A, 21B front camera, 22, 22L, 22R light projection unit, 23 output device, 81 processing circuit, 82 processor, 83 memory, 101-111 road surface drawing device.

Claims

1. an image acquisition unit that acquires captured image information, which is information on a captured image of a road surface around the vehicle captured by at least one camera mounted on the vehicle; a light projection control unit that controls projection of light onto the surrounding road surface by at least one light projection unit mounted on the vehicle; The light projection control unit includes: A light of a first reference light projection pattern for drawing a first reference drawing pattern on a reference road surface is projected from the at least one light projection unit; A first actual drawing pattern drawn on the surrounding road surface by the light of the first reference light projection pattern is extracted from the captured image information; calculating a correction amount based on a difference between the first actual drawing pattern and the first reference drawing pattern; A second reference drawing pattern, which may be the same as or different from the first reference drawing pattern, is drawn on the reference road surface; and a corrected light projection pattern is created by correcting the second reference light projection pattern, which may be the same as or different from the first reference light projection pattern, with the correction amount; by causing the at least one light-projecting unit to project light of the corrected light-projection pattern onto the surrounding road surface, a second actual drawing pattern is drawn on the surrounding road surface; the light projection control unit creates the corrected light projection pattern when a difference in area between the first actual drawing pattern and the first reference drawing pattern is equal to or greater than a predetermined threshold value, or creates the corrected light projection pattern based on a comparison between the distances between the feature points of the first actual drawing pattern corresponding to the feature points of the first reference drawing pattern, which are acquired from the captured image information, when the captured image information includes information on distances between feature points in the captured image. Road surface drawing device.

2. The at least one light-projecting unit is a plurality of light-projecting units provided at a plurality of locations on the vehicle, The at least one camera is a plurality of cameras provided at a plurality of locations on the vehicle corresponding to the plurality of light projecting units.

2. The road surface drawing device according to claim 1.

3. the light projection control unit creates the corrected light projection pattern so that an azimuth angle and a depression angle when each feature point of the second actual drawing pattern is viewed from a specific viewpoint match an azimuth angle and a depression angle when each feature point of the first reference drawing pattern corresponding to each feature point of the second actual drawing pattern is viewed from the specific viewpoint.

2. The road surface drawing device according to claim 1.

4. The second reference drawing pattern is an image that is stereoscopically viewed from a specific viewpoint.

2. The road surface drawing device according to claim 1.

5. the light of the first reference light projection pattern is invisible light, the light of the second reference light projection pattern and the correction light projection pattern is visible light; 2. The road surface drawing device according to claim 1.

6. the first reference light projection pattern and the second reference light projection pattern are identical, the light-projection control unit sets the most recent corrected light-projection pattern as a new second reference light-projection pattern, and sets the surrounding road surface at the time of creating the most recent corrected light-projection pattern as a new reference road surface to create the new corrected light-projection pattern.

2. The road surface drawing device according to claim 1.

7. and a warning control unit that causes an output device mounted on the vehicle to output a warning when a difference between the number of feature points of the first actual drawing pattern and the number of feature points of the first reference drawing pattern is equal to or greater than a predetermined threshold.

2. The road surface drawing device according to claim 1.

8. the light projection control unit increases the luminance of the light of the corrected light projection pattern when the downward gradient value of the surrounding road surface exceeds a predetermined threshold value compared to when the downward gradient value does not exceed a predetermined threshold value.

2. The road surface drawing device according to claim 1.

9. the at least one light projecting unit is a plurality of light projecting units, The image acquisition unit acquires information on an image captured by the at least one camera of an alignment pattern that is drawn by projecting light from the multiple light projection units onto the surrounding road surface, the light projection control unit corrects at least one of a positional relationship or a rotational relationship between the plurality of light projection units based on information of a captured image of the alignment pattern; 2. The road surface drawing device according to claim 1.

10. the at least one camera is a plurality of cameras; The image acquisition unit acquires information on images of the same object on the surrounding road surface captured by the multiple cameras, The light projection control unit detects a value relating to a position of the same object from information of images of the same object on the surrounding road surface captured by the multiple cameras, and corrects at least one of a positional relationship or a rotational relationship between the multiple cameras based on the detected value.

2. The road surface drawing device according to claim 1.

11. The light projection control unit detects a value relating to a position of light projected by the at least one light projection unit onto the surrounding road surface from information of an image of the surrounding road surface captured by the at least one camera, and corrects at least one of a position or a rotation between the at least one light projection unit and the at least one camera based on the detected value relating to the position of the light.

2. The road surface drawing device according to claim 1.

12. An image acquisition unit acquires captured image information, which is information on a captured image of a road surface around the vehicle captured by at least one camera mounted on the vehicle; a light projection control unit that controls projection of light onto the surrounding road surface by at least one light projection unit mounted on the vehicle; the light projection control unit causes the at least one light projection unit to project light of a first reference light projection pattern for drawing a first reference drawing pattern on a reference road surface; The light projection control unit extracts a first actual drawing pattern drawn on the surrounding road surface by the light of the first reference light projection pattern from the captured image information, the light projection control unit calculates a correction amount based on a difference between the first actual drawing pattern and the first reference drawing pattern; the light projection control unit draws a second reference drawing pattern that is the same as or different from the first reference drawing pattern on the reference road surface, and creates a corrected light projection pattern by correcting the second reference light projection pattern that is the same as or different from the first reference light projection pattern with the correction amount; the light projection control unit causes the at least one light projection unit to project light of the corrected light projection pattern onto the surrounding road surface, thereby drawing a second actual drawing pattern on the surrounding road surface; the light projection control unit creates the corrected light projection pattern when a difference in area between the first actual drawing pattern and the first reference drawing pattern is equal to or greater than a predetermined threshold value, or creates the corrected light projection pattern based on a comparison between the distances between the feature points of the first actual drawing pattern corresponding to the feature points of the first reference drawing pattern, which are acquired from the captured image information, when the captured image information includes information on distances between feature points in the captured image. Road drawing method.

Images