Distance measurement support device, distance measurement system, and distance measurement support method

Abstract

To provide a distance measurement support device, distance measurement system, and distance measurement support method that enable more accurate distance measurement of landmarks.SOLUTION: A distance measurement support device according to the present embodiment comprises: a first image acquisition unit which acquires a first image generated by an imaging device of a vehicle which images the front environment including a landmark having a main surface on which multiple regions with different reflectivities are arranged; a landmark detection unit which detects the position of the landmark in the first image; a second image acquisition unit which acquires a second image generated by a distance measurement device that measures the distance to the front environment; and a distance calculation unit which specifies the regions corresponding to the multiple regions of the landmark included in the first image in the second image, calculates the distance between the vehicle and the multiple regions of the landmark on the basis of pixel values of the specified regions, and calculates a distance between the vehicle and the landmark on the basis of the distance calculated for each region.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] FIELD An embodiment of the present invention relates to a ranging assistance device, a ranging system, and a ranging assistance method. [Background technology]

[0002] When a train controlled by an autonomous driving system stops, one possible method is to measure the distance to a landmark at a specific location using a ranging device, such as a LiDAR (Light Detection and Ranging) device, and then control the train's speed using the brakes or other measures to stop the train at the specified location.

[0003] In order to stop a train accurately at a desired position, it is necessary to be able to measure the distance from the train to the landmark with high accuracy, from when the distance is large (when braking begins) to when it is small (just before stopping). Furthermore, since the braking distance of a train is longer than that of a car, it is necessary to start measuring the distance to the landmark from a greater distance.

[0004] Furthermore, when a human (driver) stops a train, whether or not the train can be stopped at the desired location depends largely on the driver's skill, so there is great significance in being able to accurately inform the driver of the distance from the train to a landmark in real time.

[0005] Here, if the reflectance of the target is constant, when the distance from the LiDAR device to the target is large, the intensity of the reflected laser light is small, and when the distance from the LiDAR device to the target is small, the intensity of the reflected laser light is large. In this case, if the light receiving sensitivity of the receiver is adjusted to match the former, the reflected laser light of the latter will be saturated and the distance cannot be measured correctly. On the other hand, if the light receiving sensitivity of the receiver is adjusted to match the latter, the reflected laser light of the former will be too small to be detected.

[0006] One approach to this problem is to change the laser transmission strength and light receiving sensitivity depending on the reflectivity and the distance from the LiDAR device to the target object. However, this method requires dynamic control of the laser and light receiver depending on landmarks and the environment, which makes the device more complex. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] JP 2008-276331 A Summary of the Invention [Problem to be solved by the invention]

[0008] The embodiments of the present invention provide a distance measurement assistance device, a distance measurement system, and a distance measurement assistance method that enable more accurate distance measurement of landmarks. [Means for solving the problem]

[0009] The ranging support device of this embodiment includes a first image acquisition unit that acquires a first image generated by an imaging device of a vehicle that images a forward environment including a landmark having a main surface on which multiple areas with different reflectivities are arranged, a landmark detection unit that detects the position of the landmark in the first image, a second image acquisition unit that acquires a second image generated by a ranging device that measures the distance to the forward environment, and a distance calculation unit that identifies areas in the second image that correspond to the multiple areas of the landmark included in the first image, calculates the distance between the multiple areas of the landmark and the vehicle based on the pixel values ​​of the identified areas, and calculates the distance between the vehicle and the landmark based on the distance calculated for each area. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing a distance measuring system according to a first embodiment. [Diagram 2]FIG. 2 shows the ranging system in operation. [Diagram 3] Front view of the landmark. [Figure 4] FIG. 4 is a diagram showing an example of a forward image. [Diagram 5] FIG. 4 is a diagram showing an example of a distance image. [Figure 6] FIG. 13 is a diagram showing another example of a forward image. [Figure 7] FIG. 11 is a diagram showing another example of a template image. [Figure 8] 11 is a flowchart illustrating an example of a process performed by a distance measurement support device. [Figure 9] 10A to 10C are diagrams illustrating another example of the operation of the landmark detection unit. [Figure 10] FIG. 11 is a block diagram showing a distance measuring system according to a second embodiment. [Figure 11] FIG. 2 is a block diagram showing an example of a hardware configuration according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0012] (First embodiment) 1 is a block diagram showing an example of a distance measuring system 1 according to the first embodiment. The distance measuring system 1 includes a landmark 2, a positioning system 3, a camera 4, a distance measuring device 5, and a distance measuring support device 6.

[0013] 2 is a diagram showing the ranging system 1 in operation. At least the camera 4 and the ranging device 5 of the ranging system 1 are mounted on a vehicle 10, more specifically, on the leading car of the vehicle 10. The "leading car" means the car at the very front in the traveling direction of the vehicle 10. The ranging system 1 is a system for measuring the distance from the vehicle 10 to a landmark 2. In this embodiment, the vehicle 10 is described as a train car running on a track T, but the vehicle 10 may be an automobile or the like.

[0014] The example in Fig. 2 shows a state when the vehicle 10 is approaching a station platform P. In the following description, the distance between the vehicle 10 and the landmark 2 will be described as the distance component parallel to the traveling direction (distance D shown in Fig. 2) of the Euclidean distance between the vehicle 10 and the landmark 2. If the vehicle 10 is a train vehicle, the distance component parallel to the traveling direction can also be expressed as the distance along the track T.

[0015] The landmark 2 is a mark that allows the driver or the automatic driving system controlling the vehicle 10 to determine the stopping position of the vehicle 10. In the example of Fig. 2, the landmark 2 is placed near the platform P of the station, but the landmark 2 may be placed inside the platform P. The landmark 2 may also be placed in a vehicle depot or the like.

[0016] FIG. 3 is a front view of the landmark 2. The landmark 2 has a main surface S. A plurality of regions having different reflectances are arranged on the main surface S. In the example of FIG. 3, the landmark 2 has a region A high and area A low and area A high is area A low Here, the reflectance is the reflectance at the wavelength of the laser light used by the distance measuring device 5.

[0017] Area A high Since the reflectance of the area A is high, it is expected that the distance measuring device 5 can receive a sufficient amount of reflected laser light and measure the distance with high accuracy even if the distance to the distance measuring device 5 is far. low Since the reflectance of the area A is low, it is expected that the reflected laser light can be received without being saturated even when the distance to the distance measuring device 5 is short, and distance measurement can be performed with high accuracy. high When the vehicle 10 is close to the landmark 2, the distance is measured mainly using the laser light reflected from the landmark 2 in the area A. low It is preferable to measure the distance using a laser beam reflected by the target object.

[0018] Furthermore, since the landmark 2 has a main surface S on which multiple areas having different reflectivities are arranged, a pattern is formed on the main surface S, making it easier for the landmark detection unit 65, described later, to detect the landmark 2 more accurately.

[0019] 3, the landmark 2 is square, but the landmark 2 may be any shape, such as circular or triangular. In addition, there may be three or more regions having different reflectances, and the arrangement (pattern) of the regions may be any arrangement.

[0020] The positioning system 3 is a system that measures the current position of the vehicle 10 in real time. The positioning system 3 is, for example, a global positioning system (GPS) or a satellite positioning, navigation and timing system (GNSS). Instead of or in addition to the positioning system 3, an inertial measurement unit may be used to measure the current position of the vehicle 10. The positioning system 3 may be a part of the ranging support device 6.

[0021] The camera 4 is an imaging device for capturing an image of the environment ahead of the vehicle 10. The camera 4 is, for example, a camera that captures an environment ahead of the vehicle 10 to generate a forward image, which is a first image. The camera 4 is, for example, an optical camera such as a CCD (Charge Coupled Device) camera or a CMOS (Complementary Metal Oxide Semiconductor) camera, an infrared camera (thermo camera), or a laser camera, or a combination thereof. When the camera 4 includes an infrared camera, it becomes easier to detect the landmark 2 even at night. When the camera 4 includes a laser camera, it is possible to capture an image of a distant environment more clearly. The forward image is, for example, a luminance image, and each pixel value is a luminance value.

[0022] Fig. 4 is a diagram showing an example of a forward image generated by the camera 4. The symbol H indicates the horizon. In the example of Fig. 4, a landmark 2 is captured in the forward image.

[0023] The distance measuring device 5 is a device or sensor for measuring the distance to the environment in front of the vehicle 10 and generating a second image including the measured distance in each pixel. More specifically, the distance measuring device 5 acquires a distance image including information on the distance between the vehicle 10 and various objects included in the environment in front of the vehicle 10. The distance image is also called a depth image, depth data, or distance data. The object includes the railroad track T and the ground on which the railroad track T is arranged. The distance between the object and the vehicle 10 is, strictly speaking, the distance between the object and the distance measuring device 5. Hereinafter, the distance measuring device 5 will be described as a LiDAR, but the distance measuring device 5 may be a radar. The wavelength of the laser used by the distance measuring device 5 may be, for example, an eye-safe band (for example, 1.5 μm to 2.6 μm).

[0024] The distance measuring device 5, for example, irradiates a laser pulse to scan a predetermined range of the forward environment, and receives the pulse reflected from an object in the forward environment to generate a distance image including distance information from the vehicle 10 to the object included in the forward environment.

[0025] FIG. 5 is a diagram showing an example of a distance image generated by the distance measuring device 5. The pixels of the distance image are shown as circles (dots). Although the field of view of the camera 4 and the field of view of the distance measuring device 5 are different, in the example of FIG. 5, it is assumed that a correction has already been made to make the two coincident. In other words, the correction has been made so that the distance measuring device 5 observes the forward environment from the same viewpoint and the same angle of view as the camera 4. In the example shown in FIG. 5, the distance image, which is three-dimensional information, has been converted into two-dimensional information in accordance with the forward image. The correction may be made by the distance image acquisition unit 66, which will be described later.

[0026] A pixel (dot) DT corresponds to a position (observation point) where one (one) reflected pulse is received. The distance image is expressed as a collection (point cloud) of multiple pixels DT. Each pixel DT has a pixel value that indicates the measured distance (referred to as the measured distance) from the train 10 (range measuring device 5).

[0027] In the example of FIG. 5, the intensity of the reflected pulse is represented by the brightness of the pixel DT, and the stronger the intensity of the reflected pulse, the brighter the color of the pixel DT. The farther away from the train 10, the weaker the intensity of the reflected pulse becomes. As will be described later, in the example of FIG. 5, within the range B corresponding to the position of the landmark 2, the area A high The pixel DT corresponding to high is relatively bright, and area A low The pixel DT corresponding to low is relatively dark.

[0028] The absence of pixels DT indicates that the reflected pulse has not been received, i.e., has not been observed. When the train 10 moves away from the distance measuring device 5 to a certain distance, the angle of incidence of the pulsed light emitted from the distance measuring device 5 with respect to the ground or the track T increases, and the reflected pulse does not return and is not observed in many cases.

[0029] 2, the camera 4 and the distance measuring device 5 are arranged outside the vehicle 10, but the camera 4 and the distance measuring device 5 may be arranged inside the vehicle 10. In that case, the camera 4 and the distance measuring device 5 may capture images and measure distances through the windshield of the vehicle 10. The distance measurement support device 6 may be mounted on the vehicle 10, or may be located in a place other than the vehicle 10, such as a control room that operates the vehicle 10.

[0030] 1, the distance measurement support device 6 is an information processing device including a positioning data acquisition unit 61, an approximate distance calculation unit 62, an image acquisition unit 64, a distance image acquisition unit 66, a landmark detection unit 65, and a distance calculation unit 67. The distance measurement support device 6 is a device for calculating a more accurate distance from the vehicle 10 to the landmark 2 based on a forward image acquired by the camera 4 and a distance image acquired by the distance measurement device 5.

[0031] The positioning data acquisition unit 61 acquires positioning data including the current position of the vehicle 10 from the positioning system 3. The positioning data acquisition unit 61 may continue to acquire the positioning data from the positioning system 3 in real time, or may acquire the positioning data when the vehicle 10 moves. For example, the amount of movement of the vehicle 10 may be measured by an inertial measurement unit mounted on the vehicle 10, and the positioning data may be acquired when the amount of movement reaches or exceeds a certain amount.

[0032] The approximate distance calculation unit 62 calculates the distance from the vehicle 10 to the landmark 2 based on the acquired positioning data and the position information of the landmark 2 stored in the landmark information storage unit 63. The distance calculated based on this positioning data is referred to as the approximate distance. Note that the positioning data may include the position information of the landmark 2. In that case, the approximate distance calculation unit 62 may calculate the approximate distance based on the current position of the vehicle 10 and the position of the landmark 2 included in the positioning data.

[0033] The landmark information storage unit 63 stores information about the landmarks 2. The information about the landmarks 2 includes, for example, the shape, size, pattern, height, and location of the landmarks 2 for each station. The height of the landmark 2 may be the height from the ground, or may be the altitude or sea level.

[0034] Furthermore, the approximate distance calculation unit 62 determines whether the calculated approximate distance is equal to or less than a first threshold value. The first threshold value corresponds to, for example, the maximum distance at which the landmark detection unit 65 described later can detect the landmark 2, or the maximum distance at which the camera 4 can capture an image of the landmark 2 with a resolution that allows the landmark 2 to be identified. Alternatively, the first threshold value may be, for example, the maximum distance at which the distance measuring device 5 can measure the distance to the landmark 2. In other words, the first threshold value is the distance from the vehicle 10 to the landmark 2 or a value correlated to the distance, at which the landmark 2 can be detected and / or the distance to the landmark 2 can be measured. The first threshold value will be described later.

[0035] The image acquisition unit 64 corresponds to a first image acquisition unit that acquires a forward image generated by the camera 4. The image acquisition unit 64 may continue to acquire a forward image from the camera 4 in real time, or may acquire a forward image when the vehicle 10 moves.

[0036] The landmark detection unit 65 detects the landmark 2 from the forward image. For example, the size (number of pixels) of the landmark 2 in the forward image is calculated from parameters related to the camera 4, the actual shape, size, and pattern of the landmark 2 obtained from the landmark information storage unit 63, the estimated distance to the landmark 2, and the like. The parameters related to the camera 4 include internal parameters or external parameters of the camera 4, or both of these. A technique such as semantic segmentation may be used to detect the landmark 2.

[0037] Then, the landmark detection unit 65 creates a template image of the landmark 2 with the calculated size of the landmark 2 based on information such as at least one of the shape and pattern of the landmark 2, and detects the position of the landmark 2 in the forward image using a template matching method or the like. The template image of the landmark 2 is an image showing the landmark 2 itself, for example, as shown in Fig. 4. Furthermore, the presence of a pattern on the main surface S of the landmark 2 allows for more accurate detection of the landmark 2.

[0038] When the camera 4 includes a laser camera, the forward image is acquired as a brightness image of reflected laser light, as shown in Fig. 6. The background of the landmark 2 (shaded area) is observed as having a uniform minimum brightness, since the reflected laser light does not return to the camera 4.

[0039] Then, the landmark detection unit 65 may create a template image including the background on either the top, bottom, left, or right of the landmark 2 or in all directions, as in the example shown in Fig. 7, and detect the landmark 2. In the example of Fig. 7, a template image including the background on the left and right of the landmark 2 is shown. The background is the lowest brightness (e.g., black) when there is no reflection. In this way, when the background of the landmark 2 is included in the template image, the boundary between the landmark 2 and the background of the landmark 2 becomes clear, and the position of the landmark 2 can be detected more efficiently.

[0040] Here, detection of the landmark 2 by the landmark detection unit 65 is started after it is determined that the estimated distance is equal to or less than the first threshold value. This is because it is not reasonable to start detection of the landmark 2 in a state in which detection of the landmark 2 and / or measurement of the distance to the landmark 2 is not possible.

[0041] For example, when it is determined that the approximate distance is equal to or less than the first threshold, the landmark detection unit 65 itself may start detecting the landmark 2, the image acquisition unit 64 may start acquiring a forward image from the camera 4, or the camera 4 may start acquiring a forward image. In any case, when the approximate distance calculation unit 62 determines that the approximate distance is equal to or less than the first threshold, the landmark detection unit 65 starts detecting the position of the landmark 2. In this embodiment, it is described that the image acquisition unit 64 starts acquiring a forward image when it is determined that the approximate distance is equal to or less than the first threshold.

[0042] Distance image acquisition unit 66 corresponds to a second image acquisition unit that acquires a distance image (see FIG. 5) generated by distance measuring device 5. Distance image acquisition unit 66 acquires a distance image at substantially the same time (substantially the same frame) as the forward image acquired by image acquisition unit 64. Distance image acquisition unit 66 may continue to acquire distance images from distance measuring device 5 in real time, or may acquire distance images when vehicle 10 moves. Note that, for the same reason as landmark detection unit 65, distance image acquisition unit 66 may also start acquiring distance images when it is determined that the estimated distance is equal to or less than the first threshold value.

[0043] The distance image includes the measured distance from the train 10 to each of a plurality of regions having different reflectances. The distance image acquisition unit 66 may convert the measured distance from the train 10 to an object included in the distance image from the Euclidean distance to a distance component in the traveling direction of the train 10.

[0044] The distance calculation unit 67 calculates the distance to the landmark 2 based on the measured distances for each of a plurality of regions having different reflectances on the main surface S of the landmark 2. The distance to the landmark 2 calculated at this time is referred to as a composite distance.

[0045] First, the distance calculation unit 67 determines, from the position of the detected landmark 2 in the forward image (see FIG. 4) and the distance image (see FIG. 5), that a pixel DT within a range B in the distance image is an observation point for the landmark 2. The distance calculation unit 67 then further determines the pixel DT high is area A high is the observation point corresponding to pixel DT low is area A low In other words, the distance calculation unit 67 identifies the measurement distance for each of a plurality of regions having different reflectances. The measurement distance for each region is, for example, the average of the measurement distances for the pixels DT corresponding to each region.

[0046] Then, the distance calculation unit 67 calculates the pixel DT high The distance measurement distance and pixel DT lowThe synthetic distance is calculated by synthesizing the distances measured by the distance measuring device 5. Since the synthetic distance is calculated using the distances measured by the distance measuring device 5, it is expected to have higher accuracy than the approximate distance.

[0047] An example of a method for calculating the composite distance will be described below using formula (1). The composite distance is calculated by combining the measured distances in each area of ​​the landmark 2. In the example of the landmark 2 shown in FIG. 3, as shown in formula (1), the area A high Distance measurement distance D high and area A low Distance measurement distance D low are combined using a combination coefficient K(D).

[0048] K(D) is a blending coefficient where 0≦K(D)≦1. K(D) is a function of the distance D, for example, a step function or a monotonically increasing function. The distance D is D high , D low , or the average of both. The distance D may be an estimate. That is, the distance D is D high , D low , has a value based on at least one of the estimated distances.

[0049]

number

[0050] The distance calculation unit 67 calculates the composite distance by weighting the measured distances of the respective regions according to the distance between the vehicle 10 and the landmark 2 and combining them as shown in Equation (1). Specifically, when the vehicle 10 is far from the landmark 2, the main high If the vehicle 10 is close to the landmark 2, the synthetic distance is calculated based on D low This makes it possible to calculate the composite distance with high accuracy, regardless of whether the distance from the vehicle 10 to the landmark 2 is long or short.

[0051] The same applies to the case where there are three or more regions having different reflectances on the main surface S of the landmark 2. The landmark 2 has regions A1, A2, . . . , A N (N≧3), and train 10 to area A n Distance measurement distance D n In this case, equation (1) is generalized as equation (2) below. K n (D) is The blending coefficient is TIFF2025019916000003.tif10170. K n (D) is a function of the distance D, for example, a pulse function or window function that has a value only in a specific distance range. The distance D is the distance measurement distance D n or an average value thereof. The distance D may be an approximate distance.

[0052]

number

[0053] In addition, when the distance calculation unit 67 calculates the composite distance, the distance calculation unit 67 outputs the composite distance. For example, the composite distance may be output to a display that is visible to the driver of the vehicle 10, or the composite distance may be output to an automatic driving system of the vehicle 10.

[0054] After calculating the composite distance, the distance calculation unit 67 determines whether the composite distance is equal to or less than a second threshold value. The second threshold value corresponds to, for example, the distance from the vehicle 10 to the landmark 2 when the vehicle 10 reaches the stop position, or the distance when the landmark 2 is outside the field of view of the distance measuring device 5.

[0055] <Flowchart> 8 is a flowchart illustrating an example of processing performed by the distance measurement support device 6. Hereinafter, the processing performed by the distance measurement support device 6 will be described with reference to FIG.

[0056] First, the positioning data acquisition unit 61 acquires positioning data including the current position of the vehicle 10 from the positioning system 3 (step S1).

[0057] Next, the approximate distance calculation unit 62 calculates the distance (approximate distance) between the vehicle 10 and the landmark 2 based on the positioning data and the position of the landmark 2 (step S2). The position of the landmark 2 is known and is stored in the landmark information storage unit 63.

[0058] Next, the approximate distance calculation unit 62 determines whether the calculated approximate distance is equal to or less than the first threshold (step S3). If the approximate distance calculation unit 62 determines that the approximate distance exceeds the first threshold (step S3: No), the process returns to step S1, and the positioning data acquisition unit 61 acquires the positioning data again.

[0059] If the approximate distance calculation unit 62 determines that the estimated distance is less than or equal to the first threshold (step S3: Yes), then, at approximately the same time (approximately the same frame), the image acquisition unit 64 starts acquiring a forward image from the camera 4, and the distance image acquisition unit 66 starts acquiring a distance image from the distance measuring device 5 (step S4).

[0060] Next, the landmark detection unit 65 detects the landmarks 2 based on the acquired forward image (step S5).

[0061] Next, the distance calculation unit 67 identifies the distance measurement for each area of ​​the landmark 2 having a different reflectance (step S6).

[0062] Next, the distance calculation unit 67 combines the multiple measured distances to calculate a combined distance (step S7).

[0063] Next, the distance calculation unit 67 outputs the composite distance (step S8).

[0064] Next, the distance calculation unit 67 determines whether the calculated composite distance is equal to or less than the second threshold (step S9). If it is determined that the composite distance exceeds the second threshold (step S9: No), the process returns to step S4.

[0065] When it is determined that the composite distance is equal to or less than the second threshold (step S9: Yes), the process ends. Each step may be performed in parallel with other steps. For example, while the approximate distance calculation unit 62 is calculating the approximate distance in step S2, the positioning data acquisition unit 61 may continue to acquire positioning data from the positioning system 3.

[0066] In this way, the processing of steps S1 to S2 roughly calculates the distance from the vehicle 10 to the landmark 2 using the positioning data, and when the estimated distance falls below the first threshold, the processing of steps S4 to S7 more accurately calculates the distance from the vehicle 10 to the landmark 2 using the distance image acquired by the distance measuring device 5.

[0067] As described above, in this embodiment, the distance from the vehicle 10 to the landmark 2 is calculated based on the measured distance obtained by measuring the distance for each region of the landmark 2 having a main surface on which a plurality of regions having different reflectances are arranged. The weight for each reflectance of the landmark 2 is changed according to the distance from the vehicle 10 to the landmark 2, and the composite distance is calculated. Therefore, it is possible to measure the distance to the landmark 2 more accurately in a wider measuring range. As a result, even if the vehicle 10 is a train or the like and the braking distance is relatively long, it is possible to measure the distance to the landmark 2 more accurately, so that, for example, when the automatic driving system stops the vehicle 10, more accurate control is possible. In addition, the presence of a pattern on the main surface of the landmark makes it possible to detect the landmark more accurately.

[0068] Furthermore, the distance measurement support device 6 changes the method of calculating the distance D between a method of calculating an approximate distance using positioning data and a method of calculating a composite distance using the distance image and the forward image, depending on whether the distance from the vehicle 10 to the landmark 2 is equal to or less than a first threshold value. This makes it possible to efficiently calculate the distance D.

[0069] (Another example 1) Fig. 9 is a diagram showing another example of the operation of the landmark detection unit 65. As shown in Fig. 9, the landmark detection unit 65 may delete a part of the front image and detect the landmark 2.

[0070] For example, it is assumed that the approximate distance from the vehicle 10 to the landmark 2 is known by the approximate distance calculation unit 62 to be 100 m. Next, a range L1 (e.g., a range of 20 m in the forward and backward directions from the landmark 2) where the distance from the landmark 2 is constant relative to the approximate distance is identified in the distance image based on pixel values ​​(distance values) contained in the distance image (see FIG. 5). The landmark 2 is expected to be located within this range. A range corresponding to the range identified in the distance image is identified in the forward image, and the landmark 2 is detected within the identified range. In other words, the range other than the identified range is excluded or deleted from the forward image, and the detection of the landmark 2 is started. This reduces the range to be detected for the landmark 2, making it possible to detect the landmark 2 more efficiently.

[0071] Furthermore, the method of deleting a part of the forward image is not limited to this. For example, it is assumed that it is known that the landmark 2 will not be located in the lower half of the forward image, at least when the vehicle 10 is closer than the stopping position, depending on the angle of view of the camera 4 and the height of the landmark 2. In that case, the lower part of the forward image may be specified as a range in which the landmark 2 is not located, and the specified range may be excluded from the range to be detected for the landmark 2 (the specified range may be deleted from the forward image).

[0072] Furthermore, other than the lower part of the forward image, for example, if it is known that the landmark 2 is located to the left of the tracks T, the right side of the forward image may be identified as a range in which the landmark 2 is not located and deleted. Alternatively, if it is known that the landmark 2 appears at approximately the same height as the horizon H in the forward image, the horizon H may be detected from the forward image, and the range other than the vicinity of the horizon H may be identified as a range in which the landmark 2 is not located and excluded from the range subject to detection of the landmark 2.

[0073] (Another example 2) The distance measurement assistance device 6 may be linked to a brake control system of the train. The brake control system may control the brakes of the vehicle according to the composite distance output by the distance measurement assistance device 6. The second threshold value may be a value corresponding to the stopping position of the train, and the brake control system may control the train to come to a complete stop when the composite distance reaches the second threshold value.

[0074] Second embodiment In the first embodiment, a forward image is acquired using the camera 4, and the position of the landmark 2 is detected from the forward image. However, the landmark 2 may be detected from a distance image (see FIG. 5) acquired by the distance measuring device 5. In the second embodiment, a case where the landmark 2 is detected from a distance image acquired by the distance measuring device 5 will be described.

[0075] Fig. 10 is a block diagram of a distance measuring system 1A according to the second embodiment. Elements having the same names or functions as those in Fig. 1 of the first embodiment described above are given the same reference numerals. Hereinafter, explanations will be omitted except for changes or additions.

[0076] In the distance measuring system 1A, the distance measuring device 5 also functions as the camera 4 in the first embodiment. Therefore, the camera 4 is omitted.

[0077] The image acquisition unit 64 acquires a distance image as shown in Fig. 5 in the first embodiment as a forward image. That is, in this embodiment, the forward image shows the intensity distribution of the reflected laser light emitted from the distance measuring device 5 and reflected by the landmark 2. Note that, in the first embodiment, the field of view of the camera 4 and the field of view of the distance measuring device 5 are different, but in this embodiment, the two already match, and no correction is required.

[0078] The landmark detection unit 65 detects the landmarks 2 in the same manner as in the first embodiment. Here, the template image is represented by a light intensity distribution.

[0079] As described above, in the second embodiment, the intensity distribution of the laser light emitted from the distance measuring device 5 and reflected by the landmark 2 is treated as a forward image. This eliminates the need to prepare a camera for capturing the forward image, making it possible to reduce the size and cost of the entire system.

[0080] (Hardware configuration) 7 shows the hardware configuration of an information processing device according to each embodiment. The information processing device is configured with a computer device 100. The computer device 100 includes a CPU 101, an input interface 102, a display device 103, a communication device 104, a main storage device 105, and an external storage device 606, which are interconnected by a bus 107.

[0081] The CPU (Central Processing Unit) 101 executes an information processing program, which is a computer program, on the main memory device 105. The information processing program is a program that realizes each of the above-mentioned functional configurations of the information processing device. The information processing program may be realized not as a single program, but as a combination of multiple programs and scripts. Each functional configuration is realized by the CPU 101 executing the information processing program.

[0082] The input interface 102 is a circuit for inputting operation signals from input devices such as a keyboard, a mouse, a touch panel, etc., to the information processing device. The input interface 102 corresponds to the input unit of the information processing device according to each embodiment.

[0083] The display device 103 displays data output from the information processing device. The display device 103 is, for example, but not limited to, an LCD (liquid crystal display), an organic electroluminescence display, a CRT (cathode ray tube), or a PDP (plasma display). Data output from the computer device 100 can be displayed on this display device 103. The display device 103 corresponds to an output unit of the information processing device according to each embodiment.

[0084] The communication device 104 is a circuit for the information processing device to communicate with an external device wirelessly or via a wire. Data can be input from the external device via the communication device 104. The data input from the external device can be stored in the main memory device 105 or the external memory device 606.

[0085] The main storage device 105 stores an information processing program, data required for executing the information processing program, data generated by executing the information processing program, and the like. The information processing program is deployed and executed on the main storage device 105. The main storage device 105 is, for example, a RAM, a DRAM, or an SRAM, but is not limited to these. Each storage unit or database of the information processing device according to each embodiment may be constructed on the main storage device 105.

[0086] The external storage device 606 stores an information processing program, data required for executing the information processing program, data generated by executing the information processing program, etc. These information processing programs and data are read into the main storage device 105 when the information processing program is executed. The external storage device 606 is, for example, a hard disk, an optical disk, a flash memory, and a magnetic tape, but is not limited to these. Each storage unit or database of the information processing device may be constructed on the external storage device 606.

[0087] The information processing program may be pre-installed in the computer device 100, or may be stored in a storage medium such as a CD-ROM. The information processing program may also be uploaded onto the Internet.

[0088] Furthermore, the information processing device may be configured as a single computer device 100, or may be configured as a system made up of a plurality of computer devices 100 connected to each other.

[0089] The present invention is not limited to the above-described embodiments as they are, and the components can be modified and embodied in the implementation stage without departing from the gist of the invention. Various inventions can be formed by appropriately combining multiple components disclosed in the above-described embodiments. For example, a configuration in which some components are deleted from all the components shown in each embodiment can also be considered. Furthermore, the components described in different embodiments can be appropriately combined.

[0090] This embodiment can also be configured as follows. [Item 1] A first image acquisition unit that acquires a first image generated by a camera of a vehicle that captures an image of a forward environment including a landmark having a main surface on which a plurality of regions having different reflectivities are arranged; a landmark detection unit that detects the position of the landmark in the first image; a second image acquisition unit that acquires a second image generated by a distance measuring device that measures a distance to the forward environment; a distance calculation unit that identifies areas in the second image that correspond to the plurality of areas of the landmark included in the first image, calculates a distance between the vehicle and each of the plurality of areas of the landmark based on pixel values ​​of the identified areas, and calculates a distance between the vehicle and the landmark based on the distance calculated for each of the areas; A ranging assistance device comprising: [Item 2] The distance calculation unit calculates the distance from the vehicle to the landmark by combining the distances for each of the regions. Item 1. A ranging assistance device. [Item 3] The distance calculation unit determines a weight of the distance for each of the regions according to one value of the distances for each of the regions, and combines the distances for each of the regions with the weight. Item 2. A ranging assistance device. [Item 4] a positioning data acquisition unit that acquires positioning data of the position of the vehicle from a positioning system; an approximate distance calculation unit that calculates an approximate distance between the vehicle and the landmark based on the positioning data and position information of the landmark; The distance calculation unit determines a weight of the distance for each of the regions based on the estimated distance, and combines the distance for each of the regions with the weight. Item 2. A distance measurement assistance device according to item 2 or 3. [Item 5] a positioning data acquisition unit that acquires positioning data including a position of the vehicle from a positioning system; an approximate distance calculation unit that calculates an approximate distance between the vehicle and the landmark based on the positioning data and position information of the landmark; The landmark detection unit starts detecting the position of the landmark in response to the estimated distance being equal to or less than a first threshold. A distance measurement support device according to any one of items 2 to 4. [Item 6] a positioning data acquisition unit that acquires positioning data including a position of the vehicle from a positioning system; an approximate distance calculation unit that calculates an approximate distance between the vehicle and the landmark based on the positioning data and position information of the landmark; The landmark detection unit determines the number of pixels of the landmark in the first image based on the information of the landmark and the estimated distance, creates a template image of the landmark based on the number of pixels, and detects the landmark by template matching using the first image and the template image. 6. A distance measurement support device according to any one of items 1 to 5. [Item 7] a positioning data acquisition unit that acquires positioning data including a position of the vehicle from a positioning system; an approximate distance calculation unit that calculates an approximate distance between the vehicle and the landmark based on the positioning data and position information of the landmark; The landmark detection unit specifies a certain distance range from the estimated distance in the second image, and detects the landmark by template matching using an image portion of the first image corresponding to the specified distance range and the template image. Item 6. A ranging assistance device according to item 6. [Item 8] The landmark detection unit creates a template image including the landmark and at least a part of a background surrounding the landmark, and detects the landmark by template matching using the first image and the template image. 8. A distance measurement assistance device according to item 6 or 7. [Item 9] The landmark detection unit specifies a range in the first image that does not include the landmark based on a relationship between the position of the camera and the position of the landmark, and detects the landmark from a range other than the specified range. A distance measurement support device according to any one of items 1 to 8. [Item 10] The vehicle is a train A distance measurement support device according to any one of items 1 to 9. [Item 11] A camera of a vehicle that captures an image of a forward environment including a landmark having a main surface on which a plurality of regions having different reflectivities are arranged; a distance measuring device for measuring a distance to the forward environment; a first image acquisition unit that acquires a first image generated by the camera; a landmark detection unit that detects the position of the landmark in the first image; a second image acquisition unit that acquires a second image generated by the distance measuring device; a distance calculation unit that identifies areas in the second image that correspond to the plurality of areas of the landmark included in the first image, calculates a distance between the vehicle and each of the plurality of areas of the landmark based on pixel values ​​of the identified areas, and calculates a distance between the vehicle and the landmark based on the distance calculated for each of the areas; A ranging system equipped with [Item 12] acquiring a first image generated by a camera of the vehicle capturing an environment ahead including a landmark having a major surface with a plurality of regions having different reflectivities disposed thereon; Detecting the location of the landmark in the first image; acquiring a second image generated by a range finder device that measures a distance to the forward environment; Identifying areas in the second image that correspond to the plurality of areas of the landmark included in the first image; Calculating distances between the vehicle and each of the plurality of landmark regions based on pixel values ​​of the identified regions; Calculating the distance between the vehicle and the landmark based on the distance calculated for each of the regions. Ranging support method. [Explanation of symbols]

[0091] 1. 1A Distance Measuring System 2. Landmarks 3. Positioning System 4. Camera (imaging device) 5 Ranging device 6 Ranging support device 61 Positioning data acquisition unit 62 Approximate distance calculation section 63 Landmark information storage unit 64 Image acquisition unit (first image acquisition unit) 65 Landmark detection unit 66 Distance image acquisition unit (second image acquisition unit) 67 Distance Calculation Unit 10. Train 100 Computer equipment 101 CPU 102 Input Interface 103 Display device 104 Communication equipment 105 Main memory 106 External storage device 107 Bus Ahigh region A low region B Range DT Pixels H horizon K Composition coefficient P Home R Rail S Main surface

Claims

1. a first image acquisition unit that acquires a first image generated by an imaging device of a vehicle that images an environment ahead including a landmark having a main surface on which a plurality of regions having different reflectivities are arranged; a landmark detection unit that detects the position of the landmark in the first image; a second image acquisition unit that acquires a second image generated by a distance measuring device that measures a distance to the forward environment; a distance calculation unit that specifies, in the second image, areas corresponding to the plurality of areas of the landmark included in the first image, calculates a distance between the vehicle and each of the plurality of areas of the landmark based on pixel values ​​of the specified areas, and calculates a distance between the vehicle and the landmark based on the distance calculated for each of the areas; A ranging assistance device comprising:

2. The distance calculation unit calculates the distance from the vehicle to the landmark by combining the distances for each of the regions.

2. A distance measurement support device according to claim 1.

3. The distance calculation unit determines a weight of the distance for each of the regions according to one value of the distances for each of the regions or an average of the distances, and combines the distances for each of the regions with the weight.

3. A distance measurement assistance device according to claim 2.

4. a positioning data acquisition unit that acquires positioning data of the position of the vehicle from a positioning system; an approximate distance calculation unit that calculates an approximate distance between the vehicle and the landmark based on the positioning data and position information of the landmark, The distance calculation unit determines a weight of the distance for each of the regions based on the estimated distance, and combines the distance for each of the regions with the weight.

3. A distance measurement assistance device according to claim 2.

5. a positioning data acquisition unit that acquires positioning data including a position of the vehicle from a positioning system; an approximate distance calculation unit that calculates an approximate distance between the vehicle and the landmark based on the positioning data and position information of the landmark, The landmark detection unit starts detecting the position of the landmark in response to the estimated distance being equal to or less than a first threshold.

3. A distance measurement assistance device according to claim 2.

6. a positioning data acquisition unit that acquires positioning data including a position of the vehicle from a positioning system; an approximate distance calculation unit that calculates an approximate distance between the vehicle and the landmark based on the positioning data and position information of the landmark, The landmark detection unit determines the number of pixels of the landmark in the first image based on the information of the landmark and the estimated distance, creates a template image of the landmark based on the number of pixels, and detects the landmark by template matching using the first image and the template image.

2. A distance measurement support device according to claim 1.

7. a positioning data acquisition unit that acquires positioning data including a position of the vehicle from a positioning system; an approximate distance calculation unit that calculates an approximate distance between the vehicle and the landmark based on the positioning data and position information of the landmark, The landmark detection unit specifies a certain distance range from the estimated distance in the second image, and detects the landmark by template matching using an image portion of the first image corresponding to the specified distance range and the template image.

7. A distance measurement assistance device according to claim 6.

8. The landmark detection unit creates a template image including the landmark and at least a part of a background surrounding the landmark, and detects the landmark by template matching using the first image and the template image.

7. A distance measurement assistance device according to claim 6.

9. The landmark detection unit specifies a range in the first image that does not include the landmark from a relationship between a position of the imaging device and a position of the landmark, and detects the landmark from a range other than the specified range.

2. A distance measurement support device according to claim 1.

10. The vehicle is a train 2. A distance measurement support device according to claim 1.

11. An imaging device for a vehicle that images an environment ahead including a landmark having a main surface on which a plurality of regions having different reflectivities are arranged; a distance measuring device for measuring a distance to the forward environment; a first image acquisition unit that acquires a first image generated by the imaging device; a landmark detection unit that detects the position of the landmark in the first image; a second image acquisition unit that acquires a second image generated by the distance measuring device; a distance calculation unit that identifies areas in the second image that correspond to the plurality of areas of the landmark included in the first image, calculates a distance between the vehicle and each of the plurality of areas of the landmark based on pixel values ​​of the identified areas, and calculates a distance between the vehicle and the landmark based on the distance calculated for each of the areas; A ranging system equipped with

12. acquiring a first image generated by an imaging device of the vehicle imaging an environment ahead including a landmark having a major surface on which a plurality of regions having different reflectivities are disposed; Detecting the location of the landmark in the first image; acquiring a second image generated by a range finder device that measures a distance to the forward environment; Identifying areas in the second image that correspond to the plurality of areas of the landmark included in the first image; Calculating distances between the vehicle and each of the plurality of landmark regions based on pixel values ​​of the identified regions; Calculating the distance between the vehicle and the landmark based on the distance calculated for each of the regions. Ranging support method.

Images