Distance measuring support device, distance measuring system, and distance measuring support method

Abstract

To provide a distance measurement support device, distance measurement system, and distance measurement support method that enable more accurate distance measurement of landmarks.SOLUTION: A distance measurement support device according to the present embodiment comprises: a first image acquisition unit which acquires a first image generated by an imaging device of a vehicle which images the front environment including a landmark having a main surface on which multiple regions with different reflectivities are arranged; a landmark detection unit which detects the position of the landmark in the first image; a second image acquisition unit which acquires a second image generated by a distance measurement device that measures the distance to the front environment; and a distance calculation unit which specifies the regions corresponding to the multiple regions of the landmark included in the first image in the second image, calculates the distance between the vehicle and the multiple regions of the landmark on the basis of pixel values of the specified regions, and calculates a distance between the vehicle and the landmark on the basis of the distance calculated for each region.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] Embodiments of the present invention relate to a distance measurement support device, a distance measurement system, and a distance measurement support method. [Background technology]

[0002] When a train controlled by an automated driving system comes to a stop, one possible method is to control the train's speed using brakes or other means while measuring the distance to a landmark at a predetermined location using a rangefinder, such as a LiDAR (Light Detection and Ranging) device, in order to stop at the designated position.

[0003] To stop a train precisely at the desired location, it is necessary to be able to accurately measure the distance from the train to the landmark, from when the distance is large (at the start of braking) to when it is small (just before stopping). Furthermore, because trains have a longer braking distance than automobiles, it is necessary to start measuring the distance to the landmark from a greater distance.

[0004] Furthermore, when a human (driver) stops a train, whether or not they can stop the train at the desired location largely depends on the driver's skill. Therefore, it is highly significant to notify the driver of the distance from the train to the landmark in real time and with accuracy.

[0005] Here, assuming the reflectivity of the object is constant, the intensity of the reflected laser light decreases when the distance from the LiDAR device to the object is large, and increases when the distance from the LiDAR device to the object is small. If the light-receiving sensitivity of the photodetector is adjusted to match the former, the reflected laser light will saturate, making accurate distance measurement impossible. On the other hand, if the light-receiving sensitivity of the photodetector is adjusted to match the latter, the reflected laser light will be too weak and undetectable.

[0006] In response to this, one possible approach is to vary the laser transmission intensity and reception sensitivity according to the reflectivity and the distance from the LiDAR device to the target object. However, this method requires dynamically controlling the laser and receiver according to the landmark and environment, making the device complex. [Prior art documents] [Patent Documents]

[0007] [Patent Document 1] Japanese Patent Publication No. 2008-276331 [Overview of the project] [Problems that the invention aims to solve]

[0008] Embodiments of the present invention provide a distance measurement support device, a distance measurement system, and a distance measurement support method that enable more accurate distance measurement of landmarks. [Means for solving the problem]

[0009] The distance measuring support device according to this embodiment includes: a first image acquisition unit that acquires a first image generated by a vehicle imaging device that images the forward environment including a landmark having a main surface in which a plurality of regions having different reflectances are arranged; a landmark detection unit that detects the position of the landmark in the first image; a second image acquisition unit that acquires a second image generated by a distance measuring device that measures the distance to the forward environment; and a distance calculation unit that, in the second image, identifies regions corresponding to the plurality of regions of the landmark included in the first image, calculates the distance between the plurality of regions of the landmark and the vehicle based on the pixel values ​​of the identified regions, and calculates the distance between the vehicle and the landmark based on the distance calculated for each region. [Brief explanation of the drawing]

[0010] [Figure 1] A block diagram showing a distance measuring system according to the first embodiment. [Figure 2]Figure showing a distance measurement system in operation. [Figure 3] Front view of the landmark. [Figure 4] Figure showing an example of a front image. [Figure 5] Figure showing an example of a distance image. [Figure 6] Figure showing another example of a front image. [Figure 7] Figure showing another example of a template image. [Figure 8] Flowchart explaining an example of the process performed by the distance measurement support device. [Figure 9] Figure explaining another operation example of the landmark detection unit. [Figure 10] Block diagram showing the distance measurement system according to the second embodiment. [Figure 11] Block diagram showing an example of the hardware configuration in an embodiment of the present invention.

Mode for Carrying Out the Invention

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0012] (First Embodiment) FIG. 1 is a block diagram showing an example of a distance measurement system 1 according to the first embodiment. The distance measurement system 1 includes a landmark 2, a positioning system 3, a camera 4, a distance measurement device 5, and a distance measurement support device 6.

[0013] FIG. 2 is a diagram showing the distance measurement system 1 in operation. At least the camera 4 and the distance measurement device 5 of the distance measurement system 1 are mounted on the vehicle 10, more specifically, on the leading vehicle of the vehicle 10. The "leading vehicle" means the vehicle at the front in the traveling direction of the vehicle 10. The distance measurement system 1 is a system for measuring the distance from the vehicle 10 to the landmark 2. In this embodiment, the vehicle 10 will be described as being a vehicle of a train traveling on the track T, but the vehicle 10 may be an automobile or the like.

[0014] In the example shown in Figure 2, vehicle 10 is approaching station platform P. Hereafter, the distance between vehicle 10 and landmark 2 will be defined as the Euclidean distance between vehicle 10 and landmark 2 that is parallel to the direction of travel (distance D shown in Figure 2). If vehicle 10 is a train car, the distance parallel to the direction of travel can also be expressed as the distance along the track T.

[0015] Landmark 2 is a marker used by the driver or automated driving system controlling the vehicle 10 to determine the vehicle's stopping position, etc. In the example in Figure 2, landmark 2 is located near the station platform P, but landmark 2 may also be located within platform P. Alternatively, landmark 2 may be located in a train depot or similar location.

[0016] Figure 3 is a front view of landmark 2. Landmark 2 has a main surface S. Multiple regions with different reflectances are arranged on the main surface S. In the example in Figure 3, region A is on the main surface S of landmark 2. high and region A low It has, and area A high is region A low It has a higher reflectivity than [another component]. Here, reflectivity refers to the reflectivity at the wavelength of the laser light used by the distance measuring device 5.

[0017] Area A high Because of its high reflectivity, even when the distance to the distance measuring device 5 is great, the distance measuring device 5 is expected to be able to receive a sufficient amount of reflected laser light and measure the distance accurately. On the other hand, region A low Because of its low reflectivity, even when the distance to the distance measuring device 5 is short, the reflected laser light can be received without saturation, and it is expected that accurate distance measurement will be possible. In other words, when measuring the distance from vehicle 10 to landmark 2, if the distance from vehicle 10 to landmark 2 is far, mainly area A high It is preferable to measure the distance using the laser light reflected from the vehicle, and when the distance from vehicle 10 to landmark 2 is short, mainly area A low It is preferable to measure the distance using the laser light reflected from the device.

[0018] Furthermore, since landmark 2 has a main surface S on which multiple regions with different reflectivity are arranged, a pattern is formed on the main surface S, making it easier for landmark 2 to be detected more accurately by the landmark detection unit 65, which will be described later.

[0019] In the example shown in Figure 3, landmark 2 is rectangular, but the shape of landmark 2 can be any shape, such as a circle or a triangle. Furthermore, there may be three or more regions with different reflectivity, and their arrangement (pattern) can be any arrangement.

[0020] The positioning system 3 is a system that measures the current position of the vehicle 10 in real time. The positioning system 3 is, for example, GPS (Global Positioning System), GNSS (Satellite positioning, navigation and timing system), etc. Instead of the positioning system 3, or in addition to the positioning system 3, an inertial measurement unit may be used to measure the current position of the vehicle 10. The positioning system 3 may also be part of the distance measurement support device 6.

[0021] Camera 4 is an imaging device for imaging the environment in front of vehicle 10. Camera 4 is a camera that, for example, images the environment in front of vehicle 10 and generates a forward image, which is the first image. Camera 4 is, for example, an optical camera such as a CCD (Charge Coupled Device) camera or a CMOS (Complementary Metal Oxide Semiconductor) camera, an infrared camera (thermal camera), or a laser camera, or a combination thereof. If camera 4 includes an infrared camera, landmark 2 can be detected more easily even at night. If camera 4 includes a laser camera, distant environments can be imaged more clearly. The forward image is, for example, a luminance image, and each pixel value is a luminance value.

[0022] Figure 4 shows an example of a forward-facing image generated by camera 4. The symbol H indicates the horizon. In the example in Figure 4, landmark 2 is visible in the forward-facing image.

[0023] The ranging device 5 is a device or sensor for measuring the distance of the vehicle 10 to the environment in front of it and generating a second image in which the measured distance is included in each pixel. More specifically, the ranging device 5 acquires a distance image that includes distance information between the vehicle 10 and various objects in the environment in front of the vehicle 10. The distance image is also called a depth image, depth data, or distance data. The objects include the railway tracks T and the ground on which the railway tracks T are located. The distance between the objects and the vehicle 10 is, strictly speaking, the distance between the objects and the ranging device 5. Hereinafter, the ranging device 5 will be described as a LiDAR, but the ranging device 5 may also be a radar. The wavelength of the laser used by the ranging device 5 may be, for example, in the eye-safe band (e.g., 1.5 μm to 2.6 μm).

[0024] The distance measuring device 5, for example, irradiates a laser pulse and scans within a predetermined range of the environment ahead. By receiving pulses reflected back from objects in the environment ahead, it generates a distance image that includes distance information from the vehicle 10 to the objects in the environment ahead.

[0025] Figure 5 shows an example of a distance image generated by the rangefinder 5. Pixels in the distance image are schematically represented by circles (dots). Although the field of view of the camera 4 and the field of view of the rangefinder 5 are different, in the example of Figure 5, it is assumed that a correction has already been performed to make them match. In other words, the rangefinder 5 is corrected to appear as if it observed the forward environment with the same viewpoint and angle of view as the camera 4. Also, in the example shown in Figure 5, the 3D information of the distance image has been converted to 2D information to match the forward image. Note that the distance image acquisition unit 66, which will be described later, may also perform this correction.

[0026] The pixel (dot) DT corresponds to the position (observation point) where one (once) reflected pulse is received. The distance image is expressed as a set (point cloud) of a plurality of pixels DT. Each pixel DT has a pixel value representing the measured distance value (referred to as the measured distance) from the train 10 (distance measuring device 5).

[0027] Also, in the example of FIG. 5, the intensity of the reflected pulse is represented by the brightness and darkness of the pixel DT, and the greater the intensity of the reflected pulse, the brighter the color of the pixel DT is shown. The intensity of the reflected pulse decreases as the distance from the train 10 increases. Also, as will be described later, in the example of FIG. 5, among the range B corresponding to the position of the landmark 2, the region A high The corresponding pixel DT high is relatively bright, and it can be seen that the pixel DT low corresponding to the region A low is relatively dark.

[0028] The part where there is no pixel DT indicates that the reflected pulse has not been received, that is, it cannot be observed. When the distance from the train 10 is a certain amount, the incident angle of the pulsed light emitted from the distance measuring device 5 with respect to the ground or the track T becomes large, and the reflected pulse does not return and the number of cases where it is not observed increases.

[0029] In FIG. 2, the camera 4 and the distance measuring device 5 are arranged outside the vehicle 10, but the camera 4 and the distance measuring device 5 may be arranged inside the vehicle 10. In that case, the camera 4 and the distance measuring device 5 may perform imaging and distance measurement through the front glass of the vehicle 10, respectively. Also, the distance measurement support device 6 may be mounted on the vehicle 10, or may be in a location separate from the vehicle 10, such as a command room that operates the vehicle 10.

[0030] As shown in Figure 1, the distance measurement support device 6 is an information processing device comprising a positioning data acquisition unit 61, an approximate distance calculation unit 62, an image acquisition unit 64, a distance image acquisition unit 66, a landmark detection unit 65, and a distance calculation unit 67. The distance measurement support device 6 is a device for calculating a more accurate distance from the vehicle 10 to the landmark 2 based on the forward image acquired by the camera 4 and the distance image acquired by the distance measurement device 5.

[0031] The positioning data acquisition unit 61 acquires positioning data, including the current position of the vehicle 10, from the positioning system 3. The positioning data acquisition unit 61 may continuously acquire positioning data from the positioning system 3 in real time, or it may acquire positioning data when the vehicle 10 moves. For example, the amount of movement of the vehicle 10 may be measured by an inertial measurement unit mounted on the vehicle 10, and positioning data may be acquired when the amount of movement exceeds a certain amount.

[0032] The approximate distance calculation unit 62 calculates the distance from vehicle 10 to landmark 2 based on the acquired positioning data and the location information of landmark 2 stored in the landmark information storage unit 63. The distance calculated based on this positioning data is called the approximate distance. The positioning data may also include the location information of landmark 2. In that case, the approximate distance calculation unit 62 may calculate the approximate distance based on the current position of vehicle 10 and the position of landmark 2 included in the positioning data.

[0033] The landmark information storage unit 63 stores information about landmark 2. This information includes, for example, the shape, size, pattern, height, and location of landmark 2 at each station. The height of landmark 2 may be the height from the ground, or it may be the elevation or sea level.

[0034] Furthermore, the approximate distance calculation unit 62 determines whether the calculated approximate distance is less than or equal to the first threshold. The first threshold corresponds, for example, to the maximum distance at which the landmark detection unit 65 (described later) can detect landmark 2, or to the maximum distance at which the camera 4 can image landmark 2 with a resolution that allows it to identify landmark 2. Alternatively, the first threshold may be, for example, the maximum distance at which the distance measuring device 5 can measure landmark 2. In other words, the first threshold is the distance from the vehicle 10 to landmark 2, or a value correlated with that distance, at which detection of landmark 2 and / or measurement to landmark 2 becomes possible. The first threshold will be described later.

[0035] The image acquisition unit 64 corresponds to the first image acquisition unit that acquires the forward image generated by the camera 4. The image acquisition unit 64 may continuously acquire the forward image from the camera 4 in real time, or it may acquire the forward image when the vehicle 10 moves.

[0036] The landmark detection unit 65 detects landmark 2 from the forward image. For example, it calculates the size (number of pixels) of landmark 2 in the forward image from parameters related to the camera 4, the actual shape, size, and pattern of landmark 2 obtained from the landmark information storage unit 63, and the approximate distance to landmark 2. The parameters related to the camera 4 include intrinsic parameters, extrinsic parameters, or both of these. Techniques such as semantic segmentation may be used to detect landmark 2.

[0037] The landmark detection unit 65 then creates a template image of landmark 2 based on information such as the shape and pattern of landmark 2, using the calculated size of landmark 2, and detects the position of landmark 2 in the forward image using a template matching method or the like. The template image of landmark 2 is an image that shows landmark 2 itself, as shown in Figure 4, for example. Furthermore, the presence of a pattern on the main surface S of landmark 2 allows for more accurate detection of landmark 2.

[0038] If camera 4 includes a laser camera, the forward image is acquired as a brightness image of reflected laser light, as shown in Figure 6. The background of landmark 2 (shaded area) is observed as having a uniformly minimum brightness because the reflected laser light does not return to camera 4.

[0039] The landmark detection unit 65 may then create a template image that includes the background in any direction (top, bottom, left, right, or all directions) of landmark 2, as shown in the example in Figure 7, and perform landmark 2 detection. In the example in Figure 7, a template image including the background on the left and right of landmark 2 is shown. The lowest brightness value (e.g., black) without reflection is used as the background. By including the background of landmark 2 in the template image in this way, the boundary between landmark 2 and the background of landmark 2 becomes clearer, and the position of landmark 2 can be detected more efficiently.

[0040] Here, the detection of landmark 2 by the landmark detection unit 65 is started after it is determined that the approximate distance is less than or equal to the first threshold. This is because it would not be reasonable to start the detection of landmark 2 when it is not possible to detect landmark 2 and / or measure the distance to landmark 2.

[0041] For example, if the estimated distance is determined to be less than or equal to the first threshold, the landmark detection unit 65 itself may start detecting landmark 2, or the image acquisition unit 64 may start acquiring a forward image from the camera 4, or the camera 4 may start acquiring a forward image. In any case, if the estimated distance calculation unit 62 determines that the estimated distance is less than or equal to the first threshold, the landmark detection unit 65 will start detecting the position of landmark 2. In this embodiment, it will be explained that the image acquisition unit 64 starts acquiring a forward image when the estimated distance is determined to be less than or equal to the first threshold.

[0042] The distance image acquisition unit 66 corresponds to the second image acquisition unit, which acquires the distance image (see Figure 5) generated by the distance measuring device 5. The distance image acquisition unit 66 acquires the distance image at approximately the same time (approximately the same frame) as the forward image acquired by the image acquisition unit 64. The distance image acquisition unit 66 may continue to acquire distance images from the distance measuring device 5 in real time, or it may acquire distance images when the vehicle 10 moves. For the same reasons as the landmark detection unit 65, the distance image acquisition unit 66 may also start acquiring distance images when it is determined that the approximate distance is below the first threshold.

[0043] The distance image includes the measured distance for each region from the train 10 to multiple regions having different reflectivity. The distance image acquisition unit 66 may also convert the measured distance from the train 10 to the object included in the distance image from Euclidean distance to the distance component in the direction of travel of the train 10.

[0044] The distance calculation unit 67 calculates the distance to landmark 2 based on the measured distances for each of several regions on the main surface S of landmark 2 that have different reflectances. The distance to landmark 2 calculated at this time is called the combined distance.

[0045] First, the distance calculation unit 67 identifies the position of the detected landmark 2 in the forward image (see Figure 4) and the distance image (see Figure 5) to determine that pixel DT within range B in the distance image is the observation point for landmark 2. Then, the distance calculation unit 67 further determines that pixel DT high is area A high This is the observation point corresponding to pixel DT. low is area A low The unit identifies the corresponding observation point. In other words, the distance calculation unit 67 identifies the distance measurement distance for each of several regions having different reflectances. The distance measurement distance for each region is, for example, the average of the distance measurement distances of the pixels DT corresponding to each region.

[0046] Then, the distance calculation unit 67 calculates the pixel DT high The distance measurement range and pixel DT lowThe distances measured by each device are combined to calculate the combined distance. Since the combined distance is calculated using the distance measured by the distance measuring device 5, it is expected to be more accurate than the approximate distance.

[0047] Below, an example of how to calculate the combined distance is explained using equation (1). The combined distance is calculated by combining the distances measured in each region of landmark 2. In the example of landmark 2 shown in Figure 3, as shown in equation (1), region A high Distance measurement D high and region A low Distance measurement D low These are combined using the synthesis coefficient K(D).

[0048] K(D) is a composite coefficient such that 0 ≤ K(D) ≤ 1. K(D) is a function of distance D, for example, a step function or a monotonically increasing function. The distance D is D high , D low , or the average of both. Distance D may be an approximate distance. That is, distance D is D high , D low , has a value based on at least one of the approximate distances.

[0049]

number

[0050] The distance calculation unit 67 calculates the combined distance by weighting and combining the measured distances of each region according to the distance between the vehicle 10 and the landmark 2, as shown in equation (1). Specifically, when the vehicle 10 is far from the landmark 2, mainly D high Based on this, the combined distance is calculated, and if vehicle 10 is close to landmark 2, then mainly D low The combined distance is calculated based on this. This allows for accurate calculation of the combined distance regardless of whether the distance from vehicle 10 to landmark 2 is large or small.

[0051] The same applies when there are three or more regions on the main surface S of landmark 2 that have different reflectances. Landmark 2 has regions A1, A2, ..., A N (N≧3) and from train 10 to region A n The distance measured up to (1 ≤ n ≤ N) is distance D. n Let's assume that. In this case, equation (1) can be generalized to equation (2). K n (D) is, This is the composite coefficient, TIFF0007909503000002.tif10170. K n (D) is a function of distance D, such as a pulse function or window function that has values ​​only within a specific distance range. Distance D is the distance measurement distance D n It is one of these, or their average. Distance D may be an approximate distance.

[0052]

number

[0053] Furthermore, once the distance calculation unit 67 calculates the combined distance, it outputs the combined distance. For example, the combined distance may be output to a display visible to the driver of the vehicle 10, or it may be output to the automated driving system of the vehicle 10.

[0054] Furthermore, when the distance calculation unit 67 calculates the combined distance, it determines whether the combined distance is less than or equal to a second threshold. The second threshold corresponds to, for example, the distance from the vehicle 10 to the landmark 2 when the vehicle 10 reaches the stopping position, or the distance at which the landmark 2 is outside the field of view of the rangefinder 5.

[0055] <Flowchart> Figure 8 is a flowchart illustrating an example of the processing performed by the distance measurement support device 6. The following explanation of the processing performed by the distance measurement support device 6 will refer to Figure 8.

[0056] First, the positioning data acquisition unit 61 acquires positioning data, including the current position of the vehicle 10, from the positioning system 3 (step S1).

[0057] Next, the approximate distance calculation unit 62 calculates the distance (approximate distance) between the vehicle 10 and landmark 2 based on the positioning data and the location of landmark 2 (step S2). The location of landmark 2 is known and stored in the landmark information storage unit 63.

[0058] Next, the approximate distance calculation unit 62 determines whether the calculated approximate distance is less than or equal to the first threshold (step S3). If the approximate distance calculation unit 62 determines that the approximate distance exceeds the first threshold (step S3: No), the process returns to step S1, and the positioning data acquisition unit 61 acquires positioning data again.

[0059] If the approximate distance calculation unit 62 determines that the approximate distance is less than or equal to the first threshold (step S3: Yes), then approximately simultaneously (approximately in the same frame), the image acquisition unit 64 starts acquiring a forward image from the camera 4, and the distance image acquisition unit 66 starts acquiring a distance image from the distance measuring device 5 (step S4).

[0060] Next, the landmark detection unit 65 detects landmark 2 based on the acquired forward image (step S5).

[0061] Next, the distance calculation unit 67 determines the distance measurement for each region of the landmark 2 that has a different reflectivity (step S6).

[0062] Next, the distance calculation unit 67 combines the multiple measured distances to calculate the combined distance (step S7).

[0063] Next, the distance calculation unit 67 outputs the combined distance (step S8).

[0064] Next, the distance calculation unit 67 determines whether the calculated combined distance is less than or equal to the second threshold (step S9). If it is determined that the combined distance exceeds the second threshold (step S9: No), the process returns to step S4.

[0065] If it is determined that the combined distance is less than or equal to the second threshold (step S9: Yes), the process is terminated. Note that each step may be performed in parallel with other steps. For example, while the approximate distance calculation unit 62 is calculating the approximate distance in step S2, the positioning data acquisition unit 61 may continue to acquire positioning data from the positioning system 3.

[0066] In this way, the processing in steps S1 to S2 roughly calculates the distance from vehicle 10 to landmark 2 using positioning data, and when the approximate distance falls below the first threshold, the processing in steps S4 to S7 more accurately calculates the distance from vehicle 10 to landmark 2 using the distance image acquired by the distance measuring device 5.

[0067] As explained above, in this embodiment, the distance from the vehicle 10 to landmark 2 is calculated based on the distance measured for each region of the landmark 2, which has a main surface arranged with multiple regions having different reflectances. The weight of each reflectance of landmark 2 is changed according to the distance from the vehicle 10 to landmark 2, and the combined distance is calculated. Therefore, it is possible to measure the distance to landmark 2 more accurately over a wider measurement range. As a result, even if the vehicle 10 is a train or the like and the braking distance is relatively long, it is possible to measure the distance to landmark 2 more accurately, so that, for example, when an automated driving system stops the vehicle 10, more accurate control is possible. In addition, the presence of a pattern on the main surface of the landmark makes it possible to detect the landmark more accurately.

[0068] Furthermore, the distance measurement support device 6 changes the method of calculating distance D depending on whether the distance from vehicle 10 to landmark 2 is below a first threshold, using either a method that calculates an approximate distance using positioning data or a method that calculates a composite distance using distance images and forward images. This enables efficient calculation of distance D.

[0069] (Another example of operation 1) Figure 9 shows another example of the operation of the landmark detection unit 65. As shown in Figure 9, the landmark detection unit 65 may delete a portion of the forward image and then detect landmark 2.

[0070] For example, suppose the approximate distance calculation unit 62 determines that the approximate distance from vehicle 10 to landmark 2 is 100m. Next, a range L1 (for example, a range of 20m in the front-rear direction from landmark 2) that is a certain distance from landmark 2 relative to the approximate distance is identified in the distance image (see Figure 5) based on the pixel values ​​(distance values) included in the distance image. Landmark 2 is expected to be located within this range. The range corresponding to the range identified in the distance image is identified in the forward image, and landmark 2 is detected within the identified range. That is, the range other than the identified range is excluded or deleted from the forward image, and the detection of landmark 2 is started. As a result, the range that is subject to detection of landmark 2 is reduced, and landmark 2 can be detected more efficiently.

[0071] Furthermore, this is not the only method for deleting a portion of the forward image. For example, depending on the field of view of camera 4 and the height of landmark 2, it may be known that landmark 2 will not be located in the lower half of the forward image when vehicle 10 is at least in front of the stopping position. In that case, the lower part of the forward image may be identified as an area where landmark 2 is not located, and this identified area may be excluded from the area to be detected as landmark 2 (the identified area may be deleted from the forward image).

[0072] Furthermore, in addition to the lower part of the forward image, if it is known that landmark 2 is located to the left of the railway track T, the right side of the forward image may also be identified as an area where landmark 2 is not located and deleted. Alternatively, if it is known that landmark 2 appears at approximately the same height as the horizon H in the forward image, the horizon H may be detected from the forward image, and the area outside the vicinity of the horizon H may be identified as an area where landmark 2 is not located and excluded from the area to be detected for landmark 2.

[0073] (Another example of operation 2) The range measuring support device 6 may be linked to the train's brake control system. The brake control system may control the vehicle's brakes according to the combined distance output by the range measuring support device 6. The second threshold value may be set to a value corresponding to the train's stopping position, and the brake control system may control the train to come to a complete stop when the combined distance reaches the second threshold.

[0074] (Second Embodiment) In the first embodiment, a forward image was acquired using the camera 4, and the position of landmark 2 was detected from the forward image. However, landmark 2 may also be detected from a distance image acquired by the range measuring device 5 (see Figure 5). In the second embodiment, the case in which landmark 2 is detected from a distance image acquired by the range measuring device 5 will be described.

[0075] Figure 10 is a block diagram of the distance measuring system 1A according to the second embodiment. Elements with the same names or functions as those in Figure 1 of the first embodiment described above are denoted by the same reference numerals. Hereafter, descriptions will be omitted except for changes or additions.

[0076] In the distance measuring system 1A, the distance measuring device 5 also functions as the camera 4 in the first embodiment. Therefore, the camera 4 is omitted.

[0077] The image acquisition unit 64 acquires a distance image as a forward image, as shown in Figure 5 in the first embodiment. In other words, in this embodiment, the forward image shows the intensity distribution of the reflected laser light emitted from the distance measuring device 5 and reflected by the landmark 2. In the first embodiment, the field of view of the camera 4 and the field of view of the distance measuring device 5 were different, but in this embodiment they already coincide and there is no need to perform correction.

[0078] The landmark detection unit 65 detects landmark 2, similar to the first embodiment. Here, the template image is represented by the light intensity distribution.

[0079] As described above, in the second embodiment, the intensity distribution of the reflected laser light emitted from the distance measuring device 5 and reflected by the landmark 2 is treated as the forward image. This eliminates the need to prepare a camera to capture the forward image, thus enabling miniaturization and cost reduction of the entire system.

[0080] (Hardware configuration) Figure 7 shows the hardware configuration of the information processing device according to each embodiment. The information processing device consists of a computer device 100. The computer device 100 includes a CPU 101, an input interface 102, a display device 103, a communication device 104, a main memory 105, and an external memory device 606, which are interconnected by a bus 107.

[0081] The CPU (Central Processing Unit) 101 executes an information processing program, which is a computer program, on the main memory 105. The information processing program is a program that realizes each of the above-described functional configurations of the information processing device. The information processing program may not be a single program, but rather a combination of multiple programs or scripts. Each functional configuration is realized when the CPU 101 executes the information processing program.

[0082] The input interface 102 is a circuit for inputting operation signals from input devices such as keyboards, mice, and touch panels to the information processing device. The input interface 102 corresponds to the input section of the information processing device according to each embodiment.

[0083] The display device 103 displays data output from the information processing device. The display device 103 is, for example, an LCD (liquid crystal display), an organic electroluminescent display, a CRT (cathode ray tube), or a PDP (plasma display), but is not limited to these. Data output from the computer device 100 can be displayed on this display device 103. The display device 103 corresponds to the output unit of the information processing device according to each embodiment.

[0084] The communication device 104 is a circuit for the information processing device to communicate with an external device wirelessly or via a wired connection. Data can be input from an external device via the communication device 104. The data input from the external device can be stored in the main memory 105 or the external memory 606.

[0085] The main memory 105 stores information processing programs, data necessary for executing the information processing programs, and data generated by the execution of the information processing programs. The information processing programs are deployed and executed on the main memory 105. The main memory 105 is, for example, RAM, DRAM, or SRAM, but is not limited to these. Each storage unit or database of the information processing apparatus according to each embodiment may be built on the main memory 105.

[0086] The external storage device 606 stores information processing programs, data necessary for executing the information processing programs, and data generated by the execution of the information processing programs. These information processing programs and data are read into the main memory 105 when the information processing programs are executed. The external storage device 606 is, for example, a hard disk, optical disk, flash memory, and magnetic tape, but is not limited to these. Each storage unit or database of the information processing device may be built on the external storage device 606.

[0087] The information processing program may be pre-installed on the computer device 100, or it may be stored on a storage medium such as a CD-ROM. Furthermore, the information processing program may be uploaded to the internet.

[0088] Furthermore, the information processing device may consist of a single computer device 100, or it may be configured as a system consisting of multiple interconnected computer devices 100.

[0089] It should be noted that the present invention is not limited to the embodiments described above, and the components can be modified and implemented in practice without departing from the spirit of the invention. Furthermore, various inventions can be formed by appropriately combining the multiple components disclosed in the embodiments described above. For example, a configuration in which some components are removed from all the components shown in each embodiment is also conceivable. Moreover, components described in different embodiments may be appropriately combined.

[0090] This embodiment can also be configured as follows. [Item 1] A first image acquisition unit acquires a first image generated by a vehicle camera that images the forward environment including a landmark having a main surface in which multiple regions with different reflectivity are arranged, A landmark detection unit for detecting the position of the landmark within the first image, A second image acquisition unit acquires a second image generated by a distance measuring device that measures the distance to the forward environment, A distance calculation unit that, in the second image, identifies regions corresponding to the plurality of regions of the landmark included in the first image, calculates the distance between the plurality of regions of the landmark and the vehicle based on the pixel values ​​of the identified regions, and calculates the distance between the vehicle and the landmark based on the distance calculated for each region, A distance measuring support device equipped with the following features. [Item 2] The distance calculation unit calculates the distance from the vehicle to the landmark by combining the distances for each region. The distance measuring support device described in item 1. [Item 3] The distance calculation unit determines a weight for each of the distances in each region according to one of the distances in each region, and combines the distances in each region using the weight. The distance measurement support device described in item 2. [Item 4] A positioning data acquisition unit that acquires positioning data of the vehicle's location from a positioning system, The system includes an approximate distance calculation unit that calculates the approximate distance between the vehicle and the landmark based on the positioning data and the location information of the landmark, The distance calculation unit determines the weight of the distance for each region based on the estimated distance, and combines the distances for each region using the weights. A distance measuring support device as described in item 2 or 3. [Item 5] A positioning data acquisition unit that acquires positioning data including the vehicle's position from a positioning system, The system includes an approximate distance calculation unit that calculates the approximate distance between the vehicle and the landmark based on the positioning data and the location information of the landmark, The landmark detection unit starts detecting the location of the landmark when the estimated distance is less than or equal to the first threshold. A distance measuring support device as described in any one of items 2 to 4. [Item 6] A positioning data acquisition unit that acquires positioning data including the vehicle's position from a positioning system, The system includes an approximate distance calculation unit that calculates the approximate distance between the vehicle and the landmark based on the positioning data and the location information of the landmark, The landmark detection unit determines the number of pixels of the landmark in the first image based on the landmark information and the estimated distance, creates a template image of the landmark based on the number of pixels, and detects the landmark by template matching using the first image and the template image. A distance measuring support device as described in any one of items 1 to 5. [Item 7] A positioning data acquisition unit that acquires positioning data including the vehicle's position from a positioning system, The system includes an approximate distance calculation unit that calculates the approximate distance between the vehicle and the landmark based on the positioning data and the location information of the landmark, The landmark detection unit identifies a certain distance range from the estimated distance in the second image and detects the landmark by template matching using the image portion of the first image corresponding to the identified distance range and the template image. The distance measuring support device described in item 6. [Item 8] The landmark detection unit creates a template image that includes the landmark and at least a portion of the background surrounding the landmark, and detects the landmark by template matching using the first image and the template image. A distance measuring support device as described in item 6 or 7. [Item 9] The landmark detection unit identifies an area in the first image that does not contain the landmark based on the relationship between the camera's position and the landmark's position, and then performs landmark detection on the area outside the identified area. A distance measuring support device as described in any one of items 1 to 8. [Item 10] The aforementioned vehicle is a train. A distance measuring support device as described in any one of items 1 to 9. [Item 11] A vehicle camera that images the forward environment including a landmark having a main surface in which multiple regions with different reflectivity are arranged, A distance measuring device for measuring the distance to the forward environment, A first image acquisition unit that acquires a first image generated by the aforementioned camera, A landmark detection unit for detecting the position of the landmark within the first image, A second image acquisition unit that acquires a second image generated by the distance measuring device, A distance calculation unit that, in the second image, identifies regions corresponding to the plurality of regions of the landmark included in the first image, calculates the distance between the plurality of regions of the landmark and the vehicle based on the pixel values ​​of the identified regions, and calculates the distance between the vehicle and the landmark based on the distance calculated for each region, A distance measuring system equipped with the following features. [Item 12] A first image is obtained, generated by a vehicle camera that images the forward environment including a landmark having a main surface composed of multiple regions with different reflectivity, The position of the landmark in the first image is detected, A second image is obtained, which is generated by a distance measuring device that measures the distance to the forward environment. In the second image, the regions corresponding to the plurality of regions of the landmark included in the first image are identified, Based on the pixel values ​​of the identified region, the distance between each of the multiple regions of the landmark and the vehicle is calculated. Based on the distances calculated for each of the aforementioned regions, the distance between the vehicle and the landmark is calculated. Ranging support method. [Explanation of Symbols]

[0091] 1. 1A Distancing System 2 Landmarks 3. Positioning System 4. Camera (imaging device) 5 Ranging device 6 Ranging support device 61 Positioning data acquisition unit 62 Approximate distance calculation section 63 Landmark Information Storage Unit 64 Image acquisition unit (first image acquisition unit) 65 Landmark detection unit 66 Distance image acquisition unit (second image acquisition unit) 67 Distance Calculation Unit 10 trains 100 Computer devices 101 CPU 102 Input Interfaces 103 Display device 104 Communication equipment 105 Main memory 106 External storage device 107 Bus Ahigh region A low region B range DT pixels H horizon K composite coefficient P Home R Railway S Main surface

Claims

1. A first image acquisition unit acquires a first image generated by a vehicle imaging device that images the forward environment including a landmark having a main surface in which multiple regions with different reflectivity are arranged, A landmark detection unit for detecting the position of the landmark within the first image, A second image acquisition unit acquires a second image generated by a distance measuring device that measures the distance to the forward environment, A distance calculation unit that, in the second image, identifies regions corresponding to the plurality of regions of the landmark included in the first image, calculates a first distance between the plurality of regions of the landmark and the vehicle based on the pixel values ​​of the identified regions, determines a weight for the first distance calculated for each region according to one of the first distances calculated for each region or the average of the first distances, and calculates a second distance between the vehicle and the landmark by combining the first distances for each region with the weight, A distance measuring support device equipped with the following features.

2. A first image acquisition unit that acquires a first image generated by a vehicle imaging device that images the forward environment including a landmark having a main surface on which a plurality of regions having different reflectances are arranged, A landmark detection unit for detecting the position of the landmark within the first image, A second image acquisition unit acquires a second image generated by a distance measuring device that measures the distance to the forward environment, A positioning data acquisition unit that acquires positioning data of the vehicle's location from a positioning system, The system includes an approximate distance calculation unit that calculates the approximate distance between the vehicle and the landmark based on the positioning data and the location information of the landmark, A distance calculation unit that, in the second image, identifies regions corresponding to the plurality of regions of the landmark included in the first image, calculates a first distance between the plurality of regions of the landmark and the vehicle based on the pixel values ​​of the identified regions, determines the weight of the first distance calculated for each region based on the approximate distance, and calculates a second distance between the vehicle and the landmark by combining the first distances for each region using the weights, A distance measuring support device equipped with the following features.

3. A positioning data acquisition unit that acquires positioning data including the vehicle's position from a positioning system, The system includes an approximate distance calculation unit that calculates the approximate distance between the vehicle and the landmark based on the positioning data and the location information of the landmark, The landmark detection unit starts detecting the location of the landmark when the estimated distance is less than or equal to the first threshold. The distance measuring support device according to claim 1.

4. A positioning data acquisition unit that acquires positioning data including the vehicle's position from a positioning system, The system includes an approximate distance calculation unit that calculates the approximate distance between the vehicle and the landmark based on the positioning data and the location information of the landmark, The landmark detection unit determines the number of pixels of the landmark in the first image based on the landmark information and the estimated distance, creates a template image of the landmark based on the number of pixels, and detects the landmark by template matching using the first image and the template image. The distance measuring support device according to claim 1.

5. A positioning data acquisition unit that acquires positioning data including the vehicle's position from a positioning system, The system includes an approximate distance calculation unit that calculates the approximate distance between the vehicle and the landmark based on the positioning data and the location information of the landmark, The landmark detection unit identifies a certain distance range from the estimated distance in the second image and detects the landmark by template matching using the image portion of the first image corresponding to the identified distance range and the template image. The distance measuring support device according to claim 4.

6. The landmark detection unit creates a template image that includes the landmark and at least a portion of the background surrounding the landmark, and detects the landmark by template matching using the first image and the template image. The distance measuring support device according to claim 4.

7. The landmark detection unit identifies an area in the first image that does not contain the landmark based on the relationship between the position of the imaging device and the position of the landmark, and then performs landmark detection on the area outside the identified area. The distance measuring support device according to claim 1.

8. The aforementioned vehicle is a train. The distance measuring support device according to claim 1.

9. An imaging device for a vehicle that images the forward environment including a landmark having a main surface in which multiple regions with different reflectivity are arranged, A distance measuring device for measuring the distance to the forward environment, A first image acquisition unit that acquires a first image generated by the imaging device, A landmark detection unit for detecting the position of the landmark within the first image, A second image acquisition unit that acquires a second image generated by the distance measuring device, A distance calculation unit that, in the second image, identifies regions corresponding to the plurality of regions of the landmark included in the first image, calculates a first distance between the plurality of regions of the landmark and the vehicle based on the pixel values ​​of the identified regions, determines a weight for the first distance calculated for each region according to one of the first distances calculated for each region or the average of the first distances, and calculates a second distance between the vehicle and the landmark by combining the first distances for each region with the weight, A distance measuring system equipped with the following features.

10. A first image is obtained, generated by a vehicle imaging device that images the forward environment including a landmark having a main surface in which multiple regions with different reflectivity are arranged, and The position of the landmark in the first image is detected, A second image is obtained, which is generated by a distance measuring device that measures the distance to the forward environment. In the second image, the regions corresponding to the plurality of regions of the landmark included in the first image are identified, Based on the pixel values ​​of the identified region, the first distance between each of the multiple regions of the landmark and the vehicle is calculated. Based on one of the first distances calculated for each region or the average of the first distances, a weight is determined for the first distances calculated for each region, and the second distance between the vehicle and the landmark is calculated by combining the first distances for each region using the weight. Ranging support method.

11. An imaging device for a vehicle that images a forward environment including a landmark having a main surface on which a plurality of regions having different reflectances are arranged, A distance measuring device for measuring the distance to the forward environment, A first image acquisition unit that acquires a first image generated by the imaging device, A landmark detection unit for detecting the position of the landmark within the first image, A second image acquisition unit that acquires a second image generated by the distance measuring device, A positioning data acquisition unit that acquires positioning data of the vehicle's location from a positioning system, An approximate distance calculation unit calculates the approximate distance between the vehicle and the landmark based on the positioning data and the location information of the landmark, A distance calculation unit that, in the second image, identifies regions corresponding to the plurality of regions of the landmark included in the first image, calculates a first distance between the plurality of regions of the landmark and the vehicle based on the pixel values ​​of the identified regions, determines the weight of the first distance calculated for each region based on the approximate distance, and calculates a second distance between the vehicle and the landmark by combining the first distances for each region using the weights, A distance measuring system equipped with the following features.

12. A first image is obtained by a vehicle imaging device that images the forward environment including a landmark having a main surface on which a plurality of regions having different reflectances are arranged, The position of the landmark in the first image is detected, A second image is obtained, which is generated by a distance measuring device that measures the distance to the forward environment. The positioning data of the vehicle's location is obtained from the positioning system. Based on the positioning data and the location information of the landmark, the approximate distance between the vehicle and the landmark is calculated. In the second image, the regions corresponding to the plurality of regions of the landmark included in the first image are identified, Based on the pixel values ​​of the identified region, the first distance between each of the multiple regions of the landmark and the vehicle is calculated. Based on the estimated distances, the weights of the first distances calculated for each region are determined, and the second distance between the vehicle and the landmark is calculated by combining the first distances for each region using the weights. Ranging support method.

Images