Front monitoring system, front monitoring device, front monitoring method, and program

Abstract

To suitably control a sensing direction of sensing means of a LiDAR device or the like.SOLUTION: A front monitoring system comprises: sleeper detection means for detecting a sleeper; sensing means for sensing a part ahead of a train in an advancing direction; sensing control means for controlling the sensing direction of the sensing means; and sensing control information storage means for storing sensing control information for associating the number of times of detecting sleepers with the sensing direction. The sensing control means determines the sensing direction on the basis of the number of times of detecting sleepers and the sensing control information.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a forward monitoring system, a forward monitoring device, a forward monitoring method, and a program. [Background technology]

[0002] Patent Document 1 discloses a train control system that estimates the current position of a train by counting sleepers. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2021 / 106415 Summary of the Invention [Problem to be solved by the invention]

[0004] By the way, in order to operate a train traveling at a speed of 100 kilometers per hour or more properly, it is typically necessary to monitor an area 500 meters or more ahead of the train. The inventors of the present application are considering using a LiDAR device for the above-mentioned forward monitoring.

[0005] In general, the point cloud generated by a LiDAR device has a property that the density becomes lower the farther away from the LiDAR device. Therefore, in order to ensure the point cloud density, it is effective to narrow the field of view (FoV). However, railroad tracks do not necessarily extend in a straight line, and straight sections are rare, especially in urban areas. Therefore, narrowing the field of view creates a new problem in that distant railroad tracks that should be monitored are not in the field of view. This problem will be particularly evident when a three-dimensional point cloud of the running environment is reconstructed using Simultaneous Localization and Mapping (SLAM) and obstacle detection is performed based on the reconstructed three-dimensional point cloud. This is because the three-dimensional point cloud of the running environment far away in the direction of travel of the train is required for obstacle detection, but the point cloud density of the three-dimensional point cloud of the running environment far away in the direction of travel of the train cannot be sufficiently increased.

[0006] Therefore, an object of the present disclosure is to provide a technique for suitably controlling the sensing direction of a sensing means such as a LiDAR device. [Means for solving the problem]

[0007] According to a first aspect of the present disclosure, A sleeper detection means for detecting a sleeper; A sensing means for sensing a forward direction of a train; a sensing control means for controlling a sensing direction of the sensing means; a sensing control information storage means for storing sensing control information that associates the number of times the sleeper is detected with the sensing direction; Including, the sensing control means determines the sensing direction based on the number of times the sleeper is detected and the sensing control information. A forward looking system is provided.

[0008] According to a second aspect of the present disclosure, A sleeper detection means for detecting a sleeper; A sensing means for sensing a forward direction of a train; a sensing control means for controlling a sensing direction of the sensing means; a sensing control information storage means for storing sensing control information that associates the number of times the sleeper is detected with the sensing direction; Including, the sensing control means determines the sensing direction based on the number of times the sleeper is detected and the sensing control information. A forward looking system is provided.

[0009] According to a third aspect of the present disclosure, Detect sleepers, Sensing the area ahead in the direction of train travel, Controlling the sensing direction of the sensing means; storing sensing control information that associates the number of times the sleeper is detected with the sensing direction; Controlling the sensing direction of the sensing means includes determining the sensing direction based on the number of detections of the sleeper and the sensing control information. A forward looking method is provided. Effect of the Invention

[0010] According to the present disclosure, the sensing direction of a sensing means such as a LiDAR device can be suitably controlled. [Brief description of the drawings]

[0011] [Figure 1] FIG. 2 is a block diagram of a forward monitoring system. [Diagram 2] FIG. 2 is a block diagram of a train. [Diagram 3] FIG. 4 is a data structure diagram of a reference table. [Figure 4] FIG. 2 illustrates the line of sight of the LiDAR device when the rail is straight. [Diagram 5] FIG. 1 illustrates the line of sight of a LiDAR device when the rail is curved. [Figure 6] 4 is a control flow of the forward monitoring device. [Figure 7] FIG. 2 is a diagram showing a case where a processing circuit provided in a forward monitoring device is configured with a processor and a memory. [Figure 8] FIG. 13 is a diagram showing a case where a processing circuit provided in the forward monitoring device is configured with dedicated hardware. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] (Summary of the Disclosure) Hereinafter, an outline of the present disclosure will be described with reference to the drawings. Fig. 1 is a block diagram of a forward monitoring system. As shown in Fig. 1, the forward monitoring system 100 includes a sleeper detection means 101, a sensing means 102, a sensing control means 103, and a sensing control information storage means 104.

[0013] The sleeper detection means 101 detects sleepers.

[0014] The sensing means 102 senses the area ahead in the traveling direction of the train.

[0015] The sensing control means 103 controls the sensing direction of the sensing means.

[0016] The sensing control information storage means 104 stores sensing control information that associates the number of times the sleeper is detected with the sensing direction.

[0017] The sensing control means 103 determines the sensing direction based on the number of times the sleeper is detected and the sensing control information.

[0018] According to the above configuration, the sensing direction can be determined based on the number of times the sleeper is detected, so that the sensing direction of the sensing means can be suitably controlled.

[0019] (First embodiment) Next, a first embodiment of the present disclosure will be described. Fig. 2 is a block diagram of a train 1. The train 1 runs along a track structure 2.

[0020] The train 1 includes a forward monitoring device 3, a driving device 4, a braking device 5, and wheels 6.

[0021] The drive device 4 includes a motor and a motor driver. The drive device 4 drives the wheels 6 in accordance with a control signal from the forward monitoring device 3.

[0022] The braking device 5 is composed of a brake and a hydraulic circuit. The braking device 5 brakes the wheels 6 in accordance with a control signal from the forward monitoring device 3.

[0023] The railway structure 2 includes a ballast, a plurality of sleepers 2a arranged at predetermined intervals on the ballast, and two rails 2b supported by the plurality of sleepers 2a.

[0024] The forward monitoring device 3 is a device for detecting an obstacle ahead of the train 1 in the traveling direction by monitoring the forward direction of the train 1.

[0025] The forward monitoring device 3 includes a LiDAR device 10, a line of sight control device 11, an IMU 12, a sleeper sensor 13, an azimuth angle sensor 14, a reference table memory unit 20, a point cloud memory unit 21, a sleeper counting unit 29, a rough self-position estimation unit 30, a line of sight determination unit 31, a SLAM initial position calculation unit 32, a SLAM processing unit 33, an obstacle detection unit 34, and a vehicle control unit 35.

[0026] The LiDAR device 10 (Light Detection And Ranging) is a specific example of a sensing means that senses the area ahead of the train 1 in the traveling direction and generates a three-dimensional point cloud. The LiDAR device 10 of this embodiment is a direct ToF (Time of Flight) type. That is, the LiDAR device 10 emits laser light ahead of the traveling direction of the train 1 and measures the time it takes to receive the reflected light, thereby generating a three-dimensional point cloud ahead of the traveling direction of the train 1. However, instead of this, the LiDAR device 10 may be an FMCW (Frequency Modulated Continuous Wave) type that generates the above-mentioned three-dimensional point cloud based on the frequency difference between the laser light emitted ahead of the traveling direction of the train 1 and the reflected light. In addition, the LiDAR device 10 may be an indirect ToF type that generates the above-mentioned three-dimensional point cloud based on the phase difference between the laser light emitted ahead of the traveling direction of the train 1 and the reflected light. The LiDAR device 10 accumulates the generated three-dimensional point cloud (hereinafter also simply referred to as a point cloud) in the point cloud storage unit 21 as a partial point cloud 21a.

[0027] The sensing means is not limited to the LiDAR device 10. Any device capable of sensing the area ahead of the train 1 in the traveling direction may be used as the sensing means. For example, the sensing means may be configured to generate a three-dimensional point cloud using a radar device (Radio Detection And Ranging), an ultrasonic sensor, a stereo camera, or a combination of these. The sensing means may also be configured to generate a three-dimensional point cloud using Structure from Motion (SfM) from a plurality of two-dimensional images obtained by capturing an image of the area ahead of the train 1 in the traveling direction.

[0028] The line of sight control device 11 controls the line of sight of the LiDAR device 10. Here, the line of sight of the LiDAR device 10 is a specific example of the sensing direction of the LiDAR device 10. The line of sight of the LiDAR device 10 means the central axis of scanning. The central axis of scanning is an axis consisting of the center of the horizontal angle of the scanning direction and the center of the vertical angle of the scanning direction. The line of sight control device 11 is typically an electric stage that controls the line of sight of the LiDAR device 10 by panning and tilting the LiDAR device 10. Alternatively, the LiDAR device 10 may include the function of the line of sight control device 11. The line of sight control device 11 controls the line of sight of the LiDAR device 10 based on line of sight control information output from the line of sight determination unit 31.

[0029] An IMU 12 (Inertial Measurement Unit) detects the three-axis acceleration and three-axis angular velocity of the train 1 .

[0030] The sleeper sensor 13 detects the sleepers 2a while the train 1 is traveling. Typically, the sleeper sensor 13 detects the sleepers 2a based on the output value of a laser displacement meter provided on the underside of the train 1. However, the sleeper sensor 13 may also detect the sleepers 2a based on an image output from a camera that captures an image of the area below the train 1.

[0031] The azimuth angle sensor 14 detects the azimuth angle of the train 1. The azimuth angle sensor 14 detects the azimuth angle of the leading car of the train 1. Alternatively, the azimuth angle sensor 14 may detect the azimuth angle of a fixed part of the pan-tilt mechanism of the line of sight control device 11.

[0032] The reference table storage unit 20 stores the reference table 20a shown in FIG. 3. As shown in FIG. 3, the reference table 20a is a specific example of sensing control information. The reference table 20a is a table that associates the sleeper count number, the position information, and the gaze control information. The sleeper count number means the number of times the sleeper 2a is detected by the sleeper sensor 13. However, the sleeper count number can be appropriately reset at a predetermined fixed point to eliminate a deviation in the count. The position information indicates the position of the train 1 in the global coordinate system. The gaze control information indicates the gaze of the LiDAR device 10. The gaze control information includes a horizontal angle (azimuth) and a vertical angle (elevation). The forward monitoring device 3 can quickly obtain the current location of the train 1 and the optimal gaze of the LiDAR device 10 based on the number of detections by the sleeper sensor 13 by referring to the reference table 20a. For example, when the sleeper count is 244, the current location of train 1 is (x244,y244,z244) and the optimal line of sight of the LiDAR device 10 is (az244,ev244).

[0033] The optimal line of sight of the LiDAR device 10 will now be described with reference to FIGS.

[0034] 4 is a plan view of a case where a rail 2b extends linearly ahead of the train 1 in the traveling direction. As a premise, in order to properly operate a train 1 traveling at a speed of 100 kilometers per hour or more, it is typically necessary to monitor 500 meters or more ahead as seen from the train 1. In the case of FIG. 4, even if the line of sight 10a of the LiDAR device 10 is kept aligned with the traveling direction 1a of the train 1 (i.e., the longitudinal direction of the leading car of the train 1), the LiDAR device 10 will be able to monitor 500 meters or more ahead as seen from the train 1.

[0035] Fig. 5 is a plan view of a case where the rail 2b extends in a curved manner ahead of the traveling direction of the train 1. In the case of Fig. 5, in order for the LiDAR device 10 to monitor an area 500 meters or more ahead as seen from the train 1, the line of sight 10a of the LiDAR device 10 would need to be tilted to the right by about 45 degrees with respect to the traveling direction 1a of the train 1 in a plan view.

[0036] In this way, the line of sight of the LiDAR device 10 suitable for monitoring an area 500 meters or more ahead from the train 1 varies depending on whether the rail 2b extends straight or curved ahead in the direction of travel of the train 1.

[0037] In this way, it is possible to sequentially calculate the optimal line of sight of the LiDAR device 10 while the train 1 is running, for example, based on a forward image captured by a camera mounted on the train 1. However, it takes a lot of time to calculate the optimal line of sight of the LiDAR device 10 by image processing the forward image. Therefore, it is not realistic to calculate the optimal line of sight of the LiDAR device 10 in real time while the train 1 is running based on the forward image. Therefore, the inventor of the present application decided to create a reference table 20a that associates the sleeper count number with the line of sight control information one-to-one before the operation of the train 1. As a result, the line of sight determination unit 31 can quickly acquire line of sight control information by sequentially referring to the reference table 20a while the train 1 is running, and thus it becomes possible to control the line of sight of the LiDAR device 10 in real time.

[0038] 2, the point cloud storage unit 21 stores the partial point cloud 21a, the past entire point cloud 21b, and the latest entire point cloud 21c. The past entire point cloud 21b and the latest entire point cloud 21c are specific examples of the entire point cloud.

[0039] The partial point cloud 21a is a collection of point clouds generated by the LiDAR device 10 for a predetermined time width while the train 1 is traveling. As described above, the LiDAR device 10 sequentially accumulates the generated point clouds in the point cloud storage unit 21 as the partial point cloud 21a.

[0040] The past entire point cloud 21b is a point cloud generated at a certain point in the past of the past running environment of the train 1. The self-position of the train 1 can be estimated by aligning the partial point cloud 21a with the past entire point cloud 21b.

[0041] The latest entire point cloud 21c is a point cloud of the current running environment of the train 1, which is generated by overlapping the partial point clouds 21a generated multiple times on the time axis.

[0042] This method of estimating the train 1's position using the partial point cloud 21a and the past overall point cloud 21b, and generating the latest overall point cloud 21c using the self-position estimation result and the partial point cloud 21a, is called SLAM (Simultaneous Localization and Mapping).

[0043] The sleeper counting unit 29 counts the number of detections by the sleeper sensor 13 .

[0044] The rough self-position estimating unit 30 roughly estimates the self-position of the train 1 with an accuracy of about 1 meter to 2 meters. Specifically, the rough self-position estimating unit 30 acquires the self-position of the train 1 corresponding to the number of detections counted by the sleeper counting unit 29 by referring to the reference table 20a.

[0045] The line of sight determination unit 31 determines the optimal line of sight for the train 1. Specifically, the line of sight determination unit 31 acquires line of sight control information corresponding to the number of detections counted by the sleeper counting unit 29 by referring to the reference table 20a. The line of sight determination unit 31 outputs the acquired line of sight control information to the line of sight control device 11. In this embodiment, the sensing control means is composed of the line of sight control device 11 and the line of sight determination unit 31.

[0046] The SLAM processing unit 33 estimates the self-position of the train 1 using SLAM, and generates a point cloud of the latest running environment of the train 1. When the SLAM processing unit 33 estimates the self-position of the train 1, the SLAM processing unit 33 aligns the partial point cloud 21a with the past entire point cloud 21b. At this time, the SLAM processing unit 33 can typically use an ICP (Iterative Closest Point) method.

[0047] The SLAM initial position calculation unit 32 calculates an initial position of the partial point group 21a with respect to the past entire point group 21b when the SLAM processing unit 33 executes self-position estimation. Specifically, the SLAM initial position calculation unit 32 calculates the above-mentioned initial position based on the self-position of the train 1 roughly estimated by the self-position rough estimation unit 30, the detection result of the IMU 12, and the line-of-sight control information output from the line-of-sight determination unit 31.

[0048] When the SLAM initial position calculation unit 32 calculates the above-mentioned initial position, the SLAM initial position calculation unit 32 converts the partial point group 21a into the coordinate system of the past entire point group 21b using the self-position of the train 1 roughly estimated by the self-position rough estimation unit 30. This provides the following technical effects. That is, instead of the self-position of the train 1 roughly estimated by the self-position rough estimation unit 30, it is also possible to use the self-position of the train 1 based on the detection result of the IMU 12. However, the train 1 travels a long distance of several tens of kilometers, and the self-position estimation of the train 1 based on the detection result of the IMU 12 becomes extremely poor in accuracy due to the accumulation of detection errors of the IMU 12. In contrast, the self-position estimation by the self-position rough estimation unit 30 is based on the number of detections of the sleeper sensor 13, so although the estimation accuracy necessarily includes an error of about several tens of centimeters, it is more robust than the IMU 12.

[0049] Moreover, when the SLAM initial position calculation unit 32 calculates the above-mentioned initial position, the SLAM initial position calculation unit 32 uses the detection result of the IMU 12. This makes it possible to realize highly accurate movement compensation of the partial point group 21a. That is, since the train 1 runs continuously, the movement amount of the train 1 gradually increases, whereas the rough estimation by the self-position rough estimation unit 30 is performed, for example, every 1 meter, so that it is practically impossible to perform movement compensation of the partial point group 21a by the rough estimation by the self-position rough estimation unit 30. Therefore, in this embodiment, the SLAM initial position calculation unit 32 performs movement compensation of the partial point group 21a using the detection result of the IMU 12. Alternatively, the SLAM initial position calculation unit 32 may perform movement compensation of the partial point group 21a using both the detection result of the IMU 12 and the self-position of the train 1 roughly estimated by the self-position rough estimation unit 30. Note that the above-mentioned movement compensation is performed individually for each of the multiple ranging points included in the partial point group 21a.

[0050] Furthermore, when the SLAM initial position calculation unit 32 calculates the above-mentioned initial position, the SLAM initial position calculation unit 32 converts the partial point group 21a into the coordinate system of the past entire point group 21b using the line-of-sight control information. This improves the convergence of the self-position estimation (ICP processing). By improving the convergence of the self-position estimation by the SLAM processing unit 33, the time required for the self-position estimation by the SLAM processing unit 33 is shortened, and self-position estimation with good real-time performance is realized.

[0051] The accuracy of the self-location estimation by the SLAM processing unit 33 is higher than that of the self-location estimation by the rough self-location estimation unit 30. While the error in the self-location estimation by the rough self-location estimation unit 30 is expected to be on the order of several tens of centimeters, the accuracy of the self-location estimation by the SLAM processing unit 33 can be expected to be on the order of several centimeters.

[0052] Furthermore, when the SLAM initial position calculation unit 32 calculates the above-mentioned initial position, the SLAM initial position calculation unit 32 may further take into account the detection result of the azimuth sensor 14. This further improves the convergence of the above-mentioned self-position estimation (ICP processing). Note that it is also possible to investigate in advance the azimuth of the leading car of the train 1 corresponding to the sleeper count number, and include the correspondence between the sleeper count number and the azimuth angle in the reference table 20a. In this case, the azimuth sensor 14 can be omitted.

[0053] The obstacle detection unit 34 detects obstacles in the traveling environment of the train 1 based on the point cloud of the latest traveling environment of the train 1 generated by the SLAM processing unit 33. Various methods may be adopted as a method for detecting obstacles by the obstacle detection unit 34. For example, the obstacle detection unit 34 can detect an obstacle by comparing the past entire point cloud 21b with the latest entire point cloud 21c. In addition, the obstacle detection unit 34 may specify a swept volume through which the train 1 passes based on the latest entire point cloud 21c, and detect an obstacle by detecting an object within the swept volume using PointNet or the like.

[0054] When the obstacle detection unit 34 detects an obstacle, the vehicle control unit 35 outputs a control signal to the braking device 5 to stop the train 1. Alternatively, when the obstacle detection unit 34 detects an obstacle, the vehicle control unit 35 may output a warning signal to an output device (not shown). The output device is typically a display or a speaker.

[0055] Next, a description will be given of the control flow of the forward monitoring device 3. Fig. 6 shows the control flow of the forward monitoring device 3. The forward monitoring device 3 repeatedly executes the control flow shown in Fig. 6 while traveling.

[0056] First, the sleeper sensor 13 detects a sleeper (S100). Next, the sleeper counting unit 29 counts the number of detections of the sleeper sensor 13 (S110). Next, the line of sight determination unit 31 refers to the reference table 20a to acquire line of sight control information corresponding to the current number of detections (S120). Next, the line of sight control device 11 controls the line of sight of the LiDAR device 10 based on the line of sight control information output from the line of sight determination unit 31 (S130). Next, the LiDAR device 10 measures the distance ahead in the traveling direction of the train 1 (S140). Next, the SLAM initial position calculation unit 32 determines whether the obstacle detection timing has arrived (S150). The obstacle detection timing is set, for example, every 100 milliseconds. If the SLAM initial position calculation unit 32 determines that the obstacle detection timing has not arrived (S150: NO), the process returns to step S100. On the other hand, when the SLAM initial position calculation unit 32 determines that the obstacle detection timing has arrived (S150: YES), the process proceeds to step S160. Next, the SLAM initial position calculation unit 32 calculates the position of the partial point cloud 21a relative to the past whole point cloud 21b as an initial position (S160). Next, the SLAM processing unit 33 uses the above-mentioned initial position calculated by the SLAM initial position calculation unit 32 to estimate the self-position of the train 1 using SLAM and constructs a point cloud of the latest running environment of the train 1 (S170). Next, the obstacle detection unit 34 detects an obstacle in the running environment of the train 1 based on the point cloud of the latest running environment of the train 1 generated by the SLAM processing unit 33 (S180). When the obstacle detection unit 34 detects an obstacle (S180: YES), the vehicle control unit 35 outputs a control signal to the braking device 5 to stop the train 1 (S190). On the other hand, if the obstacle detection unit 34 has not detected an obstacle (S180: NO), the vehicle control unit 35 returns the process to step S100.

[0057] The first embodiment of the present disclosure has been described above. The first embodiment has the following features.

[0058] That is, as shown in FIG. 2, the forward monitoring device 3 (forward monitoring system) includes a sleeper sensor 13, a LiDAR device 10, a line of sight determination unit 31, a line of sight control device 11, and a reference table storage unit 20. The sleeper sensor 13 (sleeper detection means) detects the sleeper 2a. The LiDAR device 10 (sensing means) senses the front of the train 1 in the traveling direction. The line of sight determination unit 31 and the line of sight control device 11 constitute a sensing control means that controls the line of sight (sensing direction) of the LiDAR device 10. The reference table storage unit 20 (sensing control information storage means) stores a reference table 20a (sensing control information) that associates the number of times the sleeper 2a is detected with the line of sight of the LiDAR device 10. The line of sight determination unit 31 determines the line of sight of the LiDAR device 10 based on the number of times the sleeper 2a is detected and the reference table 20a. According to the above configuration, the sensing direction of the sensing means such as the LiDAR device can be suitably controlled.

[0059] The LiDAR device 10 is configured to generate a three-dimensional point cloud in the direction ahead of the train 1 .

[0060] The forward monitoring device 3 further includes a SLAM processing unit 33 (self-position estimation means), a reference table storage unit 20 (position reference information storage means), and a SLAM initial position calculation unit 32 (initial position determination means). The SLAM processing unit 33 estimates the self-position of the train 1 by aligning a partial point cloud 21a (partial three-dimensional point cloud) as a three-dimensional point cloud generated by the LiDAR device 10 with a past entire point cloud 21b (entire three-dimensional point cloud) which is a three-dimensional point cloud of the running environment of the train 1. The reference table storage unit 20 stores a reference table 20a (position reference information) that associates the number of detections of the sleeper 2a with the position of the train 1. The SLAM initial position calculation unit 32 determines the initial position of the partial point cloud 21a with respect to the past entire point cloud 21b. Then, the SLAM initial position calculation unit 32 estimates the current position of the train 1 based on the number of times the sleeper 2a is detected and the reference table 20a, and determines the initial position of the partial point cloud 21a with respect to the past entire point cloud 21b based on the estimated current position and the line of sight (line of sight control information) of the LiDAR device 10 determined by the line of sight determination unit 31. According to the above configuration, the convergence of the self-position estimation by the SLAM processing unit 33 can be significantly improved.

[0061] Moreover, the forward monitoring device 3 includes a SLAM initial position calculation unit 32 (movement compensation means) that performs movement compensation for the partial point group 21a. According to the above configuration, movement compensation for the partial point group 21a is realized.

[0062] Moreover, the line of sight (sensing direction) of the LiDAR device 10 means the central axis of sensing by the LiDAR device 10. In other words, the line of sight (sensing direction) of the LiDAR device 10 means an average emission direction composed of an average value of the horizontal angle and an average value of the vertical angle of the laser light emitted from the LiDAR device 10.

[0063] The forward monitoring device 3 further includes a SLAM processing unit 33 (reconstruction means) that reconstructs a latest overall point cloud 21c (overall three-dimensional point cloud) based on the partial point cloud 21a, and an obstacle detection unit 34 (obstacle detection means) that detects an obstacle ahead in the traveling direction of the train 1 based on the reconstructed latest overall point cloud 21c. With the above configuration, an obstacle ahead in the traveling direction of the train 1 can be detected in real time.

[0064] The forward monitoring device 3 further includes a vehicle control unit 35 (action execution means) that executes a predetermined action in response to the detection of an obstacle by the obstacle detection unit 34. The predetermined action typically includes stopping the train 1. With the above configuration, it is possible to avoid a collision between the train 1 and the obstacle.

[0065] Next, a hardware configuration of the forward monitoring device 3 will be described. In the forward monitoring device 3, the sleeper counting unit 29, the self-position rough estimating unit 30, the line of sight determining unit 31, the SLAM initial position calculating unit 32, the SLAM processing unit 33, the obstacle detecting unit 34, and the vehicle control unit 35 are realized by processing circuits. The reference table storage unit 20 and the point cloud storage unit 21 are realized by storage circuits. The processing circuit may be a processor and memory that executes a program stored in a memory, or may be dedicated hardware.

[0066] FIG. 7 is a diagram showing an example in which the processing circuit of the forward monitoring device 3 is configured with a processor and a memory. When the processing circuit is configured with a processor 1000 and a memory 1001, each function of the processing circuit of the forward monitoring device 3 is realized by software, firmware, or a combination of software and firmware. The software or firmware is described as a program and stored in the memory 1001. In the processing circuit, each function is realized by the processor 1000 reading and executing the program stored in the memory 1001. That is, the processing circuit includes a memory 1001 for storing a program that results in the processing of the forward monitoring device 3 being executed. It can also be said that these programs cause a computer to execute the procedures and methods of the forward monitoring device 3.

[0067] Here, the processor 1000 may be a CPU (Central Processing Unit), a processing device, an arithmetic device, a microprocessor, a microcomputer, or a DSP (Digital Signal Processor), etc. Also, the memory 1001 may be, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable ROM), an EEPROM (Electrically EPROM), or other non-volatile or volatile semiconductor memory, a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, or a DVD (Digital Versatile Disc), etc.

[0068] Fig. 8 is a diagram showing an example in which the processing circuit of the forward monitoring device 3 is configured with dedicated hardware. When the processing circuit is configured with dedicated hardware, the processing circuit 1002 shown in Fig. 33 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination of these. Each function of the forward monitoring device 3 may be realized by the processing circuit 1002 on a function-by-function basis, or each function may be realized collectively by the processing circuit 1002.

[0069] It should be noted that the functions of the forward monitoring device 3 may be partially realized by dedicated hardware and partially realized by software or firmware. In this manner, the processing circuit can realize each of the above-mentioned functions by dedicated hardware, software, firmware, or a combination of these.

[0070] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure.

[0071] For example, the reference table 20a is required to be available in real time, and therefore needs to be stored in the forward monitoring device 3. However, it is not necessary for all the reference tables 20a to be stored in the forward monitoring device 3. For example, it is possible to download the latest reference table 20a when the train stops at a station.

[0072] Furthermore, the line of sight of the LiDAR device 10 does not necessarily have to be directed toward a track structure 2 that is 500 meters or more away from the train 1. For example, in a location where there are blind spots caused by buildings on both sides of the track, it is not possible to monitor a track structure 2 that is 500 meters or more away from the train 1 in the first place, so the line of sight of the LiDAR device 10 may be controlled to be directed toward a track structure 2 immediately in front of the blind spot.

[0073] The line of sight of the LiDAR device 10 may also be directed at a specific target near the tracks, such as a special traffic light.

[0074] In addition, the deviation of the line of sight of the LiDAR device 10 increases the farther away it is, so even if the deviation is as small as 0.5 degrees, it becomes a deviation of 4.3 meters at a distance of 500 meters, which cannot be ignored. Therefore, depending on the vehicle, it is possible that the line of sight control information contains errors due to differences in the installation conditions of the LiDAR device 10. In response to this, the error of the line of sight control information may be calculated using the SLAM result for each vehicle, and the reference table 20a may be updated appropriately. In other words, as long as there is IMU information and line of sight control information, ideally, train movement compensation is possible without matching the point cloud. However, since it actually contains errors, the residual is corrected by SLAM point cloud matching. Therefore, by statistically processing the SLAM result (point cloud movement amount / rotation amount that minimizes the point cloud matching error), it is possible to extract error information such as, for example, an average horizontal line of sight control angle of 3 degrees. If the line of sight control information recorded in the reference table storage unit 20 is corrected based on these statistical results, a more accurate point cloud can be generated, improving the accuracy of forward monitoring.

[0075] Each drawing is merely an example for describing one or more embodiments. Each drawing may relate not only to one particular embodiment, but to one or more other embodiments. As can be understood by those skilled in the art, various features or steps described with reference to any one drawing may be combined with features or steps shown in one or more other drawings to create an embodiment not explicitly shown or described, for example. Not all features or steps shown in any one drawing are necessary to describe an exemplary embodiment, and some features or steps may be omitted. The order of steps described in any drawing may be changed as appropriate.

[0076] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes. (Appendix 1) A sleeper detection means for detecting a sleeper; A sensing means for sensing a forward direction of a train; a sensing control means for controlling a sensing direction of the sensing means; a sensing control information storage means for storing sensing control information that associates the number of times the sleeper is detected with the sensing direction; Including, the sensing control means determines the sensing direction based on the number of times the sleeper is detected and the sensing control information. Forward monitoring system. (Appendix 2) 2. A forward monitoring system according to claim 1, The sensing means is configured to generate a three-dimensional point cloud ahead of the train in a direction of travel. Forward monitoring system. (Appendix 3) 3. A forward monitoring system according to claim 2, a self-position estimation means for estimating a self-position of the train by aligning the partial three-dimensional point cloud as the three-dimensional point cloud generated by the sensing means with an overall three-dimensional point cloud which is a three-dimensional point cloud of the running environment of the train; a position reference information storage means for storing position reference information that associates the number of times the sleepers are detected with the position of the train; an initial position determination means for determining an initial position of the partial three-dimensional point cloud with respect to the overall three-dimensional point cloud when aligning the partial three-dimensional point cloud with respect to the overall three-dimensional point cloud; Further comprising: the initial position determination means estimates a current position of the train based on the number of times the sleepers are detected and the position reference information, and determines the initial position based on the estimated current position and the sensing direction determined by the sensing control means. Forward monitoring system. (Appendix 4) 4. A forward monitoring system according to claim 3, A motion compensation unit for performing motion compensation on the partial three-dimensional point cloud is further included. Forward monitoring system. (Appendix 5) 2. A forward monitoring system according to claim 1, The sensing direction of the sensing means is a central axis of sensing by the sensing means. Forward monitoring system. (Appendix 6) 4. A forward monitoring system according to claim 3, A reconstruction means for reconstructing the entire 3D point cloud based on the partial 3D point cloud; an obstacle detection means for detecting an obstacle ahead of the train in a traveling direction based on the reconstructed overall 3D point cloud; Further comprising: Forward monitoring system. (Appendix 7) 7. The forward monitoring system according to claim 6, and an action execution means for executing a predetermined action in response to the obstacle detection means detecting an obstacle. Forward monitoring system. (Appendix 8) A sleeper detection means for detecting a sleeper; A sensing means for sensing a forward direction of a train; a sensing control means for controlling a sensing direction of the sensing means; a sensing control information storage means for storing sensing control information that associates the number of times the sleeper is detected with the sensing direction; Including, the sensing control means determines the sensing direction based on the number of times the sleeper is detected and the sensing control information. Forward monitoring device. (Appendix 9) Detect sleepers, Sensing the area ahead in the direction of train travel, Controlling the sensing direction of the sensing means; storing sensing control information that associates the number of times the sleeper is detected with the sensing direction; Controlling the sensing direction of the sensing means includes determining the sensing direction based on the number of detections of the sleeper and the sensing control information. Forward monitoring method. (Appendix 10) A program for causing a computer to execute the forward monitoring method described in appendix 9.

[0077] Some or all of the elements (e.g., configurations and functions) described in Supplementary Notes 2 to 7 that are dependent on Supplementary Note 1 may also be dependent on Supplementary Notes 8, 9, and 10 in the same dependent relationship as Supplementary Notes 2 to 7. Some or all of the elements described in any Supplementary Note may be applied to various hardware, software, recording means for recording software, systems, and methods. [Explanation of symbols]

[0078] 1 Train 1a Direction of travel 2 Track structure 2a Sleepers 2b Rail 3 Forward monitoring device 4. Drive unit 5 Braking device 6 wheels 10 LiDAR device 10a line of sight 11 Eye gaze control device 12 IMU 13 Sleeper sensor 14 Azimuth sensor 20 Reference table storage unit 20a Reference Table 21 Point cloud storage 21a Partial point cloud 21b Past whole point cloud 21c Latest whole point cloud 29 Sleeper Counting Department 30 Self-position coarse estimator 31 Line of sight determination part 32 SLAM initial position calculation unit 33 SLAM processing section 34 Obstacle detection section 35 Vehicle control unit

Claims

1. A sleeper detection means for detecting a sleeper; A sensing means for sensing a forward direction of a train; a sensing control means for controlling a sensing direction of the sensing means; a sensing control information storage means for storing sensing control information that associates the number of times the sleeper is detected with the sensing direction; Including, The sensing control means determines the sensing direction based on the number of times the sleeper is detected and the sensing control information. Forward monitoring system.

2. 2. The forward monitoring system according to claim 1, The sensing means is configured to generate a three-dimensional point cloud ahead of the train in a direction of travel. Forward monitoring system.

3. 3. The forward monitoring system according to claim 2, a self-position estimation means for estimating a self-position of the train by aligning the partial three-dimensional point cloud as the three-dimensional point cloud generated by the sensing means with an overall three-dimensional point cloud which is a three-dimensional point cloud of the running environment of the train; a position reference information storage means for storing position reference information that associates the number of times the sleepers are detected with the position of the train; an initial position determination means for determining an initial position of the partial three-dimensional point cloud with respect to the overall three-dimensional point cloud when aligning the partial three-dimensional point cloud with respect to the overall three-dimensional point cloud; Further comprising: the initial position determination means estimates a current position of the train based on the number of times the sleepers are detected and the position reference information, and determines the initial position based on the estimated current position and the sensing direction determined by the sensing control means. Forward monitoring system.

4. 4. The forward monitoring system according to claim 3, A motion compensation unit for performing motion compensation on the partial three-dimensional point cloud is further included. Forward monitoring system.

5. 2. The forward monitoring system according to claim 1, The sensing direction of the sensing means is a central axis of sensing by the sensing means. Forward monitoring system.

6. 4. The forward monitoring system according to claim 3, A reconstruction means for reconstructing the entire 3D point cloud based on the partial 3D point cloud; an obstacle detection means for detecting an obstacle ahead of the train in a traveling direction based on the reconstructed overall three-dimensional point cloud; Further comprising: Forward monitoring system.

7. 7. The forward monitoring system according to claim 6, The obstacle detection means further includes an action execution means for executing a predetermined action in response to the detection of an obstacle by the obstacle detection means. Forward monitoring system.

8. A sleeper detection means for detecting a sleeper; A sensing means for sensing a forward direction of a train; a sensing control means for controlling a sensing direction of the sensing means; a sensing control information storage means for storing sensing control information that associates the number of times the sleeper is detected with the sensing direction; Including, The sensing control means determines the sensing direction based on the number of times the sleeper is detected and the sensing control information. Forward monitoring device.

9. Detect sleepers, Sensing the area ahead in the direction of train travel, Controlling a sensing direction of the sensing; storing sensing control information that associates the number of times the sleeper is detected with the sensing direction; Controlling the sensing direction includes determining the sensing direction based on the number of times the sleeper is detected and the sensing control information. Forward monitoring method.

10. A program for causing a computer to execute the forward monitoring method according to claim 9.

Images