Obstacle detection system and train equipped with the same

Abstract

To provide a technology capable of continuously performing operation of a vehicle even in the case that a malfunction occurs in a portion of a plurality of sensors.SOLUTION: A representative obstacle detection device according to the present invention is an obstacle detection system having M sensors and a signal processing part corresponding to the M sensors, and includes an m-th sensor for detecting an M-th detection range being a range having expansion in the direction of travel being a direction in which the vehicle heads toward a course, and a controller for controlling the m-th sensor and the signal processing part. The controller performs at least either change processing of first change processing for making a first end part approach the vehicle or second change processing for keeping a second end part away from the vehicle, and changes an obstacle detection range by changing at least a p-th detection range.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an obstacle detection system and a train equipped with the same. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there is known a technique for detecting an abnormality in an obstacle sensor used in an obstacle detection system for a vehicle.

[0003] Patent document 1 discloses a driving assistance device that includes a monitoring unit that monitors the operation / non-operation of a sensor that can be mounted on a vehicle, and an output unit that outputs information on the operation / non-operation monitored by the monitoring unit, wherein the monitoring unit detects a malfunction of the sensor based on the detection accuracy of the sensor that is input when the sensor is operating, and the output unit outputs information on the malfunction in addition to the operation / non-operation information when the monitoring unit detects a malfunction of the sensor. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2017-178267 A Summary of the Invention [Problem to be solved by the invention]

[0005] Here, when applying an obstacle detection system to a moving body such as a train that requires a long braking distance, it is possible to divide the detection section in the moving direction of the moving body, assign a sensor to each detection section, and ensure the detection range up to a position far from the moving body. In the event of a sensor malfunction, it is possible to substitute the detection range, but Patent Document 1 does not fully consider this point of view.

[0006] Therefore, an object of the present invention is to provide a technique that enables a vehicle to continue operating even if some of the multiple sensors malfunction. [Means for solving the problem]

[0007] In order to solve the above problems, a representative obstacle detection system of the present invention is an obstacle detection system having M (M is a positive integer of 2 or more) sensors and a signal processing unit corresponding to the M sensors in order to detect an obstacle in a path of a vehicle, and includes an m-th (m is an integer of 1 to M) sensor that detects an m-th detection range, which is a range that extends in a traveling direction, which is a direction in which the vehicle moves toward the path, and a controller that controls the m-th sensor and the signal processing unit, and the signal processing unit performs signal processing to detect an obstacle based on an output signal of the m-th sensor, the obstacle detection range, which is the range of the entire obstacle detection system, is configured to include the m-th detection range, and the distance from the m+1 detection range to the vehicle is greater than the distance from the m-th detection range to the vehicle, and the controller When it is determined that the nth sensor (n is an integer between 1 and M) has malfunctioned, the obstacle detection range is changed by performing at least one of a first change process that moves the first end closer to the vehicle or a second change process that moves the second end farther from the vehicle, where an end of the pth detection range of a pth sensor (p is an integer between 1 and M other than n) that is closest to the vehicle is defined as a first end and an end of the pth detection range that is farthest from the vehicle is defined as a second end, based on at least one of the nth detection range of the nth sensor, the presence or absence of a human alternative means of obstacle detection, the traveling speed of the vehicle, and the planned traveling speed of the vehicle after a certain period of time or after traveling a certain distance. Effect of the Invention

[0008] According to the present invention, even if a malfunction occurs in some of the multiple sensors, it is possible to continue operating the vehicle. Problems, configurations and effects other than those described above will become apparent from the description of the following embodiments. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of an obstacle detection system and a vehicle equipped with the system. [Diagram 2] FIG. 2 is a diagram showing a schematic arrangement of the normal detection ranges of the sensors. [Diagram 3] FIG. 3 is a diagram showing the detection range when a malfunction occurs in the millimeter wave radar sensor. [Figure 4] FIG. 4 is a diagram showing a case where the detection range of the stereo camera sensor and the detection range of the monocular camera sensor are changed. [Diagram 5] FIG. 5 is a diagram showing a case where the detection range of the LiDAR sensor becomes an undetectable range and the detection range of the stereo camera sensor is expanded. [Figure 6] FIG. 6 is a diagram showing a case where the detection range of the monocular camera sensor becomes an undetectable range and the detection range of the stereo camera sensor is expanded. [Figure 7] FIG. 7 is a diagram showing a case where the detection range of the stereo camera sensor becomes an undetectable range and the detection range of the monocular camera sensor is expanded. [Figure 8] FIG. 8 is a diagram showing the configuration of a signal processing means of the stereo camera sensor. [Figure 9] FIG. 9 is a flowchart showing the operation performed by the controller of the obstacle detection system. [Figure 10] FIG. 10 is a diagram showing a schematic diagram of an image that has been subjected to the cutting and compression processing by the cutting and compression means. [Figure 11] FIG. 11 is a diagram showing an example of an image that has been cut and compressed by the cut and compress means when the detection range is changed. [Figure 12] FIG. 12 is a diagram showing the angle of view of the stereo camera. [Figure 13]FIG. 13 is a diagram showing a case where the direction of the optical axis of one of the cameras constituting the stereo camera is changed. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, examples will be described with reference to the drawings. Note that the present invention is not limited to these embodiments. In addition, in the description of the drawings, the same parts are denoted by the same reference numerals.

[0011] In this disclosure, the term "detection range" refers to a range in which information such as the position and speed of an obstacle can be obtained using a detection device such as a sensor. Taking a stereo camera as an example of a detection device, the detection range is determined through the functions of the cameras constituting the stereo camera, the sensor that outputs the captured image as a signal, and the signal processing means that processes the output signal of the sensor and identifies the obstacle. Such a detection range is determined by the detection device and is referred to as the "detection range of the stereo camera" or the "detection range of the sensor of the stereo camera." Furthermore, "the detection range includes the range from distance x1 to distance x3" means that the boundaries of the detection range, x1 and x3, are also included.

[0012] [Example 1] In the first embodiment, an example of an obstacle detection system and a vehicle equipped with the same will be described. Fig. 1 is a diagram showing a schematic configuration of an obstacle detection system and a vehicle equipped with the same. First, each functional block will be described.

[0013] (composition) FIG. 1 shows a case where an obstacle detection system is mounted on a vehicle. The obstacle detection system 100 detects an obstacle on the path of the vehicle 400. The vehicle 400 is, for example, a train. The obstacle detection system 100 mounted on the vehicle 400 includes a plurality of sensors 111-114. The left camera sensor 111 constitutes the left camera of a stereo camera sensor. The right camera sensor 112 constitutes the right camera of the stereo camera. The left camera sensor 111 and the right camera sensor 112 constitute a stereo camera sensor. The LiDAR sensor 113 constitutes a LiDAR (light detection and ranging). The millimeter wave radar sensor 114 constitutes a millimeter wave radar. The signal processing means 121-124 perform signal processing to detect an obstacle based on the output signals of the sensors 111-114. That is, the signal processing means 121 performs signal processing to detect an obstacle based on the left and right camera images acquired by the left camera sensor 111 and the right camera sensor 112. The switch 130 selectively sends the image signal to the signal processing means 122. The signal processing means 122 performs signal processing to detect an obstacle based on a monocular camera image acquired by either the left camera sensor 111 or the right camera sensor 112. The signal processing means 123 performs signal processing to detect an obstacle based on an output signal of the LiDAR sensor 113. The signal processing means 124 performs signal processing to detect an obstacle based on an output signal of the millimeter wave radar sensor 114. The signal processing means 121 to 124 perform their respective signal processing using the memory 131 and the computational resources of the computing unit 132. The user interface 133 is controlled by the controller 140 and reports to the crew. The controller 140 controls the sensors 111 to 114 and the signal processing means 121 to 124. Other functions of the controller 140 will be described later.

[0014] The left camera sensor 111 and the right camera sensor 112 configure a stereo camera, while being configured to function as a monocular camera by switching the switch 130. This configuration is shown as an example, and the embodiment is not limited to this. A configuration in which a sensor for the stereo camera and a sensor for the monocular camera are provided separately may also be used.

[0015] For example, when the obstacle detection system 100 is installed in the leading vehicle of the vehicle 400, the obstacle detection system 200 is installed in the last vehicle and is used when the vehicle 400 turns around and travels in the opposite direction to detect obstacles. The obstacle detection system 200 has the same components as the obstacle detection system 100. That is, the obstacle detection system 200 includes a plurality of sensors 211-214. The left camera sensor 211 constitutes the left camera of the stereo camera. The right camera sensor 212 constitutes the right camera of the stereo camera. The left camera sensor 111 and the right camera sensor 112 constitute a stereo camera sensor. The LiDAR sensor 213 constitutes a LiDAR (light detection and ranging). The millimeter wave radar sensor 214 constitutes a millimeter wave radar. The signal processing means 121-124 perform signal processing to detect obstacles based on the output signals of the sensors 111-114. That is, the signal processing means 221 performs signal processing to detect an obstacle based on the left and right camera images acquired by the left camera sensor 211 and the right camera sensor 212. The switch 230 selectively sends the image signal to the signal processing means 222. The signal processing means 222 performs signal processing to detect an obstacle based on the monocular camera image acquired by either the left camera sensor 111 or the right camera sensor 112. The signal processing means 223 performs signal processing to detect an obstacle based on the output signal of the LiDAR sensor 113. The signal processing means 224 performs signal processing to detect an obstacle based on the output of the millimeter wave radar sensor 214. The signal processing means 221 to 224 perform their respective signal processing using the memory 231 and the computational resources of the computing unit 232. The user interface 233 is controlled by the controller 240 and reports to the crew. The controller 240 controls the sensors 211 to 214 and the signal processing means 221 to 224. Other functions of the controller 240 will be described later.

[0016] Furthermore, the vehicle controller 300 controls the obstacle detection system 100, the obstacle detection system 200, or a vehicle 400 equipped with these. The vehicle 400 has a communication means for transmitting and receiving information between the controller 140 of the obstacle detection system 100 and the controller 240 of the obstacle detection system 200.

[0017] (Relationships between functions) Next, the relationship between the blocks will be described. For example, when the vehicle 400 is a vehicle at one end of a train, the sensors 111-114 included in the obstacle detection system 100 detect obstacles present in the traveling direction and in the vicinity of the traveling direction when the vehicle 400 runs with the vehicle at the front. The traveling direction is, for example, the direction of the track along which the train runs. The sensors 211-214 included in the obstacle detection system 200 are installed in front of the vehicle at the other end opposite to the vehicle on which the sensors 111-114 included in the obstacle detection system 100 are installed, and when the vehicle runs with the vehicle at the front, they detect obstacles ahead of the traveling direction and in the vicinity. The vehicle 400 does not have to be included in a train. When the train includes only one vehicle, the sensors 111-114 of the obstacle detection system 100 may be installed on the front side of the vehicle, and the sensors 211-214 of the obstacle detection system 200 may be installed on the rear side of the vehicle.

[0018] (Details of each sensor) The left camera sensor 111 and the right camera sensor 112 generate image signals indicating images captured synchronously from different viewpoints, and send the generated image signals to the signal processing means 121. The image signals are also sent to the switch 130, which selectively sends the image signals to the signal processing means 122. The LiDAR sensor 113 sends a signal indicating scattered light detected by the LiDAR to the signal processing means 123. The millimeter wave radar sensor 114 sends a signal indicating electromagnetic waves detected by the millimeter wave radar to the signal processing means 124. The signal processing means 121 to 1124 perform signal processing such as noise removal, grouping, labeling, and tracking using the memory 131 and the calculator 132 based on each input signal, and send object information including the coordinates and speed of the detected object to the controller 140. The controller 140 identifies an obstacle based on the object information sent from each of the signal processing means 121-124, detects the distance to the obstacle, and sends the detected information to the user interface 133 and the vehicle controller 300. While the obstacle detection system 100 is operating, the obstacle detection system 200 may stop operating because it will detect obstacles in places that have already been passed. However, in the event of a sensor malfunction, which will be described later, it is necessary to know the status of both systems, so a state in which communication is possible is maintained between the controller 140, the controller 240, and the vehicle controller 300.

[0019] In the obstacle detection system 100, the signal processing means 121-124 and the controller 140 may have a storage unit that accumulates the input signals sent by the sensors 111-114 and the object information detected from the input signals. It is also possible to detect the direction and moving speed of the object by analyzing the change in the input signal. The obstacle detection system 200 may also have a storage unit.

[0020] (Detection range of sensors in obstacle detection systems) In this way, the obstacle detection systems 100 and 200 are configured with multiple sensors, and therefore robustness against the environment and usage conditions can be ensured. However, because sensors have different obstacle detection distance ranges and viewing angles that are best suited for each sensor depending on their characteristics, the detection range of each sensor is set so that an optimal detection range can be configured.

[0021] (An example of sensor allocation) The normal detection range of each sensor will be described with reference to FIG. 2. FIG. 2 is a diagram showing a schematic arrangement of the normal detection range of each sensor. Sensors 111-114 of the obstacle detection system 100 are installed on a vehicle 400. The detection range of each sensor is indicated by an arrow, and includes detection range 403 of the LiDAR sensor 113, detection range 404 of the millimeter wave radar sensor 114, detection range 401 of the stereo camera sensor (i.e., a sensor including the left camera sensor 111 and the right camera sensor 112), and detection range 402 of the monocular camera sensor (i.e., either the left camera sensor 111 or the right camera sensor 112). The obstacle detection range, which is the detection range of the entire obstacle detection system 100, is configured to include detection ranges 401-404.

[0022] Here, in FIG. 2, the direction in which the vehicle 400 heads toward the path is indicated by an arrow as a traveling direction Td. The m-th (m is an integer between 1 and M, and M is a positive integer of 2 or more) sensor detects an m-th detection range that is a range that spreads in the traveling direction Td. The m-th signal processing means performs signal processing to detect an obstacle based on the output signal of the m-th sensor. The distances from the vehicle 400 in the traveling direction Td are designated x1 (first distance), x2 (second distance), x3 (third distance), ..., the m-th distance (m is an integer between 1 and M, and M is a positive integer of 2 or more), .... The LiDAR sensor 113 (first sensor) detects a detection range 403 (first detection range) that includes at least the range from x1 to x3. In other words, in the detection range 403 of the LiDAR sensor 113, x1 is the end (first end) of the detection range 403 closest to the vehicle 400, and x3 is the end (second end) of the detection range 403 farthest from the vehicle 400. Similarly, the millimeter wave radar sensor 114 (second sensor) detects a detection range 404 (second detection range) including at least a range from x2 to x5. The stereo camera sensor (third sensor) detects a detection range 401 (third detection range) including at least a range from x4 to x7. The monocular camera sensor (fourth sensor) detects a detection range 402 (fourth detection range) including at least a range from x6 to x8. Note that the obstacle detection system 100 is configured to include M sensors and M signal processing means (here, M=4), but the configuration of the sensors and the signal processing means is not limited to this. A configuration may be adopted in which four or more sensors and signal processing means are provided, and a configuration in which signal processing corresponding to M sensors is performed by one signal processing means is also possible.

[0023] In this way, the detection ranges are arranged according to the characteristics of each sensor, and the detection range of the vehicle 400 can be configured as a whole. Here, the detection ranges 403 and 404 have an overlapping range between x2 and x3. The detection ranges 403 and 404 can be said to be arranged continuously. The detection ranges 404 and 401 have an overlapping range between x4 and x5. The detection ranges 401 and 402 have an overlapping range between x6 and x7. For example, if an obstacle is detected by the LiDAR sensor 113 in the overlapping range from x2 to x3 but not by the millimeter wave radar sensor 114, it can be determined that the millimeter wave radar sensor 114 has malfunctioned. By arranging the detection ranges continuously in this way, it is possible to determine the malfunction of the sensor early. Since the obstacle detection systems 100 and 200 are configured by a plurality of sensors, robustness against the environment and use conditions can be ensured.

[0024] Note that what is shown here is an example of the arrangement of the detection ranges, and the order of the detection ranges and the length of the detection distance are not limited to this. Also, here, the detection ranges 401-404 are assumed to include the range between the distances measured from the vehicle 400 in the traveling direction, but the detection ranges 401-404 also have a certain degree of spread in directions other than the traveling direction. For example, if the traveling direction Td is the X-axis direction, the detection ranges 401-404 also have a certain degree of spread in the ±Y direction and ±Z direction.

[0025] (Measures to be taken when some sensors malfunction) When the controller 140 determines that a sensor has malfunctioned, it considers at least one of the following: the detection range of the malfunctioning sensor, the presence or absence of a human alternative means for detecting an obstacle, the running speed of the vehicle 400, and the planned running speed of the vehicle 400 after a certain period of time or after traveling a certain distance. For example, the running speed of the vehicle is limited by the detection range of the obstacle. That is, until the vehicle stops, a stopping distance is required that is the sum of the free running distance where the brakes are not applied and the braking distance where the brakes are applied. Here, since there is a predetermined upper limit value for the deceleration of the vehicle, there is a limit to shortening the braking distance. In addition, there are also limitations in terms of the hardware and software of the sensor in enlarging the detection range. Therefore, in order to stop the vehicle before reaching the obstacle, it is necessary to limit the running speed of the vehicle based on the detection range of the obstacle. In FIG. 2, the upper limit value of the running speed of the vehicle 400 is set based on the farthest x8 of the detection range. The human alternative means will be described later.

[0026] The smaller the detection range, the more the driving speed needs to be limited in order to reduce the stopping distance. Therefore, in the first embodiment, the detection range of the normally functioning sensor is changed to prevent the detection range from becoming smaller. This makes it possible to suppress the decrease in the driving speed.

[0027] (If the mid-range sensor is not functioning) 3 to 6, a change in the detection range of the other normal sensors when any of the sensors malfunctions will be described. FIG. 3 is a diagram showing the detection range when the millimeter wave radar sensor 114 malfunctions. When the millimeter wave radar sensor 114 malfunctions, an undetectable range 409 in which an obstacle cannot be detected occurs between the detection range 403 of the LiDAR sensor 113 and the detection range 401 of the stereo camera sensor. Since the vehicle 400 is moving, the undetectable range 409 is a range that is once detected by the monocular camera sensor or the stereo camera sensor, but after the absence of the obstacle is confirmed, an obstacle may enter the area of ​​the vehicle. Therefore, in this case, the range in which the obstacle detection system 100 can guarantee the presence or absence of an obstacle is equivalent to the detection range 403 by the LiDAR sensor 113, and the upper limit of the vehicle's traveling speed is set based on x3, which is the boundary of the detection range 403. That is, when the controller 140 determines that the millimeter wave radar sensor 114 has malfunctioned, it sets an upper limit for the traveling speed of the vehicle 400 based on the distance x3 from the vehicle 400 to the boundary of the detection range 403.

[0028] Therefore, measures are taken to eliminate the undetectable range. FIG. 4 is a diagram showing a case where the detection range of the stereo camera sensor and the detection range of the monocular camera sensor are changed. Here, when the controller 140 determines that the nth (n is an integer between 1 and M) sensor has malfunctioned, the controller 140 performs a change process to change the pth detection range of the pth (p is an integer between 1 and M excluding n) sensor so that the pth detection range includes at least a part of the nth detection range. That is, a change process (first change process) is performed to bring the end x4 of the detection range 401 closest to the train 400 closer to the train 400, and the detection range 401 is changed to a detection range 401a. As a result, the detection range 401 of the stereo camera sensor is expanded as a whole. Also, when the distance between x5 and x6 is xi1, the detection range 401a after the change of the stereo camera sensor includes the range from x2 to xi1. As a result, the detection range 401a includes the detection range 404 (range from x2 to x5) of the millimeter wave radar sensor 114. Since the detection range of the stereo camera sensor has changed in a direction approaching the vehicle 400, a change process is performed so that the detection range of the stereo camera sensor is supplemented by the monocular camera sensor. Here, when the distance between x7 and x8 is xi2, the changed detection range 402a of the monocular camera sensor includes the range from x5 to xi2. Regarding the obstacle detection range, which is the detection range of the entire obstacle detection system 100, the obstacle detection range after the change process is configured to include the detection range 403, detection range 401a, and detection range 402a, which are arranged consecutively.

[0029] By changing the detection range 401a of the stereo camera sensor and the detection range 402a of the monocular camera sensor in this way, the undetectable range 409 is eliminated by the detection range 401a, and the detection range of the obstacle detection system 100 as a whole can be arranged continuously from the vehicle 400 to the boundary of the detection range 402a at a distance xi2. The controller 140 sets the upper limit of the traveling speed of the vehicle 400 based on the detection range after the change processing. Since the boundary of the detection range of the obstacle detection system 100 as a whole can be limited to narrowing from x8 to xi2, the restriction of the upper limit of the traveling speed of the vehicle 400 can be minimized. The method of shortening the obstacle detection range of the stereo camera may be, for example, by compressing the image according to the degree of shortening. The method of extending the detection range is to increase the magnification of the image, and then perform detection based on a super-resolution image when it is handled by image processing, or perform geometric correction of the stereo camera as necessary when it is handled by an optical system such as a lens. The method of changing the detection range of the stereo camera sensor will be described in detail later.

[0030] The detection areas of the stereo camera sensor and the monocular camera sensor in FIG. 4 are each indicated by two arrows in order to clearly show that the detection range is expanded from that in FIG. 2. This indicates that the operation of the signal processing means 124 of the millimeter wave radar sensor 114 is no longer necessary, and the computational resources of the memory 131 and the computing unit 132 used by the signal processing means 124 are allocated to and used by the signal processing means 121 and 122 of the stereo camera and the monocular camera, making it possible to detect obstacles over a wider range. Such allocation of computational resources can be performed by, for example, the controller 140. For example, the signal processing means 121 to 124 determine that the sensor is no longer capable of detection, send a flag signal indicating this to the controller 140, and stop and release the use of the computational resources. When the controller 140 detects a flag signal from the signal processing means 121-124, it determines that the sensor corresponding to the flag signal has malfunctioned, and preferentially allocates the computational resources allocated to the sensor determined to have malfunctioned to the sensor and the signal processing means of the sensor that are the target of the change processing.

[0031] (Short-range sensor malfunctions) Fig. 5 is a diagram showing a case where the detection range 403 of the LiDAR sensor 113 becomes an undetectable range and the detection range of the stereo camera sensor is expanded. Fig. 5 shows a case where the LiDAR sensor 113 malfunctions, and the detection range 401b of the stereo camera sensor compensates for the undetectable range of the LiDAR sensor 113, i.e., the detection range 401 including the range from x1 to x3. The operation of the stereo camera in this case is shown in Fig. 8. Note that the detection range 401b from x1 to x3 and the detection range 401 from x4 to x7 are assigned to the stereo camera sensor, while the range from x3 to x4 is excluded from the detection range of the stereo camera sensor. The computational resources assigned to the malfunctioning LiDAR sensor 113 are also assigned to the stereo camera sensor, but in the example shown in Fig. 5, it is shown that the computational resources assigned to the stereo camera sensor are not sufficient to make the entire range from x1 to x7 the detection range. Since the processing load increases if the range from x3 to x4 is also detected by the stereo camera sensor, the detection range of the stereo camera sensor is limited to the detection range 401b and the detection position range 401.

[0032] 8 is a diagram showing the configuration of the signal processing means 121 of the stereo camera sensor. In normal operation, the stereo camera performs geometric correction on the image signal Sl based on the left camera and the image signal Sr based on the right camera using geometric correction means 1211 and 1212, and then cuts out the geometrically corrected corrected image and appropriately compresses it by cutting out the necessary areas using cutting and compression means 1213 and 1214. After that, a parallax image is calculated using stereo matching means 1215 using the left and right cut-out images, and the distance to an obstacle is detected by obstacle detection means 1216 using the parallax image.

[0033] On the other hand, when detecting a detection range 401b near the vehicle using a stereo camera sensor instead of the detection range 401 of the LiDAR sensor 113, the corrected image is cropped using cropping and compression means 1217 and 1218 to crop a required area and compress the image at a compression rate higher than the normal compression rate. This is processing for detecting a distance closer than the normal detection range 401. Thereafter, a stereo matching means 1219 and an obstacle detection means 1220 detect an obstacle at a short distance, and send the obstacle detection information to the downstream controller 140.

[0034] When there are insufficient resources to supplement the undetectable range of the LiDAR sensor 113 in addition to the detection range of the conventional stereo camera sensor, priority is given to detecting an alternative range to the undetectable range of the LiDAR sensor 113. This is to prevent an undetectable range from remaining between the vehicle 400 and the boundary of the detection range 404 of the millimeter wave radar sensor, which is a distance x2. When there are insufficient resources for signal processing of the normal stereo camera, the detection range 401 of the stereo camera becomes shorter and an undetectable range occurs between x4 and x7, it is possible to supplement the undetectable range by changing the boundary x6 of the detection range 402 of the monocular camera sensor to a point on the vehicle 400 side.

[0035] (If the long range sensor is not functioning) FIG. 6 is a diagram showing a case where the detection range 402 of the monocular camera sensor becomes an undetectable range and the detection range 401 of the stereo camera sensor is expanded. FIG. 6 shows a case where the monocular camera sensor malfunctions. The detection range 401c after the change of the stereo camera sensor is extended to the far side compared to the normal detection range 401. That is, a change process (second change process) is performed so that the end x7 of the detection range 401, which is the farthest from the train 400, is moved away from the train 400, and the detection range 401 is changed to the detection range 401c. As a result, the detection range of the stereo camera sensor 401 is expanded as a whole. That is, when the distance between x7 and x8 is xi3, the detection range 401c after the change of the stereo camera sensor includes the range from x4 to xi3. This is a result of the signal processing means 121 of the stereo camera sensor expanding the detection range of the stereo camera sensor by using the resource that the signal processing means 122 of the monocular camera used. In this way, the detection range 401c of the stereo camera can compensate for a part of the detection range 402 of the monocular camera that has become an undetectable range. Although the entire detection range 402 of the monocular camera is not compensated for, the boundary of the detection range of the obstacle detection system 100 as a whole can be kept between x8 and xi3, so that the limit on the driving speed can be minimized.

[0036] (If the stereo camera is not functioning) FIG. 7 is a diagram showing a case where the detection range 401 of the stereo camera sensor becomes an undetectable range and the detection range of the monocular camera sensor is expanded. FIG. 7 shows a case where the stereo camera composed of the left camera sensor 111 and the right camera sensor 112 malfunctions. The detection range 402a of the monocular camera sensor is expanded to compensate for the range that was the detection range 401 of the stereo camera sensor. That is, when the distance between x7 and x8 is xi4, the changed detection range 402a of the monocular camera sensor includes the range from x4 to xi4. The signal processing means 122 of the monocular camera sensor utilizes the computational resources used by the signal processing means 121 of the stereo camera to expand the detection range of the monocular camera and compensate for a part of the range that became undetectable. In the first embodiment, the input image of the monocular camera is used together with either the left or right image of the stereo camera, so that if the cause of the malfunction of the stereo camera is in the camera unit, the input of the signal processing means 122 is switched by the switch 130. This can be configured so that the switch 130 switches to the signal processing means 122 when an abnormality is detected in at least one of the signals from the left camera sensor 111 and the right camera sensor 112. Also, the signal processing means 121 may send a flag notifying the controller 140 that obstacle detection by the stereo camera is not possible, and the controller 140 may issue an instruction to switch the switch. However, this is not limited to the case where a camera and a sensor dedicated to the monocular camera are provided.

[0037] (Summary of detection range changes) As described above, when some sensors malfunction, the detection range is changed so as to compensate for the range that other normal sensors are unable to detect. That is, the detection range of each sensor is changed so that the detection range of the obstacle detection system 100 as a whole is continuous from the vehicle 400 and extends as far as possible. This is not simply a change in the detection range of the system due to some sensors failing to function, but a change to a new detection range that is different from the detection range that was normally placed in order to compensate for the range that is no longer detectable, and is also a change to compensate for at least a part of the range that is no longer detectable. By carrying out such a change process, it is possible to continue operating the vehicle even if a malfunction occurs in some of the multiple sensors.

[0038] (Handover to crew) The controller 140 recognizes whether or not the human resource (such as the crew of the train 400) can detect an obstacle. When the controller 140 determines that at least one of the M sensors is malfunctioning when the human resource can detect an obstacle, the controller 140 determines whether or not the malfunction range can be compensated for by a healthy sensor excluding the obstacle detection by the human resource, and whether or not the malfunction range is within an acceptable range as an actual damage. As a result, when the human resource needs to detect an obstacle, the controller 140 outputs information to the effect that a handover is to be performed to transfer the operation and range of the obstacle detection to the human resource. Here, the controller 140 outputs information to the effect that a handover is to be performed to transfer the operation and range of the obstacle detection to the human resource, and outputs information indicating that the handover is possible by the human resource, and then executes the handover, or the human resource switches the handover, recognizes the execution of the handover, and transmits the human detection range to the human resource via a user interface such as a monitor or a head mounted display (HMD).

[0039] Here, if the controller 140 recognizes that the obstacle detection in the range that has become impossible to detect can be performed by the train crew, there is no need to change the detection range of the other sensors. If the controller 140 determines that a malfunction has occurred in any of the sensors, it determines whether the detection range of the malfunctioning sensor is within a range that can be substituted (handed over) by the train crew. The criterion for the determination is, for example, whether the detection range is in a position where the distance can be visually determined. Taking the case of FIG. 3 as an example, if the crew can immediately substitute for the detection range of the millimeter wave radar sensor 114, the obstacle detection is handed over to the crew, and the normal sensors are operated as a system without changing the detection range. The controller 140 of the obstacle detection system 100 may share information with the controller 240 and the vehicle controller 300 and know in advance whether there is a crew member who can substitute for the obstacle detection. When substitution is possible, the controller 140 indicates the human detection range to the driver of the vehicle 400 via the user interface 133. After receiving information indicating substitution is possible from the driver via the user interface 133, the controller 140 executes handover regarding obstacle detection. In order to reliably take over monitoring, it is desirable that the handover is triggered by the driver who will be substituting. Note that the human detection range is not limited to the detection range of the sensor that has malfunctioned. The obstacle detection range after the correction process also includes the range in which the driver is responsible for obstacle detection. In addition, since the visible distance is limited, it is also assumed that another sensor A covers the failed range and the driver covers the range of sensor A. For example, the obstacle detection range after the change process includes the human detection range by visual inspection of the human resource.

[0040] At this time, it is important to clearly indicate the monitoring range to the driver who has been entrusted with obstacle detection. For example, the user interface 133 is equipped with a monitor or HMD (, and informs the driver of the detection range by so-called AR technology, which clearly indicates and superimposes the detection range that the driver is responsible for on the real-time forward image. Alternatively, it is also possible to provide the vehicle windshield with a head-up display function and project the detection range onto the windshield.

[0041] (Operation from sensor malfunction to change of detection range) (Failure detection and recovery) FIG. 9 explains the procedure for determining whether a malfunction has occurred and the operation procedure after the determination. FIG. 9 is a flowchart showing the operation performed by the controller 140 of the obstacle detection system 100. The controller 140 determines whether a malfunction has occurred (step 901). The determination of a malfunction can be performed by the sensors 111-114 and the signal processing means 121-124, or by hardware or software not shown. If a malfunction has been determined by either method (yes in step 901), the controller 140 attempts to restore the part where the malfunction has occurred (step 902). If no malfunction has occurred (no in step 901), the controller 140 continues obstacle detection.

[0042] Malfunctions can be classified into two types, one that can be restored immediately and the other that cannot be restored immediately, depending on the cause. For example, the former is a situation where an object that blocks the view is attached to the window glass in front of the camera, or where the headlights or windshield wipers are forgotten to be turned on. Since the crew may be able to remove the cause of such temporary malfunctions, the controller 140 issues a warning by voice or display on the screen via the user interface 133, and assumes that the malfunction has been restored when it is detected that the crew has taken action to remove the cause.

[0043] (Judgment based on vehicle condition) If it is determined that recovery is impossible (step 903), for example, because a certain time has passed without detecting that a recovery operation has been performed, the controller 140 formulates a rearrangement plan for the detection ranges of the remaining normal sensors. In this case, the controller 140 considers, for example, the detection range of the malfunctioning sensor, the presence or absence of a crew member covering the detection range, etc. Furthermore, the controller 140 calculates an upper limit for the running speed of the vehicle based on the rearrangement plan (step 904). This is because, while the service schedule of other vehicles is also adjusted based on the running speed of the vehicle 400, if the vehicle 400 continues to run at a speed exceeding the upper limit, this will affect the service schedule.

[0044] In addition, information such as the presence or absence of a crew member, the vehicle's maximum speed on the track, the current position, map information, and the presence or absence of a crew member may be obtained in advance by the vehicle controller 300 or from an external source and sent to the controllers 140 and 240.

[0045] (Determining the direction of travel) Next, the controller 140 determines the traveling direction (step 905). Usually, when a malfunction is detected in the obstacle detection system, it is considered that the vehicle crew takes over and continues normal operation. However, when the detection range is rearranged, although it is sufficient if the vehicle crew can supplement the detection range by visual inspection, there is a risk that an undetectable range that cannot be detected by visual inspection will remain. Also, when the traveling speed is restricted and there is a high possibility that the operation of other vehicles will be hindered, it is necessary to evacuate the vehicle and evacuate the passengers quickly. In addition, when an obstacle detection system is applied to an autonomous driving system that realizes driverless or license-free operation, where no passengers are driving, if a malfunction is detected in the obstacle detection system, it becomes difficult to continue operation, so after passengers have safely disembarked, the vehicle must be evacuated from the main line so as not to interfere with other services. If the vehicle's speed is restricted or it becomes impossible to drive, not only will it cause inconvenience to passengers, but it will also cause further disruptions to the service schedule.

[0046] Considering these, the controller 140 determines the traveling direction based on map information such as the upper limit of the traveling speed, the current location, the distance to the stations ahead and behind, the presence or absence of passengers, the result of checking the soundness of the obstacle detection system 200 behind, the position where assistance from human resources can be obtained, and other information, while also considering the arrival time to the nearest stations ahead and behind, the presence or absence of crew members at the nearest stations ahead and behind, and the impact on the train schedule. If the traveling direction is not to be changed (no in step 905), the controller 140 compares the calculated upper limit of the traveling speed with the actual speed at the current time, and decelerates if the actual speed is higher (yes in step 906). If deceleration is completed after the actual speed falls below the upper limit of the traveling speed (yes in step 907), the controller 140 performs a process of changing the detection range of each sensor based on the rearrangement plan (step 908).

[0047] On the other hand, if the traveling direction has been changed (yes in step 905), the controller 140 safely stops the vehicle 400 and then starts traveling using the obstacle detection system 200 (step 909). When the vehicle arrives at the destination station, passengers are allowed to disembark or an auxiliary crew member is allowed to board. The controller 140 again formulates and executes a placement plan for the detection range. When formulating a placement plan, the vehicle may be moved to a final repair yard, or may be moved to a shelter temporarily to minimize the impact on the train schedule, and then moved to a final repair yard or garage after business hours.

[0048] When an obstacle detection system is applied to a driverless or license-free vehicle in this way, passengers can be allowed to disembark quickly and the vehicle can be evacuated so as not to disrupt other operations. In addition, since the running speed is limited by the process of changing the detection range, it is desirable to avoid changing the detection range more than necessary. The operation shown in Fig. 9 is performed periodically while the vehicle 400 is in operation. By performing such an operation, it is possible to reliably determine the occurrence of a malfunction and a situation in which the process of changing the detection range is necessary, and it is possible to avoid changing the detection range more than necessary.

[0049] (Details of stereo camera sensor change process) (When changing the detection range) Here, the details of the process of changing the detection range in the stereo camera will be described with reference to Fig. 10 to Fig. 13. Fig. 10 is a diagram showing an example of an image that has been cut and compressed by the cut and compress means 1213 and 1214. Fig. 11 is a diagram showing an example of an image that has been cut and compressed by the cut and compress means 1217 and 1218 when the detection range is changed.

[0050] FIG. 10 shows an image after the input image to the cropping and compression means 1213 and cropping and compression means 1214 in FIG. 8 has been appropriately cropped. Here, the closer the object is, the greater the disparity obtained from the left and right images, so in order to make the detection range closer, the search range for stereo matching can be widened. However, simply widening the search range increases the processing load and memory usage. On the other hand, the closer the object is, the smaller the effect of disparity error on distance error. Therefore, when changing and widening the detection range, a method of compressing the image as shown in FIG. 11 is appropriate for nearby detection ranges. This is carried out by the cropping and compression means 1217 and cropping and compression means 1218. The calculation formula for obtaining object distance from disparity is shown as follows (Equation 1):

[0051]

number

[0052] Here, L is the object distance, d is the object parallax, pp is the pixel pitch, fc is the focal length of the camera lens, and B is the baseline length of the stereo camera. Here, compression reduces the image size, so the number of pixels decreases and the pixel pitch expands. For example, the parallax obtained from an image compressed to 1 / 2 is d / 2, and the pixel pitch taking compression into account is 2·pp, so if the following (Equation 2) is used, the obtained object distance L will be the same.

[0053]

number

[0054] Therefore, if the obstacle detection means 1220 knows the compression ratio of the parallax images sent from the stereo matching means 1219, it can obtain the object distance appropriately. For example, the controller 140 determines the compression ratio when rearranging the detection range, and performs signal processing in the signal processing means 121 based on the determined compression ratio, thereby obtaining the object distance.

[0055] (Cross-eye control of stereo camera) Correction for tilting the optical axis of the stereo camera will be described with reference to Fig. 12 and Fig. 13. Fig. 12 is a diagram showing the angle of view of the stereo camera. Fig. 13 is a diagram showing a case where the direction of one of the optical axes of the cameras constituting the stereo camera is changed. The controller 140 changes the detection range by performing at least one of the following: changing the magnification of the images acquired by the left and right cameras, changing the search range of stereo matching, or rotating the optical axis of at least one of the left and right cameras inward in the yaw direction.

[0056] In order for detection to be performed by a stereo camera, a three-dimensional object must be reflected in the left and right images. If an extremely close object is detected, the three-dimensional object will not be included in the angle of view of either image. In FIG. 12, a left camera corresponding to the left camera sensor 111 and a right camera corresponding to the right camera sensor 112 are shown. The left camera and the right camera have an angle of view fov. The optical axis of the left camera is oa1, and the optical axis of the right camera is oa2. The optical axes of the left and right cameras are generally installed parallel to each other as shown in FIG. 12, but as shown in FIG. 13, the optical axis of one of the cameras may be directed inward. In this case, it may be difficult to detect objects in the distance, but it is effective when prioritizing the detection of nearby objects. When the optical axis is directed inward, in addition to the normal geometric correction, the geometric correction means 1211 or the geometric correction means 1212 in FIG. 8 requires image correction based on the rotation angle, rotation axis, and image capture surface position. In this case, if the rotation axis is aligned with the entrance pupil position of the camera lens, the correction calculation can be simplified.

[0057] (Modification of detection range change) After the detection range is changed, the detection performance and distance measurement performance of the sensor may be verified in a simple manner. The controller 140 uses a sensor whose detection range includes the first end changed after the first change process, or a sensor whose detection range includes the second end changed after the second change process, to confirm the detection performance in the detection range after the change. For example, as shown in FIG. 4 and FIG. 5, by overlapping the detection range after the change with the detection range of another normal sensor, the detection performance and detection distance performance of the object can be evaluated in the overlapping range. For example, in FIG. 4, the controller 140 evaluates the detection performance and detection distance performance for the detection range 401a after the change process by the LiDAR sensor 113 corresponding to the detection range 403 and the monocular camera sensor corresponding to the detection range 402a.

[0058] (How to determine malfunction) Here is an example of a method for determining whether a malfunction has occurred in a stereo camera or a monocular camera. When the frame rate of the camera is constant, if the output signal sent from the camera is not updated for a certain period of time, there is a possibility that a malfunction has occurred in the camera body or the camera cable. In addition, the controller 140 estimates the cause of the malfunction of the nth sensor based on statistical information such as the average brightness, brightness variance, and brightness histogram of the entire image or each area, and the difference between the statistical information of each area. For example, as a cause of the malfunction of the stereo camera, if the brightness is significantly lower than the brightness information of the acquired image, there may be an obstruction in front of the camera lens. If the brightness is an appropriate value but the brightness variance of the entire screen is small, there may be a snowstorm or backlight. If the difference with adjacent pixels is small or the signal level of the high spatial frequency is small, there may be a possibility that the wipers are not operating even in heavy rain. In this way, the controller 140 or the stereo camera sensor may analyze the image included in the output signal of the stereo camera sensor, estimate the cause of the malfunction, and instruct the driver of the vehicle 400 to take measures to eliminate the cause.

[0059] (If the stereo camera sensor malfunctions) There are cases where the stereo camera sensor malfunctions. For example, let us assume that the left camera sensor 111 and the right camera sensor 112 constitute the stereo camera sensor, and the signal processing unit 121 performs obstacle detection using a monocular image with the left camera sensor 111 as an input. Here, if the left camera sensor 111 malfunctions, the controller 140 causes the right camera sensor 112 to perform obstacle detection using a monocular image as an input. Specifically, the controller 140 identifies the sensor that malfunctions among the left camera sensor 111 and the right camera sensor 112 that constitute the stereo camera. It is also assumed that the signal processing unit 122 performs obstacle detection using a monocular image such as AI. If it is found that one sensor malfunctions and the other sensor does not malfunction, the controller 140 switches the switch 130 so that the sensor that does not malfunction is input to the signal processing unit 122.

[0060] (Action and effect) As described above, in the present disclosure, by changing the detection range of the normally functioning sensor, it is possible to continue driving the vehicle while suppressing the limitation of the driving speed. According to the present disclosure, even if a malfunction occurs in some of the multiple sensors, it is possible to continue driving the vehicle.

[0061] Although the embodiments of the present invention have been described above, the present disclosure is not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present invention. For example, in the first embodiment, the obstacle detection systems 100 and 200 are provided in a vehicle, but are not limited to being mounted on a vehicle. For example, only the sensors 111-114 and the sensors 211-214 may be installed in a vehicle, and the other signal processing means, controller, memory, and CPU may be provided outside the train.

[0062] [Other Examples] The present disclosure also includes the following aspects. (Aspect 1) An obstacle detection system having M sensors (M is a positive integer of 2 or more) and signal processing units corresponding to the M sensors for detecting obstacles in a path of a vehicle, an m-th sensor (m is an integer between 1 and M) configured to detect an m-th detection range, the m-th detection range being a range extending in a traveling direction in which the vehicle travels along the path; a controller that controls the m sensor and the signal processing unit, the signal processing unit performs signal processing to detect an obstacle based on an output signal of the m-th sensor, The obstacle detection range, which is the range of the entire obstacle detection system, is configured to include an m-th detection range, The distance from the (m+1) detection range to the vehicle is greater than the distance from the (m)-th detection range to the vehicle; The controller: When it is determined that the nth (n is an integer between 1 and M) sensor has malfunctioned, Based on at least one of the n-th detection range of the n-th sensor, the presence or absence of a human alternative means for obstacle detection, the traveling speed of the vehicle, and the traveling speed of the vehicle expected after a certain period of time or after traveling a certain distance, In the p-th detection range of a p-th sensor (p is an integer other than n and not less than 1 and not more than M), an end of the p-th detection range closest to the vehicle is defined as a first end, and an end of the p-th detection range farthest from the vehicle is defined as a second end, changing the obstacle detection range by performing at least one of a first change process for moving the first end closer to the vehicle and a second change process for moving the second end away from the vehicle and changing at least the p detection range; An obstacle detection system comprising: (Aspect 2) 2. The obstacle detection system according to claim 1, The changed p-th detection range of the p-th sensor includes at least a portion of the n-th detection range. An obstacle detection system comprising: (Aspect 3) 3. The obstacle detection system according to claim 1, further comprising: The obstacle detection range after the change process has a detection range arranged continuously. An obstacle detection system comprising: (Aspect 4) The obstacle detection system according to any one of aspects 1 to 3, The obstacle detection range after the change process includes the human detection range by visual inspection. An obstacle detection system comprising: (Aspect 5) 5. The obstacle detection system according to any one of aspects 1 to 4, The controller sets an upper limit value of the traveling speed of the vehicle based on the obstacle detection range in which the change process is to be performed. Obstacle detection system. (Aspect 6) 6. The obstacle detection system according to any one of aspects 1 to 5, The controller performs the change process after the actual speed of the vehicle falls below the upper limit value of the traveling speed. An obstacle detection system comprising: (Aspect 7) 7. The obstacle detection system according to any one of aspects 1 to 6, comprising: The signal processing unit further includes a calculation resource used when the signal processing unit performs the signal processing, the controller allocates the computational resources allocated to the signal processing of the nth sensor in which it has been determined that a malfunction has occurred, to the signal processing of the pth sensor which is the target for performing the change processing. An obstacle detection system comprising: (Aspect 8) 8. The obstacle detection system according to any one of aspects 1 to 7, The sensor on which the change process is performed includes a stereo camera sensor including left and right cameras, The controller: The detection range is changed by performing at least one of the following: changing the magnification of the images acquired by the left and right cameras; changing the search range of stereo matching; and rotating the optical axis of at least one of the left and right cameras inward in the yaw direction. An obstacle detection system comprising: (Aspect 9) 9. The obstacle detection system according to any one of aspects 1 to 8, comprising: The controller: A sensor having a detection range including the first end portion changed after the first change process or a sensor having a detection range including the second end portion changed after the second change process is used to confirm detection performance in the p-th detection range after the change. An obstacle detection system comprising: (Aspect 10) 10. The obstacle detection system of any one of aspects 1 to 9, comprising: The controller: Recognizing whether human resources can detect obstacles, When the human resource is capable of detecting an obstacle, if it is determined that at least one of the M sensors is malfunctioning, Output information to the effect that a handover of the obstacle detection operation and range will be performed to a human resource; outputting information indicating that the handover is possible by the human resource and then executing the handover, or performing a handover switch by the human resource and recognizing the execution of the handover; Communicating the human detection range to the human resource via a user interface such as a monitor or a head-mounted display; An obstacle detection system comprising: (Aspect 11) 11. The obstacle detection system according to any one of aspects 1 to 10, The controller: determining that a malfunction has occurred if the output signal of the nth sensor is not updated for a certain period of time; An obstacle detection system comprising: (Aspect 12) 12. The obstacle detection system according to any one of aspects 1 to 11, The controller: A cause of the malfunction of the nth sensor is estimated based on statistical information such as a luminance average value, a luminance variance value, and a luminance histogram for the entire image or for each area, and a difference in statistical information for each area; Instruct the vehicle driver to take measures to eliminate the cause. An obstacle detection system comprising: (Aspect 13) 13. The obstacle detection system of any one of aspects 1 to 12, comprising: the nth sensor and the qth sensor (q is an integer other than n, which is equal to or greater than 1 and equal to or less than M) constitute a stereo camera sensor; When the signal processing unit detects an obstacle using a monocular image with the nth sensor as an input, The controller switches to the monocular image input from the q sensor to perform obstacle detection. An obstacle detection system comprising: (Aspect 14) A train including a first obstacle detection system and a second obstacle detection system, the first obstacle detection system and the second obstacle detection system being the obstacle detection system according to any one of aspects 1 to 13, The first obstacle detection system is provided in a leading car of the train, and the second obstacle detection system is provided in a trailing car of the train, the first obstacle detection system detects an obstacle when the train moves in the forward direction, and the second obstacle detection system detects an obstacle when the train moves in a reverse direction opposite to the forward direction; the train has a communication means for transmitting and receiving information between a controller of the first obstacle detection system and a controller of the second obstacle detection system; If a malfunction is detected in the first obstacle detection system, The controller of the first obstacle detection system confirming the soundness of the second obstacle detection system, and determining whether to change the direction of travel or the position to change the direction of travel based on at least one of information on the confirmed soundness, current location information of the vehicle, route map information, the presence or absence of passengers, a position where assistance from human resources can be obtained, and an upper limit value of the travel speed calculated by a controller of the first obstacle detection system; A train characterized by [Explanation of symbols]

[0063] 100, 200 Obstacle Detection System 111, 211 Left camera sensor 112, 212 Right camera sensor 113, 213 LiDAR sensors 114, 214 Millimeter wave radar sensor 121-124, 221-224 Signal processing means 130, 230 Switch 131, 231 memory 132, 232 Arithmetic unit 133, 233 User Interface 140, 240 Controller 300 Vehicle Controller 400 vehicles 401~404, 401a, 401b, 401c, 402a Detection range 1211, 1212 Geometric correction means 1213, 1214, 1217, 1218 Cutting and compression means 1215, 1219 Stereo matching means 1216, 1220 Obstacle detection means

Claims

1. An obstacle detection system having M sensors (M is a positive integer of 2 or more) and signal processing units corresponding to the M sensors for detecting an obstacle in a path of a vehicle, an m-th (m is an integer between 1 and M) sensor configured to detect an m-th detection range, the m-th detection range being a range extending in a traveling direction in which the vehicle travels along the path; a controller that controls the mth sensor and the signal processing unit, The signal processing unit performs signal processing to detect an obstacle based on an output signal of the mth sensor, An obstacle detection range, which is the range of the entire obstacle detection system, is configured to include an m-th detection range, The distance from the (m+1) detection range to the vehicle is greater than the distance from the m-th detection range to the vehicle, The controller: When it is determined that the nth sensor (n is an integer of 1 to M) has malfunctioned, Based on at least one of the n-th detection range of the n-th sensor, the presence or absence of a human alternative means for obstacle detection, the traveling speed of the vehicle, and the traveling speed of the vehicle expected after a certain period of time or after traveling a certain distance, In the p-th detection range of a p-th sensor (p is an integer other than n and not less than 1 and not more than M), an end of the p-th detection range closest to the vehicle is defined as a first end, and an end of the p-th detection range farthest from the vehicle is defined as a second end, changing the obstacle detection range by performing at least one of a first change process for moving the first end closer to the vehicle and a second change process for moving the second end away from the vehicle, thereby changing at least the p detection range; The obstacle detection range after the change process has a detection range arranged continuously. An obstacle detection system comprising:

2. 2. The obstacle detection system according to claim 1, The changed p-th detection range of the p-th sensor includes at least a portion of the n-th detection range. An obstacle detection system comprising:

3. 2. The obstacle detection system according to claim 1, The obstacle detection range after the change process includes the human detection range by visual inspection. An obstacle detection system comprising:

4. 2. The obstacle detection system according to claim 1, The controller sets an upper limit value of the traveling speed of the vehicle based on the obstacle detection range in which the change process is to be performed. Obstacle detection system.

5. 5. The obstacle detection system according to claim 4, The controller performs the change process after the actual speed of the vehicle falls below the upper limit value of the traveling speed. An obstacle detection system comprising:

6. 2. The obstacle detection system according to claim 1, The signal processing unit further includes a calculation resource used when the signal processing unit performs the signal processing. the controller allocates the computational resources allocated to the signal processing of the nth sensor in which it has been determined that a malfunction has occurred, to the signal processing of the pth sensor which is the target for performing the change processing. An obstacle detection system comprising:

7. 2. The obstacle detection system according to claim 1, The sensor on which the change process is performed includes a stereo camera sensor including left and right cameras, The controller: The detection range is changed by performing at least one of the following: changing the magnification of the images acquired by the left and right cameras; changing the search range of stereo matching; and rotating the optical axis of at least one of the left and right cameras inward in the yaw direction. An obstacle detection system comprising:

8. 2. The obstacle detection system according to claim 1, The controller: A sensor having a detection range including the first end portion changed after the first change process or a sensor having a detection range including the second end portion changed after the second change process is used to confirm detection performance in the p-th detection range after the change. An obstacle detection system comprising:

9. 4. The obstacle detection system according to claim 3, The controller: Recognizing whether human resources can detect obstacles, When the human resource is capable of detecting an obstacle, if it is determined that at least one of the M sensors is malfunctioning, Output information to the effect that a handover of the obstacle detection operation and range will be performed to a human resource; outputting information indicating that the handover is possible by the human resource and then executing the handover, or performing a handover switch by the human resource and recognizing the execution of the handover; Communicating the human detection range to the human resource via a user interface such as a monitor or a head-mounted display; An obstacle detection system comprising:

10. 2. The obstacle detection system according to claim 1, The controller: determining that a malfunction has occurred if the output signal of the nth sensor is not updated for a certain period of time; Obstacle detection system.

11. 2. The obstacle detection system according to claim 1, The controller: A cause of the malfunction of the nth sensor is estimated based on statistical information such as a luminance average value, a luminance variance value, and a luminance histogram for the entire image or for each area, and a difference in statistical information for each area; Instruct the vehicle driver to take measures to eliminate the cause. Obstacle detection system.

12. 2. The obstacle detection system according to claim 1, the nth sensor and the qth sensor (q is an integer other than n, which is equal to or greater than 1 and equal to or less than M) constitute a stereo camera sensor; When the signal processing unit performs obstacle detection using a monocular image with the nth sensor as an input, The controller switches to the monocular image input from the q sensor to perform obstacle detection. Obstacle detection system.

13. A train comprising a first obstacle detection system and a second obstacle detection system, the first obstacle detection system and the second obstacle detection system being the obstacle detection system according to claim 4, The first obstacle detection system is provided in a leading car of the train, and the second obstacle detection system is provided in a rearmost car of the train, the first obstacle detection system detects an obstacle when the train moves in the forward direction, and the second obstacle detection system detects an obstacle when the train moves in a reverse direction opposite to the forward direction; the train has a communication means for transmitting and receiving information between a controller of the first obstacle detection system and a controller of the second obstacle detection system; If a malfunction is detected in the first obstacle detection system, a controller of the first obstacle detection system, confirming the soundness of the second obstacle detection system, and determining whether to change the traveling direction or the position to change the traveling direction based on at least one of information on the confirmed soundness, information on the current location of the vehicle, route map information, the presence or absence of passengers, a position where assistance from human resources can be obtained, and an upper limit value of the traveling speed calculated by a controller of the first obstacle detection system; A train characterized by

Images