Obstacle detection device, program, and obstacle detection method

Abstract

To provide an obstacle detection device, a program, and an obstacle detection method capable of detecting an obstacle in a railroad track or a roadbed regardless of the height, size, and shape of an obstacle.SOLUTION: According to an embodiment, an obstacle detection device includes an acquisition part and an obstacle detection part. The acquisition part acquires an image photographed by a camera for photographing a railroad track including a track on which a train passes or a roadbed including a road on which a vehicle for a BRT passes. The obstacle detection part performs at least either processing for detecting an object in the railroad track or the roadbed by using background difference using the image or processing for detecting the obstacle in the railroad track or the roadbed by using a method for inputting the image in a learned model to acquire an output showing whether the obstacle is reflected in the image.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an obstacle detection device, a program, and an obstacle detection method. [Background technology]

[0002] In some cases, obstacle detection devices are installed on railroad tracks, such as at railroad crossings, to detect obstacles on the railroad tracks. However, existing obstacle detection devices are installed for the purpose of detecting vehicles. For this reason, existing obstacle detection devices are limited in that they can only detect obstacles above a certain height or can only detect obstacles above a certain size. For this reason, existing obstacle detection devices may not be able to detect obstacles depending on their height, size, shape, etc. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-192181 Summary of the Invention [Problem to be solved by the invention]

[0004] The problem that the embodiments of the present invention aim to solve is to provide an obstacle detection device, program, and obstacle detection method that can detect obstacles within a railway or road area regardless of the height, size, shape, etc. of the obstacle. [Means for solving the problem]

[0005] The obstacle detection device of the embodiment includes an acquisition unit and an obstacle detection unit. The acquisition unit acquires an image captured by a camera that captures a railroad track including a track on which trains run or a roadway including a road on which BRT vehicles run. The obstacle detection unit performs at least one of a process of detecting an obstacle on the railroad track or the roadway using background subtraction using the image, and a process of detecting an obstacle on the railroad track or the roadway using a method of inputting the image into a trained model and obtaining an output indicating whether an obstacle is captured in the image. [Brief description of the drawings]

[0006] [Figure 1] 1 is a block diagram showing an example of a detection system according to an embodiment and a main configuration of components included in the detection system; [Diagram 2] 2 is a flowchart showing an example of processing by a processor of the server device in FIG. 1; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, a detection system according to an embodiment will be described with reference to the drawings. Note that, for the sake of explanation, the drawings used in the following description of the embodiment may omit configurations. In addition, in each drawing and in this specification, the same reference numerals denote similar elements. FIG. 1 is a block diagram showing an example of a detection system 1 according to an embodiment and a main configuration of components included in the detection system 1. Note that each component of each device may be built-in or external. The detection system 1 is a system for detecting an obstacle at a railroad crossing. A railroad crossing is typically installed as a level crossing between a railroad and a road. The detection system 1 includes, as an example, a server device 100, a router 200, a camera 300, a railroad crossing relay 400, an obstacle detection relay 500, a special signal relay 600, and a terminal device 700. Note that the detection system 1 may include some of these. Also, the number of each device shown in FIG. 1 is one, but the number of each device is not limited.

[0008] The server device 100 and the router 200 are connected to a network NW1. The network NW1 is typically a communication network including a wide area network (WAN). The network NW1 is typically a communication network including a private network such as an intranet. The network NW1 may be a communication network including a local area network (LAN). The network NW1 may be a communication network including the Internet.

[0009] The router 200 and the camera 300 are connected to a network NW2. The network NW2 is typically a communication network including a LAN. The network NW2 may be a communication network including the Internet. The network NW2 may be a communication network including a WAN.

[0010] The server device 100 and the terminal device 700 are connected to a network NW3. The network NW3 is typically a communication network including the Internet. The network NW3 is typically a communication network including a WAN. The network NW3 may be a communication network including a private network such as an intranet. The network NW3 may be a communication network including a LAN.

[0011] Each of the networks NW1 to NW3 may be a wireless line or a wired line, or may be a mixture of wireless and wired lines. Each of the networks NW1 to NW3 may be a communication network including a dedicated line or a public mobile phone network. Any two or more of the networks NW1 to NW3 may be a common communication network.

[0012] The detection system 1 may include a network device (not shown).

[0013] The server device 100 is a device that performs processing for detecting obstacles in a railroad crossing. The server device 100 also controls the detection system 1. The server device 100 includes, as an example, a processor 101, a ROM (read-only memory) 102, a RAM (random-access memory) 103, an auxiliary storage device 104, and a communication I / F (interface) 105. A bus 106 and the like connect these components together. The server device 100 is an example of an obstacle detection device.

[0014] The processor 101 is a central part of a computer that performs processes such as calculations and control necessary for the operation of the server device 100, and performs various calculations and processes. The processor 101 is, for example, a central processing unit (CPU), a micro processing unit (MPU), a system on a chip (SoC), a digital signal processor (DSP), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field-programmable gate array (FPGA). Alternatively, the processor 101 is a combination of a plurality of these. The processor 101 may also be a combination of these with a hardware accelerator or the like. The processor 101 controls each part to realize various functions of the server device 100 based on programs such as firmware, system software, and application software stored in the ROM 102 or the auxiliary storage device 104. The processor 101 also executes processes described below based on the programs. A part or all of the programs may be incorporated in the circuitry of the processor 101.

[0015] The ROM 102 and the RAM 103 are main storage devices of the computer with the processor 101 at its core. The ROM 102 is a non-volatile memory used exclusively for reading data. The ROM 102 stores, for example, firmware among the above programs. The ROM 102 also stores data used by the processor 101 when performing various processes.

[0016] The RAM 103 is a memory used for reading and writing data. The RAM 103 is used as a work area for storing data that is temporarily used when the processor 101 performs various processes. The RAM 103 is typically a volatile memory.

[0017] The auxiliary storage device 104 is an auxiliary storage device of a computer with the processor 101 at its core. The auxiliary storage device 104 is, for example, an EEPROM (electric erasable programmable read-only memory), a HDD (hard disk drive), or a flash memory. The auxiliary storage device 104 stores, for example, system software and application software among the above programs. The auxiliary storage device 104 also stores data used by the processor 101 in performing various processes, data generated by the processes in the processor 101, various setting values, and the like.

[0018] The application software stored in the auxiliary storage device 104 includes software for controlling the detection system 1 (hereinafter referred to as "detection software").

[0019] The communication I / F 105 is an interface for the server device 100 to communicate via the network NW1, the network NW3, and the like.

[0020] The bus 106 includes a control bus, an address bus, a data bus, etc., and transmits signals exchanged among the various components of the server device 100 .

[0021] The router 200 is a device that connects different communication networks. The router 200 relays communications between the different communication networks. The router 200 also inputs and outputs signals to and from a relay device. For example, one router 200 is installed for each railroad crossing. The router 200 includes, for example, a processor 201, a ROM 202, a RAM 203, an auxiliary storage device 204, a first communication I / F 205, a second communication I / F 206, and a relay I / F 207. A bus 208 and the like connect these components.

[0022] The processor 201 is a central part of a computer that performs various calculations and processes such as calculations and control required for the operation of the router 200. The processor 201 is, for example, a CPU, an MPU, an SoC, a DSP, a GPU, an ASIC, a PLD, or an FPGA. Alternatively, the processor 201 is a combination of a plurality of these. The processor 201 may also be a combination of these with a hardware accelerator. The processor 201 controls each part to realize various functions of the router 200 based on programs such as firmware, system software, and application software stored in the ROM 202 or the auxiliary storage device 204. The processor 201 also executes processes described below based on the programs. A part or all of the programs may be incorporated in the circuitry of the processor 201.

[0023] The ROM 202 and the RAM 203 are main storage devices of the computer with the processor 201 at its core. The ROM 202 is a non-volatile memory used exclusively for reading data. The ROM 202 stores, for example, firmware among the above programs. The ROM 202 also stores data used by the processor 201 when performing various processes.

[0024] The RAM 203 is a memory used for reading and writing data. The RAM 203 is used as a work area for storing data that is temporarily used when the processor 201 performs various processes. The RAM 203 is typically a volatile memory.

[0025] The auxiliary storage device 204 is an auxiliary storage device of a computer with the processor 201 at its core. The auxiliary storage device 204 is, for example, an EEPROM, a HDD, or a flash memory. The auxiliary storage device 204 stores, for example, system software and application software among the above programs. The auxiliary storage device 204 also stores data used by the processor 201 in performing various processes, data generated by the processes in the processor 201, various setting values, and the like.

[0026] The ROM 202 or the auxiliary storage device 204 stores the crossing ID of the crossing where the router 200 is installed. The crossing ID (identifier) ​​is identification information unique to each crossing. The crossing ID may be a router ID. The router ID is identification information unique to each router 200.

[0027] The first communication I / F 205 is an interface for the router 200 to communicate via the network NW1 or the like.

[0028] The second communication I / F 206 is an interface for the router 200 to communicate via the network NW2 or the like.

[0029] The router 200 connects the network NW1 and the network NW2 using a first communication I / F 205 and a second communication I / F 206.

[0030] The relay I / F 207 is an interface for connecting to a relay device. The router 200 receives an input of a signal output by the relay device via the relay I / F 207. The router 200 also outputs a signal to the relay device via the relay I / F 207.

[0031] The bus 208 includes a control bus, an address bus, a data bus, etc., and transmits signals exchanged among the various parts of the router 200 .

[0032] Camera 300 is a camera installed to capture images inside the crossing. A plurality of cameras 300 are installed at each crossing. It is preferable that the plurality of cameras 300 capture images inside the crossing from different directions to eliminate blind spots. Camera 300 also outputs captured image data. The output image data is input to router 200 via network NW2. Note that a moving image is a type of image. Camera 300, for example, continues to capture the moving image.

[0033] The crossing relay 400 is a relay device that outputs a signal indicating the crossing's blocked state. For example, the crossing relay 400 outputs a signal when the crossing is blocked. The crossing relay 400 does not output a signal when the crossing is not blocked. Note that a crossing being blocked means that the crossing is blocked by a crossing gate installed at the crossing. The crossing relay blocks the crossing by, for example, outputting a signal to the crossing gate.

[0034] The obstacle detection relay 500 is a relay device that outputs a signal indicating that an obstacle has been detected in response to receiving an input of a signal (hereinafter referred to as a "detection signal") that is sent when an obstacle is detected. The detection signal may be output by the router 200 or by an existing obstacle detection device. Each device that receives the signal output by the obstacle detection relay 500 executes a predetermined operation to be performed when an obstacle is detected. The device to which the obstacle detection relay 500 outputs the signal may include a special signal relay 600.

[0035] The special signal relay 600 is a relay device that outputs a signal to a special signal emitting device in response to receiving a signal input. The special signal emitting device that receives a signal input from the special signal relay 600 displays a light emitting signal that is a special signal.

[0036] The terminal device 700 is a device that displays various information related to the detection system 1. The terminal device 700 is capable of displaying images captured by the camera 300 in real time. The terminal device 700 may be a general-purpose device such as a PC, a tablet terminal, or a smartphone. The terminal device 700 includes, as an example, a processor 701, a ROM 702, a RAM 703, an auxiliary storage device 704, a communication I / F 705, an input device 706, and a display device 707. A bus 708 and the like connect these components.

[0037] The processor 701 is a central part of a computer that performs processes such as calculations and control necessary for the operation of the terminal device 700, and performs various calculations and processes. The processor 701 is, for example, a CPU, an MPU, an SoC, a DSP, a GPU, an ASIC, a PLD, or an FPGA. Alternatively, the processor 701 is a combination of a plurality of these. The processor 701 may also be a combination of these with a hardware accelerator. The processor 701 controls each unit to realize various functions of the terminal device 700 based on programs such as firmware, system software, and application software stored in the ROM 702 or the auxiliary storage device 704. The processor 701 also executes processes described below based on the programs. A part or all of the programs may be incorporated into the circuitry of the processor 701.

[0038] The ROM 702 and the RAM 703 are main storage devices of the computer with the processor 701 at its core. The ROM 702 is a non-volatile memory used exclusively for reading data. The ROM 702 stores, for example, firmware among the above programs. The ROM 702 also stores data used by the processor 701 when performing various processes.

[0039] The RAM 703 is a memory used for reading and writing data. The RAM 703 is used as a work area for storing data that is temporarily used when the processor 701 performs various processes. The RAM 703 is typically a volatile memory.

[0040] The auxiliary storage device 704 is an auxiliary storage device of a computer with the processor 701 at its core. The auxiliary storage device 704 is, for example, an EEPROM, a HDD, or a flash memory. The auxiliary storage device 704 stores, for example, system software and application software among the above programs. The auxiliary storage device 704 also stores data used by the processor 701 in performing various processes, data generated by the processes in the processor 701, various setting values, and the like.

[0041] The communication I / F 705 is an interface for the terminal device 700 to communicate via the network NW3 or the like.

[0042] The input device 706 accepts operations by an operator of the terminal device 700. The input device 706 is, for example, a keyboard, a keypad, a touchpad, a mouse, a controller, etc. The input device 706 may also be a device for inputting voice.

[0043] The display device 707 displays a screen for notifying the operator of the terminal device 700 of various information. The display device 707 is, for example, a display such as a liquid crystal display or an organic EL (electro-luminescence) display. A touch panel can also be used as the input device 706 and the display device 707. That is, a display panel provided in the touch panel can be used as the display device 707, and a pointing device provided in the touch panel by touch input can be used as the input device 706.

[0044] The bus 708 includes a control bus, an address bus, a data bus, etc., and transmits signals exchanged between each section of the terminal device 700 .

[0045] The operation of the detection system 1 according to the embodiment will be described below with reference to FIG. 2 and other figures. Note that the contents of the processing in the following description of the operation are merely examples, and various processing that can obtain similar results can be used as appropriate. FIG. 2 is a flowchart showing an example of processing by the processor 101 of the server device 100. The processor 101 executes the processing of FIG. 2 based on a program stored in, for example, the ROM 102 or the auxiliary storage device 104. The processor 101 starts the processing shown in FIG. 2, for example, when the detection software is executed. Note that the processor 101 executes the processing of FIG. 2 for each railroad crossing. In the following description, the processing of FIG. 2 performed for one specific railroad crossing will be described. This one specific railroad crossing will be referred to as the "target railroad crossing" hereinafter.

[0046] In step ST11 of FIG. 2, the processor 101 of the server device 100 determines whether the target crossing is blocked or not. To this end, the processor 101, for example, checks whether the crossing relay 400 is outputting a signal by querying the router 200 installed at the target crossing. The processor 101 then determines that the target crossing is blocked if the crossing relay 400 is outputting a signal. On the other hand, the processor 101 determines that the target crossing is not blocked if the crossing relay 400 is not outputting a signal. If the processor 101 determines that the target crossing is not blocked, it determines No in step ST11 and repeats the processing of step ST11. On the other hand, if the processor 101 determines that the target crossing is blocked, it determines Yes in step ST11 and proceeds to step ST12.

[0047] As described above, by performing the processing of step ST11, processor 101 functions as an example of a crossing determination unit that determines whether a railroad crossing is blocked.

[0048] In step ST12, the processor 101 acquires images captured by the multiple cameras 300 installed at the target crossing via the communication I / F 105 and the router 200. It is preferable that the images acquired here are the most recent images. Each image acquired here will be referred to as an "acquired image" below. The processor 101 also stores the acquired images in the auxiliary storage device 104 or the like.

[0049] As described above, the processor 101 functions as an example of an acquisition unit that acquires an image captured by a camera that captures an image inside a railroad crossing by performing the process of step ST12. Alternatively, the processor 101 functions as an example of an acquisition unit by performing the process of step ST12 in cooperation with the communication I / F 105.

[0050] In step ST13, the processor 101 performs image processing on the acquired image to detect a train from within the target crossing. Note that the train is not limited to two or more coupled railcars, but may be a single uncoupled railcar. The processor 101 detects a train within the target crossing by, for example, object recognition. The processor 101 detects a train within the target crossing by, for example, object recognition using deep learning. The processor 101 detects a train, for example, when there is a part in the acquired image that is similar to the feature amount of the train. For example, YOLO or the like can be used as an algorithm for detecting an object.

[0051] The processor 101 performs the process of step ST13 for each of the multiple obtained images. When performing the process of step ST13 for each of the multiple obtained images, the processor 101 may perform the process sequentially, or may perform the process in parallel or in a parallel manner.

[0052] In step ST14, processor 101 determines whether or not a train is present in the target crossing using the processing results of step ST13 for each of the multiple acquired images. For example, processor 101 determines that a train is present in the target crossing when a train is detected in at least one of the multiple acquired images. Alternatively, processor 101 determines that a train is present in the target crossing when a train is detected in at least a predetermined number of the multiple acquired images. The predetermined number is, for example, the minimum number that constitutes a majority. Alternatively, processor 101 determines that a train is present in the target crossing when a train is detected in all of the multiple acquired images.

[0053] In the processing of steps ST13 and ST14, processor 101 may determine whether or not a train is present at the target crossing using a score (hereinafter referred to as the "train score") indicating the likelihood that a train is present at the target crossing. For example, processor 101 calculates the train score in step ST13. Then, processor 101 determines that a train is present at the target crossing if a statistical value such as the sum, average or median of the train scores of multiple acquired images in step ST14 is equal to or greater than a predetermined threshold value TH1.

[0054] As described above, the processor 101 functions as an example of a train detection unit that detects a train at a railroad crossing using an image by performing the process of step ST13. Alternatively, the processor 101 functions as an example of a train detection unit by performing the processes of steps ST13 and ST14.

[0055] In step ST15, processor 101 determines whether or not a train is present in the target crossing in step ST14. If processor 101 does not determine that a train is present in the target crossing in step ST14, it determines No in step ST15 and returns to step ST11. On the other hand, if processor 101 determines that a train is present in the target crossing in step ST14, it determines Yes in step ST15 and proceeds to step ST16.

[0056] In step ST16, processor 101 detects the presence of an obstacle in the target crossing by performing image processing on the acquired image. Processor 101 detects the obstacle by, for example, performing background difference using the acquired image and a background image captured before the acquired image. That is, processor 101 compares the acquired image with the background image to extract a portion that differs by a predetermined amount or more. Then, processor 101 determines that an obstacle exists when the differing portion is extracted. Processor 101 may determine that an obstacle exists when the area of ​​the differing portion is equal to or larger than a predetermined area.

[0057] The processor 101, for example, acquires images captured by the camera 300 under various shooting conditions in advance as background images. The camera 300 is the same as the camera 300 that captures the acquired image. The shooting conditions include the time of year, the time period, the brightness, and the weather at the time of shooting. Furthermore, the background image satisfies the conditions that the crossing is blocked, that no train is present within the crossing, and that no obstacle is present at the time of shooting. The condition that the crossing is blocked can be determined by the same process as step ST11. The condition that no train is present within the crossing can be determined by the same process as steps ST13 and ST14. The condition that no obstacle is present can be determined by the same process as steps ST16 to ST18.

[0058] The processor 101 selects an image captured under shooting conditions similar to those of the captured image from among background images previously acquired. The processor 101 then performs background subtraction using the selected background image and the acquired image. Note that the processor 101 may perform color correction on the selected background image before performing background subtraction to reduce differences from the acquired image due to differences in shooting conditions such as brightness.

[0059] The processor 101 may also perform background subtraction using a background image captured in the past in which the magnitude of change in the image due to changes in the shooting conditions is small enough not to affect the background subtraction. The past image may be, for example, a few frames ago, a few seconds ago, or a few minutes ago. The processor 101 uses, as the background image, an image that satisfies the conditions that the crossing is closed, that there is no train within the crossing, and that there is no obstacle.

[0060] Furthermore, the processor 101 may detect an obstacle by background subtraction using a known method other than the method described above.

[0061] The processor 101 performs the process of step ST16 for each of the multiple obtained images. When performing the process of step ST16 for each of the multiple obtained images, the processor 101 may perform the process sequentially, or may perform the process in parallel or in a parallel manner.

[0062] In step ST17, the processor 101 detects the presence of an obstacle in the target crossing by image processing the acquired image. In step ST17, the processor 101 detects an obstacle by a method different from that in step ST16. The processor 101 detects an obstacle using a trained model trained using an autoencoder. Preferably, the processor 101 detects an obstacle using a trained model trained using a variational autoencoder. The processor 101 generates a trained model in advance using a plurality of images taken by the same camera 300 as the camera 300 that takes the acquired image. It is preferable that the image used for learning satisfies the conditions that the crossing is blocked, that a train is not present in the crossing, and that no obstacle is present. The processor 101 inputs the acquired image to the trained model. When an obstacle is captured in the acquired image, the trained model outputs information indicating that an obstacle has been detected. When an obstacle is not captured in the acquired image, the trained model outputs information indicating that an obstacle has not been detected. The information indicating that an obstacle has been detected may be information indicating that an abnormality has been detected. The information indicating that an obstacle has not been detected may be information indicating that an abnormality has not been detected. The processor 101 regards the information indicating that an abnormality has been detected as information indicating that an obstacle has been detected. The processor 101 regards the information indicating that an abnormality has not been detected as information indicating that an obstacle has not been detected.

[0063] Additionally, the processor 101 may use known techniques related to autoencoders other than the methods described above.

[0064] The processor 101 performs the process of step ST17 for each of the multiple obtained images. When performing the process of step ST17 for each of the multiple obtained images, the processor 101 may perform the process sequentially, or may perform the process in parallel or in a parallel manner.

[0065] The processor 101 may perform the processes of steps ST16 and ST17 in parallel or in parallel.

[0066] In step ST18, the processor 101 uses the processing results of step ST16 for each of the multiple captured images and the processing results of step ST17 for each of the multiple captured images to determine whether or not an obstacle exists in the target crossing. If the number of cameras 300 installed in the target crossing is N, the total number of processing results of step ST16 and step ST17 is 2N. Here, N is an integer equal to or greater than 2. The processor 101 determines that an obstacle exists in the target crossing if, for example, any one or more of the 2N processing results indicate that an obstacle has been detected. Alternatively, the processor 101 determines that an obstacle exists in the target crossing if, for example, a predetermined number or more of the 2N processing results indicate that an obstacle has been detected. The predetermined number is, for example, the minimum number that constitutes a majority. Alternatively, the processor 101 determines that an obstacle exists in the target crossing if, for example, all of the 2N processing results indicate that an obstacle has been detected. Alternatively, the processor 101 may determine that an obstacle is present within the target crossing if the 2N processing results satisfy other predetermined conditions.

[0067] When the processor 101 determines whether or not a predetermined number or more of the 2N processing results indicate that an obstacle has been detected, the processor 101 may weight each of the processing results. As an example, the processing result of step ST16 is weighted by 1.2 processing results, and the processing result of step ST17 is weighted by 0.8 processing results.

[0068] In the processing of steps ST16 to ST18, processor 101 may determine whether or not an obstacle is present within the target crossing using a score (hereinafter referred to as the "obstacle score") indicating the likelihood that an obstacle is present within the target crossing. For example, processor 101 calculates an obstacle score in each of steps ST16 and ST17. Then, in step ST18, processor 101 determines that an obstacle is present within the target crossing if a statistical value such as the sum, average or median of the multiple obstacle scores calculated in steps ST16 and ST17 is equal to or greater than a predetermined threshold value TH2. Processor 101 may weight the obstacle score of each of the 2N processing results.

[0069] As described above, the processor 101 functions as an example of an obstacle detection unit that performs a process of detecting an obstacle in a railroad crossing by using background difference using an image, by performing the process of step ST16. Alternatively, the processor 101 functions as an example of an obstacle detection unit that performs a process of detecting an obstacle in a railroad crossing by using background difference using an image, by performing the processes of step ST16 and step ST18.

[0070] Moreover, by performing the processing of step ST17, the processor 101 functions as an example of an obstacle detection unit that performs processing to detect an obstacle in a railroad crossing using a method of inputting an image into a trained model and obtaining an output indicating whether or not an obstacle is reflected in the image. Alternatively, by performing the processing of steps ST17 and ST18, the processor 101 functions as an example of an obstacle detection unit that performs processing to detect an obstacle in a railroad crossing using a method of inputting an image into a trained model and obtaining an output indicating whether or not an obstacle is reflected in the image.

[0071] In addition, by performing the processing of steps ST16 to ST18, processor 101 functions as an example of an obstacle detection unit that performs processing to detect an obstacle within a railroad crossing by using background subtraction using an image and a method of inputting an image into a trained model and obtaining an output indicating whether or not an obstacle is reflected in the image.

[0072] In step ST19, processor 101 determines whether or not it was determined in step ST18 that an obstacle exists in the target crossing. If processor 101 does not determine in step ST18 that an obstacle exists in the target crossing, it determines No in step ST19 and returns to step ST11. On the other hand, if processor 101 determines in step ST18 that an obstacle exists in the target crossing, it determines Yes in step ST19 and proceeds to step ST20.

[0073] In step ST20, the processor 101 executes a predetermined process (hereinafter referred to as the "predetermined process") to be performed when an obstacle is detected. As the predetermined process, the processor 101 performs a process to stop a train traveling on a line that passes through the target railroad crossing. As the predetermined process, the processor 101 inputs a signal to the obstacle detection relay 500 via the router 200, for example. As the predetermined process, the processor 101 inputs a signal to the special signal relay 700 via the router 200, for example. As the predetermined process, the processor 101 stores information indicating the processing results of steps ST12 to ST19 in the auxiliary storage device 104 or the like.

[0074] As a predetermined process, the processor 101 generates, for example, detection information. The detection information includes the crossing ID of the target crossing, the acquired image, and information indicating the processing results of steps ST12 to ST19. The detection information is information indicating that an obstacle has been detected at the crossing identified by the crossing ID. After generating the detection information, the processor 101 instructs the communication I / F 105 to transmit the detection information to the terminal device 700. Upon receiving this transmission instruction, the communication I / F 105 transmits the detection information to the terminal device 700. The transmitted detection information is received by the communication I / F 705 of the terminal device 700.

[0075] In response to receiving the detection information, the processor 701 of the terminal device 700 notifies that an obstacle has been detected at the crossing identified by the crossing ID included in the detection information. The processor 701 causes the display device 707 to display, for example, an image indicating that an obstacle has been detected at the crossing, an acquired image included in the detection information, and an image including information indicating a processing result included in the detection information. Note that characters are a type of image.

[0076] The processor 101 may also transmit the detection information to a train traveling on a line that passes through the target railroad crossing.

[0077] After processing step ST20, processor 101 returns to step ST11.

[0078] According to the detection system 1 of the embodiment, the server device 100 detects obstacles in the crossing using background subtraction. This allows the server device 100 of the embodiment to detect obstacles that cannot be detected by existing obstacle detection devices. In addition, the method using background subtraction has a high processing speed.

[0079] According to the detection system 1 of the embodiment, the server device 100 detects obstacles in a railroad crossing using the trained model. This enables the server device 100 of the embodiment to detect obstacles that cannot be detected by existing obstacle detection devices.

[0080] According to the detection system 1 of the embodiment, the server device 100 detects obstacles in a railroad crossing using a method using background subtraction and a method using a trained model. This improves the accuracy of obstacle detection in the server device 100 of the embodiment compared to the case where only one of the methods is used.

[0081] Moreover, according to the detection system 1 of the embodiment, the server device 100 detects an obstacle in a railroad crossing using a trained model trained using a variational autoencoder. This allows the server device 100 of the embodiment to detect an obstacle with high accuracy. Furthermore, the server device 100 of the embodiment can reject noise.

[0082] Furthermore, according to the detection system 1 of the embodiment, the server device 100 detects an obstacle in a railroad crossing using a plurality of images captured by a plurality of cameras 300. This allows the server device 100 of the embodiment to detect an obstacle even if the obstacle is located in a blind spot that would be covered by only one camera 300.

[0083] Furthermore, according to the detection system 1 of the embodiment, the server device 100 does not detect an obstacle when the railroad crossing is not blocked. Therefore, the server device 100 of the embodiment can reduce unnecessary processing by not detecting an obstacle while it is not necessary to detect the obstacle.

[0084] Furthermore, according to the detection system 1 of the embodiment, the server device 100 determines that a train is present in the crossing. When a train is present in the crossing, the server device 100 of the embodiment does not detect an obstacle. Therefore, the server device 100 of the embodiment can reduce unnecessary processing by not detecting an obstacle when it is not necessary to detect an obstacle.

[0085] Furthermore, according to the detection system 1 of the embodiment, the server device 100 does not use object recognition for obstacle determination, which allows the server device 100 of the embodiment to detect an obstacle even if it is an unknown object.

[0086] The above embodiment can be modified as follows. In a typical example of the detection system 1 in the above embodiment, the server device 100 performs processing related to obstacle detection at multiple railroad crossings. However, the detection system 1 may include an edge computer instead of the server device 100. For example, one edge computer is installed for each railroad crossing. The edge computer performs the processing of FIG. 2 with the railroad crossing on which it is installed as the target railroad crossing. The edge computer is an example of an obstacle detection device.

[0087] In the above embodiment, the processor 101 of the server device 100 performs a process of detecting an obstacle using two types of methods, background subtraction and autoencoder. However, the processor 101 may perform a process of detecting an obstacle using M types of methods, where M is an integer equal to or greater than 3. In this case, the processor 101 uses M×N processing results in step ST18.

[0088] In the above embodiment, the processor 101 of the server device 100 performs the process of detecting an obstacle using two types of methods, background subtraction and autoencoder. However, the processor 101 may perform the process of detecting an obstacle using only either background subtraction or autoencoder. In this case, the processor 101 uses N processing results in step ST18.

[0089] The processor 101 may use images in which obstacles are present as images used for model training. However, it is preferable to use images in which obstacles are present in supervised learning or self-supervised learning in which a teacher label indicating that the image indicates the presence of an obstacle is added to the images in which obstacles are present.

[0090] The processor 101 may also use images of other crossings to train the learning model for the target crossing. The processor 101 may also use trained models for other crossings to train the learning model for the target crossing.

[0091] The processor 101 may use a trained model trained using a learning method other than the autoencoder. The processor 101 may also use a trained model trained using a combination of multiple learning methods. Examples of learning methods other than the autoencoder include convolutional neural networks and metric learning.

[0092] The processor 101 may detect obstacles by using the trained model as a feature extractor.

[0093] The processor 101 may detect an obstacle using a trained model by using a method other than the above-mentioned method. Examples of the method other than the above-mentioned method include SURF (Speed ​​Up Robust Features) and DOC (Learning Deep Features for One-Class Classification). The processor 101 may also detect an obstacle using a trained model by combining a plurality of methods.

[0094] The generation of the trained model may be performed by a device other than the server device 100.

[0095] One crossing may be provided with one camera 300. In this case, the processor 101 of the server device 100 performs the process in FIG. 2 using one captured image.

[0096] The server device 100 and the router 200 may obtain information indicating the blocking state of the crossing gate from a device other than the railroad crossing relay 400.

[0097] The on / off signal of the crossing relay 400 may be reversed from that of the above embodiment. That is, the crossing relay 400 outputs a signal when the crossing is not blocked. And the crossing relay 400 does not output a signal when the crossing is blocked. Also, the other relay devices may have the on / off signal reversed from that of the above embodiment.

[0098] The detection system 1 may detect an obstacle regardless of whether the crossing is closed or not. In this case, the processor 101 of the server device 100 performs the process of step ST12 without performing the process of step ST11, for example. It is preferable that the background image used when the crossing is closed is one that was photographed when the crossing was closed.

[0099] The detection system 1 may detect an obstacle regardless of whether or not a train is present in the crossing. In this case, the processor 101 of the server device 100 performs, for example, the process of step ST12 and then the process of step ST16. It is preferable that the background image used when a train is present in the crossing is one that is photographed when a train is present in the crossing.

[0100] In the above embodiment, the processor 101 of the server device 100 determines whether the crossing is blocked in step ST11. However, the processor 101 may determine whether the crossing alarm is sounding in step ST11, instead of whether the crossing is blocked. In this case, the router 200 receives a signal input from, for example, a relay device that determines whether a signal is output depending on whether the crossing alarm is sounding, instead of the crossing relay 400. Alternatively, the crossing relay 400 may also function as such a relay device. The detection system 1 can be used at a crossing where there is no barrier and a crossing alarm is installed by using the state of the crossing alarm. In addition, the processor 101 may use both the determination of whether the crossing is blocked and the determination of whether the crossing alarm is sounding. The processor 101 functions as an example of a crossing determination unit that determines whether the crossing alarm is sounding by performing the process of step ST11 that determines whether the crossing alarm is sounding.

[0101] In the above embodiment, the detection system 1 detects obstacles in a railroad crossing. However, the detection system 1 may detect obstacles not only in a railroad crossing but also in the tracks on which a train runs and in the track area including the tracks. The track area includes the tracks between stations, the tracks within the stations, and the grounds through which the tracks between stations pass. As described above, the detection system 1 of the embodiment can detect obstacles not only in a railroad crossing but also in the track area.

[0102] The railroad crossing in the above embodiment is a railroad crossing. However, the detection system of the embodiment can also target a railroad crossing for BRT (bus rapid transit) instead of a railroad crossing. The detection system of the embodiment can also target a BRT vehicle instead of a railroad train. The detection system of the embodiment can also detect an obstacle not only in a railroad crossing but also in a road area including a road exclusively used by BRT vehicles. The road area includes a dedicated road between stations (bus stops), a dedicated road within a station, and a site through which the dedicated road between stations passes. As described above, the detection system of the embodiment can detect an obstacle in a road area through which a BRT road passes.

[0103] In the above embodiment, the detection system 1 detects obstacles within a crossing for each crossing. However, the detection system 1 may also detect obstacles within a crossing for each area. One crossing includes one or more areas. The detection system 1 identifies an area by using an area ID instead of a crossing ID. The area ID is identification information that is unique for each area.

[0104] In the above embodiment, a part or all of the processes performed by the server device 100 may be performed by the router 200. In the above embodiment, a part or all of the processes performed by the router 200 may be performed by the server device 100. In the above embodiment, a part or all of the processes performed by the terminal device 700 may be performed by the server device 100.

[0105] Each device in the embodiment may be made up of multiple devices.

[0106] The processor 101, the processor 201, and the processor 701 may implement part or all of the processes implemented by the programs in the above embodiments by a hardware configuration of the circuits.

[0107] The program for implementing the process of the embodiment is transferred in a state in which it is stored in, for example, a non-transitory computer-readable storage medium in the device. However, the device may be transferred in a state in which the program is not stored. The program may then be transferred separately and written to the device. The program may be transferred in this case by, for example, recording it in a removable non-transitory computer-readable storage medium, or by downloading it via a network such as the Internet or a LAN.

[0108] Although the embodiment of the present invention has been described above, it is merely an example and does not limit the scope of the present invention. The embodiment of the present invention can be implemented in various forms without departing from the gist of the present invention. [Explanation of symbols]

[0109] 1. Detection System 100 Server Devices 101,201,701 processors 102,202,702 ROM 103,203,703 RAM 104,204,704 Auxiliary storage device 105,705 Communication I / F 106,208,708 buses 200 Router 205 1st communication I / F 206 Second communication I / F 207 Relay I / F 300 Camera 400 Railroad Crossing Relay 500 Obstacle Detection Relay 600 Special Signal Relay 700 Terminal Equipment 706 Input Devices 707 Display Device

Claims

1. An acquisition unit that acquires images captured by a camera that captures a railroad track including a track on which trains run or a roadway including a road on which BRT vehicles run; An obstacle detection device comprising: an obstacle detection unit that performs processing to detect obstacles within the railroad or road using background difference using the image.

2. An acquisition unit that acquires images captured by a camera that captures a railroad track including a track on which trains run or a roadway including a road on which BRT vehicles run; and an obstacle detection unit that performs processing to detect obstacles within the railroad track or road using a method of inputting the image into a trained model and obtaining an output indicating whether or not an obstacle is captured in the image.

3. 2. The obstacle detection device according to claim 1, wherein the obstacle detection unit performs a process of detecting the obstacle within the railroad or road using the background subtraction using the image and a method of inputting the image into a trained model to obtain an output indicating whether or not an obstacle is reflected in the image.

4. The obstacle detection device according to claim 2 or 3, wherein the trained model is a trained model trained using a variational autoencoder.

5. The acquisition unit acquires the images captured by each of the plurality of cameras that capture the railroad track or the road, 3. The obstacle detection device according to claim 1, wherein the obstacle detection unit performs processing for detecting the obstacle using a plurality of the images captured by different cameras.

6. The railroad or road is a railroad crossing; The system further includes a crossing determination unit that determines whether the crossing is crossed or an alarm of the crossing is sounding, The obstacle detection device according to claim 1 or claim 2, wherein the obstacle detection unit performs processing to detect the obstacle when the blocking determination unit determines that the railroad crossing is blocked or when the blocking determination unit determines that the alarm is sounding.

7. A train detection unit detects the train on the railroad or the vehicle for BRT on the road using the image, 3. The obstacle detection device according to claim 1, wherein the obstacle detection unit performs a process of detecting the obstacle when the train or the BRT vehicle is not detected by the train detection unit.

8. The processor in the obstacle detection device is An acquisition unit that acquires images captured by a camera that captures a railroad track including a track on which trains run or a roadway including a road on which BRT vehicles run; A program that functions as an obstacle detection unit that performs processing to detect obstacles within the railroad track or road using background difference using the image.

9. The processor in the obstacle detection device is An acquisition unit that acquires images captured by a camera that captures a railroad track including a track on which trains run or a roadway including a road on which BRT vehicles run; A program that functions as an obstacle detection unit that performs processing to detect obstacles within the railroad tracks or road using a method of inputting the image into a trained model and obtaining an output indicating whether or not an obstacle is captured in the image.

10. Acquire an image captured by a camera photographing a railroad track including a track on which trains run or a roadway including a road on which BRT vehicles run; An obstacle detection method that performs processing to detect obstacles within the railroad or road using background difference using the image.

11. Acquire an image captured by a camera photographing a railroad track including a track on which trains run or a roadway including a road on which BRT vehicles run; An obstacle detection method that performs processing to detect obstacles within the railroad tracks or road using a method of inputting the image into a trained model and obtaining an output indicating whether or not an obstacle is captured in the image.

Images