Line detection device and line detection method

Abstract

The opening and closing state of a turnout in a line can be more accurately and easily determined according to a shot image. A line detection device is provided with: a line detection unit that uses a line detection model, which is a learned model that inputs an image and outputs a line detection image as a result of line detection, to detect a line from an image captured by an image capturing device that captures an image of an area around a railway vehicle, and to output a line detection image; the turnout detection part is used for carrying out image processing on the line detection image, detecting a pair of triangular areas between the tracks from the line detection image, and detecting the turnout when the triangular areas are detected; outputting a determination image for determining the vertex of each of the pair of triangular regions as a switch for diverging a line into a plurality of lines or joining the plurality of lines; and a turnout state determination unit for determining the opening / closing state of the turnout according to the determination image by using a turnout state determination model, which is a learned model that inputs the image of the switch including the turnout and outputs the opening / closing state of the turnout.

Description

Technical Field

[0001] Embodiments of the present invention relate to a line detection device and a line detection method. Background Art

[0002] In recent years, there is a demand for automatic driving of railway vehicles. In order to realize automatic driving of railway vehicles, a more accurate and detailed map database (for example, a database including branching positions and merging positions) is required. In addition, the existing map database needs to be frequently maintained.

[0003] Here, there are multiple lines in the railway facilities, and switches are provided in the middle of the lines for branching the lines into multiple lines or merging multiple lines. When moving a railway vehicle to another line, it is necessary to switch the opening and closing of a switch, which refers to the part of the switch that actually branches the line, that is, the switch machine. In order to determine whether the opening and closing direction of such a switch is consistent with the traveling direction of the railway vehicle, it is necessary to process the image captured by the on-board camera mounted on the railway vehicle to accurately determine the state of the opening and closing direction of the switch.

[0004] Prior art literature:

[0005] Patent Literature:

[0006] Patent Document 1: Japanese Patent No. 6855712

[0007] Patent Document 2: Japanese Patent Application Publication No. 2021-157486

[0008] Patent Document 3: Japanese Patent No. 7068026 Summary of the invention

[0009] Problems to be solved by the invention:

[0010] However, in the running environment of railway vehicles, the illumination will change greatly due to the weather, seasons and other environmental changes and time periods, and the appearance of the track will change greatly. In addition, due to the shaking or tilting of the railway vehicle during running, the shooting angle will change greatly, making it difficult to always stably detect the track in the captured image. Therefore, it is even more difficult to detect the switch from the track and accurately determine the open and closed state of the switch.

[0011] One of the objects of the present embodiment is to provide a track detection device and a track detection method that can more accurately and easily determine the open / closed state of a switch existing on a track based on a captured image.

[0012] Means used to solve the problem:

[0013] A track detection device according to an embodiment is a track detection device mounted on a railway vehicle, comprising: a track detection unit that detects a track consisting of a pair of rails from an image captured by a camera that captures an area around the railway vehicle using a track detection model, and outputs a track detection image, the track detection model being a learned model that inputs an image and outputs the track detection image as a detection result of the track; a switch detection unit that detects a pair of triangular areas between the rails from the track detection image by performing image processing on the track detection image, and, when the pair of triangular areas are detected, outputs a determination image that determines the vertices of each of the pair of triangular areas as a switch (point) of a switch for branching the track into a plurality of tracks or merging the plurality of tracks; a switch state determination unit that determines the open / closed state of the switch based on the determination image using the switch state determination model, the switch state determination model being a learned model that inputs an image including a switch of the switch and outputs the open / closed state of the switch; and an output unit that outputs the determination result of the open / closed state of the switch. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a diagram showing an example of the overall configuration of a vehicle system according to an embodiment.

[0015] Figure 2 This is a block diagram showing an example of the functional structure of the line detection device according to the embodiment.

[0016] Figure 3 It is a schematic diagram which shows an example of the turnout in embodiment.

[0017] Figure 4 This is a diagram showing an example of a branching situation in the output result of the determination unit according to the embodiment.

[0018] Figure 5 This is a diagram showing an output example of the determination unit when two lines merge in the embodiment.

[0019] Figure 6 This is a diagram for explaining an example of a method for determining the range of a branching region in the embodiment.

[0020] Figure 7 This is a diagram for explaining an example of a method for determining the range of a branching region in the embodiment.

[0021] Figure 8 This is a schematic diagram showing an example of a data structure of a layer of information related to branching of a route in a map DB (database) according to the embodiment.

[0022] Fig. 9This is a diagram showing an example of a case where the comparison result is normal in the embodiment.

[0023] Fig.10 This is a diagram showing an example of a case where the comparison result is abnormal in the embodiment.

[0024] Fig.11 This is a diagram showing an example of a case where a switch is detected from a location that is not registered in the map DB in the embodiment.

[0025] Fig.12 This is a flowchart showing an example of the sequence of the line detection process according to the embodiment.

[0026] Fig.13 This is a diagram showing an example of an image generated during the line detection process according to the embodiment.

[0027] Fig.14 This is a flowchart showing an example of the procedure of the matching process according to the embodiment.

[0028] Fig.15 This is a diagram showing an example of the determination result of the open / closed state of the turnout switch when the track branches into two tracks in the embodiment.

[0029] Fig.16 This is a diagram showing an example of the result of determination of the state of a turnout when two tracks merge into one track in the embodiment.

[0030] Fig.17 This is a diagram showing an example of the determination result of the open / closed state of the turnout when both the divergence and the merging exist in one captured image in the embodiment.

[0031] Fig.18 This is a diagram showing an example of complex branching in the embodiment.

[0032] Fig.19 This is a diagram showing an example of a captured image of a plurality of links including a branch having a crossover in the embodiment.

[0033] Fig. 20 This is a diagram for explaining an example of setting the range of the branching region in the case of a complex branching structure in the embodiment.

[0034] Description of reference numerals:

[0035] 1. Vehicle System

[0036] 10 cameras

[0037] 13 Judgment Department

[0038] 14 Line information generation unit

[0039] 15Comparison Department

[0040] 16 Positioning unit

[0041] 17 Storage

[0042] 18 Display processing unit

[0043] 19 Ministry of Communications

[0044] 20 Reporting Department

[0045] 30 Display device

[0046] 50-day antenna

[0047] 100 line detection device

[0048] 171 Map DB

[0049] 131 Line Detection Department

[0050] 132 Turnout Inspection Department

[0051] 133 Turnout status determination unit

[0052] Triangle area 401a, 401b, 2001a, 2001b, 2001c

[0053] 403, 403a, 403b, 403c bifurcation area

[0054] 1311 line detection model

[0055] 1331 Turnout State Determination Model

[0056] 200 Central Device DETAILED DESCRIPTION

[0057] Hereinafter, this embodiment will be described using the drawings.

[0058] (Implementation Method)

[0059] (Overall Structure of Vehicle System 1)

[0060] Figure 1 FIG. 1 is a diagram showing an example of the overall configuration of a vehicle system 1 according to an embodiment. Figure 1 As shown, the vehicle system 1 of this embodiment includes a railway vehicle RV and a central device 200 installed outside the railway vehicle RV. The railway vehicle RV travels on the track R. The railway vehicle RV includes a camera 10, an antenna 50, a display device 30, and a line detection device 100. The line detection device 100 and the central device 200 are connected via a wireless network. The antenna 50 and the display device 30 are respectively connected to the line detection device 100 by wire or wirelessly.

[0061] The camera 10 can capture the traveling direction of the railway vehicle RV. The camera 10 can also be a device that can measure the distance to an object, such as a stereo camera. The camera 10 faces the front of the railway vehicle RV, that is, the traveling direction of the railway vehicle RV. Thus, the camera 10 captures an image of the traveling direction of the railway vehicle RV. The camera 10 sends the captured image to the track detection device 100. In addition, the image captured by the camera 10 is not limited to the front of the railway vehicle RV, as long as it is an image captured around the railway vehicle RV. For example, for the purpose of line inspection, maintenance, etc., in order to grasp the location and opening and closing status of the turnout, the camera 10 can also be arranged at the rear of the railway vehicle RV to capture the rear of the railway vehicle RV. The camera 10 is an example of a shooting device. In addition, the camera is not limited to a stereo camera, and can also be a monocular camera.

[0062] The antenna 50 is installed near the center in the width direction of the railway vehicle RV, and receives radio wave signals transmitted from artificial satellites for positioning. The radio wave signals are signals containing information for measuring the position of the railway vehicle RV.

[0063] In addition, the railway vehicle RV may include autonomous sensors such as a triaxial acceleration sensor, a triaxial gyro sensor, a triaxial geomagnetic sensor, a wheel speed sensor, and a tachometer generator (TG: tachometer generator).

[0064] The display device 30 is a display device such as a display device, is connected to the line detection device 100, and can display various information.

[0065] The track detection device 100 detects a track from an image captured by the camera 10, detects a switch in the track, and determines the open / closed state of the switch of the switch. Here, the track is composed of a pair of left and right rails R. The switch is a device for branching a track into a plurality of tracks or merging the plurality of tracks.

[0066] The central device 200 is an information processing device such as a server device. The central device 200 communicates with the track detection device 100 to control the travel of the railway vehicle RV. The central device 200 of this embodiment generates and updates a map database (hereinafter referred to as "map DB") on which the railway vehicle RV can travel, and transmits it to the track detection device 100. In addition, the central device 200 transmits information related to the track and turnout on which the railway vehicle RV travels to the track detection device 100 at a predetermined timing. In addition, the central device 200 receives various information from the track detection device 100.

[0067] (Structure of line detection device)

[0068] Next, the line detection device 100 will be described in detail.

[0069] Figure 2 1 is a block diagram showing an example of the functional structure of the line detection device 100 according to the embodiment. Figure 2 As shown, the line detection device 100 of this embodiment mainly includes a determination unit 13, a positioning unit 16, a line information generation unit 14, a matching unit 15, a display processing unit 18, a reporting unit 20, a communication unit 19 and a storage unit 17.

[0070] The positioning unit 16 locates the current position of the railway vehicle RV. Specifically, the positioning unit 16 can measure the position of the railway vehicle RV based on the radio wave signal from the artificial satellite received by the antenna 50. The position of the railway vehicle RV is the position information near the center of the left and right tracks of the line. The positioning unit 16 is, for example, a GNSS (Global Navigation Satellite System), and locates the railway vehicle RV (satellite positioning) based on the radio wave received by the antenna 50, and outputs the position information (also called positioning information) as the measurement result.

[0071] Here, GNSS is a general term for satellite positioning systems such as the GPS of the United States, the Quasi-Zenith Satellite (QZSS) of Japan, the GLONASS (GLO) of Russia, the Galileo System (GAL) of the European Union, and the BeiDou System (BDS) of China.

[0072] Therefore, the receiver constituting the antenna 50 and the positioning unit 16 is configured so as to be able to perform positioning using any of these satellite positioning systems.

[0073] Furthermore, the positioning unit 16 can also measure the current position of the railway vehicle RV based on the positioning result based on the radio waves received by the antenna 50 or using the sensing result from the inertial sensor (not shown) alone. For example, the positioning unit 16 can measure the current position by integrating the output value of the tachometer generator of the wheel of the inertial sensor (not shown).

[0074] The determination unit 13 detects the track and the switch and determines the open / closed state of the switch based on the captured image obtained from the camera 10. Figure 2 As shown, the determination unit 13 includes a track detection unit 131, a switch detection unit 132, and a switch state determination unit 133. In addition, hereinafter, the open / closed state of a switch of a switch may be referred to as "the open / closed state of the switch".

[0075] The track detection unit 131 receives an image obtained by photographing the front of the railway vehicle RV in the traveling direction from the camera 10, detects the track from the image, and outputs a track detection image. The track detection image is an image of the detection result of the track. The track detection image is classified into three categories: a pair of left and right rails constituting the track, rail spaces representing the space between the pair of left and right rails, and other pixels.

[0076] Specifically, the line detection unit 131 includes a line detection model 1311 , detects one or more lines from a captured image using the line detection model 1311 , and outputs a line detection image.

[0077] The line detection model 1311 is a learned model that inputs an image and outputs a line detection image as a detection result of a line composed of a pair of rails. The line detection model 1311 is generated in advance by machine learning using learning data by an arbitrary computer. The learning algorithm used in generating the line detection model 1311 can use, for example, semantic segmentation in deep learning, but is not limited to this.

[0078] The line detection model 1311 is a model obtained by learning in advance using a large number of learning images obtained by photographing a pair of rails and switches constituting the line, and annotations, which are annotations that are pre-assigned as correct data for each learning image and each pixel represents a pair of rails, between rails, and other pixels. That is, the line detection model 1311 is generated by updating the parameters of the learning model so that the output value obtained by inputting each learning image to the learning model modeled by a neural network or the like is consistent with the correct data corresponding to the learning image.

[0079] The line detection unit 131 can output a line detection image by classifying the captured image from the camera 10 into a pair of left and right line tracks, between rails, and other areas as an output obtained by semantic segmentation using the line detection model 1311 .

[0080] Figure 3 Schematic diagram showing an example of a turnout in the embodiment. Figure 3 In the example of FIG. 1 , an example is shown in which a pair of tracks R1 is branched into a pair of tracks R1 and a pair of tracks R2 .

[0081] Figure 4 This is a diagram showing an example of a branching state in the output result of the determination unit 13 according to the embodiment. Figure 4 (a) shows the circuit detection unit 131 according to Figure 4 An example of a track detection image outputted from a captured image of a track including a turnout as shown. Figure 4As shown in the example of the track detection image of (a), the track categories of left track R1L, R2L, right track R1R, R2R, and track space are represented by different shading, and other categories are represented by no shading.

[0082] Alternatively, the track detection model 1311 may be configured to classify the left tracks R1L, R2L and the right tracks R1R, R2R into the same category, and further classify them into the left tracks R1L, R2L and the right tracks R1R, R2R by post-processing such as image processing.

[0083] The switch detection unit 132 performs image processing on the track detection image output by the track detection unit 131, detects a pair of triangular regions between the tracks from the track detection image, and outputs a determination image that determines the vertices of the pair of triangular regions as switches when the pair of triangular regions are detected. Here, the switch refers to a switch machine.

[0084] When a plurality of tracks are detected in the track detection image, the turnout detection unit 132 determines the track on which the railway vehicle RV is traveling based on the track detection image, detects one or more pairs of triangular areas in front of the determined traveling track, and outputs a determination image in which the vertices of the triangles of the one or more pairs of triangular areas are determined as switches of one or more turnouts. In addition, when a single track is detected in the track detection image, the turnout detection unit 132 determines the track on which the railway vehicle RV is traveling.

[0085] like Figure 3 As in the example of the turnout shown in FIG. 1 , the turnout is composed of a switch portion 101 and a guide rail portion 102. In order to accurately determine the switch state of the turnout, the switch portion 101 needs to be detected with high accuracy.

[0086] The switch detection unit 132 performs a dilation process on the track detection image to dilate the width of each rail in the track detection image, and detects a pair of triangular areas from the image after the dilation process. Here, as the dilation process, for example, morphological processing can be used.

[0087] Figure 4 (b) shows the Figure 4 The example of the track detection image shown in (a) after the expansion process is performed. The turnout detection unit 132 Figure 4 The left rails R1L, R2L and the right rails R1R, R2R in the line detection image shown in (a) are expanded respectively to expand the width of each rail. Figure 4 As shown in (b), the right guide rail R1R is in contact with the right guide rail R2R, and the left guide rail R1L is in contact with the left guide rail R2L.

[0088] The results, such as Figure 4As shown in (b), the shape of the track between the switch part and the guide rail part of the turnout is a triangle. The triangular part is called a triangular area. A pair of left and right triangular areas appear corresponding to the left and right tracks. Therefore, the turnout detection unit 132 is expanded from Figure 4 The track portion is deleted from the image of (b), and a pair of left and right triangular areas 401a and 401b are detected from the inter-track area. Figure 4 (c) shows an example in which a pair of triangular regions 401a and 401b are extracted.

[0089] Here, if Figure 4 As shown in (c), the vertices of the pair of triangular regions 401a and 401b correspond to the ends of the switch of the turnout. Therefore, the turnout detection unit 132 determines the vertex of the triangular region as the front end of the switch of the turnout, sets the specified range including the vertex as the branching region 403 representing the range of the switch of the turnout and outputs it as a determined image.

[0090] Figure 4 is an example of a line branching through a bifurcation. Figure 5 This is a diagram showing an output example of the determination unit 13 when two tracks merge in the embodiment. In the case of merging, the track detection unit 131 and the switch detection unit 132 also perform the same Figure 4 The example shown is processed similarly.

[0091] Here, the range of the branching region 403 will be described.

[0092] The switch detection unit 132 sets the length of the branch region 403 in a direction intersecting the traveling direction of the railway vehicle RV, that is, the lateral width, to be longer than the interval between the pair of rails.

[0093] The switch detection unit 132 sets the longitudinal width of the branch region 403 in the traveling direction of the railway vehicle based on the transverse width. For example, the switch detection unit 132 may set the longitudinal width to be the same as the transverse width or to be 1 / 2 of the transverse width.

[0094] The turnout detection unit 132 can also set the longitudinal width of the branching area 403 based on the lateral width and the internal parameters and external parameters related to the camera 10. In addition, the longitudinal width of the branching area 403 can also be set so that the longer the distance from the current position of the railway vehicle RV to the switch of the turnout is, the longer the longitudinal width is. Moreover, the turnout detection unit 132 can adjust the longitudinal width of the branching area in coordination with the angle of the vertex of the detected triangular area (i.e., the angle between the main rail and the point rail). As an example, since the switch portion becomes longer when the angle of the vertex of the triangular area is small, the turnout detection unit 132 can also be configured in a manner that sets the longitudinal width of the turnout area to be longer.

[0095] Figure 6 , Figure 7 This is a diagram for explaining an example of a method for determining the range of the branch region 403 in the embodiment. Figure 6 The figure shows a case where the angle of the apex of the triangular region representing the switch of the turnout is small. Figure 7 The angle ratio of the vertices of the triangular area representing the switch of the turnout is shown. Figure 6 Big situation.

[0096] like Figure 6 As shown in FIG. 1 , when the angle of the vertex of the triangular region is small, the angle change of the traveling direction of the railway vehicle RV is small, and the switch portion of the turnout becomes longer, so the turnout detection unit 132 sets the longitudinal width of the branching region to be larger. Figure 7 As shown, when the angle of the vertex of the triangular area is large, the angle change of the traveling direction of the railway vehicle RV is large, and the switch portion of the turnout becomes short, so the turnout detection unit 132 sets the longitudinal width of the branching area to be small.

[0097] return Figure 2 The turnout state determination unit 133 uses the turnout state determination model 1331 to determine the open / closed state of the turnout based on the determined image output by the turnout detection unit 132. The turnout state determination model 1331 is a learned model that inputs an image of a switch including a turnout and outputs the open / closed state of the turnout. The turnout state determination model 1331 is generated in advance by machine learning using learning data by any computer. The learning algorithm used in generating the turnout state determination model 1331 may be, for example, a category classification in deep learning, but is not limited thereto.

[0098] The switch state determination model 133 is a model obtained by pre-learning for a given fixed image of a switch including a switch, using annotations indicating the results of multi-category classification such as left branching, right branching, left merging, right merging, and straight-ahead, which are pre-assigned as correct data for each learning image. That is, the parameters of the learning model are updated in such a way that the output value obtained by inputting each learning image into the learning model modeled by a neural network or the like is consistent with the correct data corresponding to the learning image, thereby generating the switch state determination model 1331. Here, left branching, right branching, left merging, right merging, and straight-ahead indicate the open and closed states of the switch.

[0099] The switch state determination unit 133 uses the switch state determination model 1331 to classify and output the determination image of the switch including the switch output by the switch detection unit 132. As a result, the determination image can be classified into left branching, right branching, left merging, right merging, and straight going, which are the opening and closing states of the switch.

[0100] The track information generating unit 14 generates track information based on the processing result of the determining unit 13, that is, the turnout detection result as the output result of the turnout detecting unit 132, the turnout opening and closing state determination result as the output result of the turnout state determining unit 133, and the vehicle position of the railway vehicle RV measured by the positioning unit 16. As an example, the track information generating unit 14 can detect the turnout 200 m (longitude and latitude information) ahead in the traveling direction of the vehicle position of the railway vehicle RV, and generate the turnout state as a left turnout.

[0101] The storage unit 17 is a storage medium such as a HDD (Hard Disc Drive) or a SSD (Solid State Drive). The storage unit 17 of the present embodiment stores a map DB 171. The map DB 171 is an example of map information.

[0102] The map DB 171 contains information on a map of a region including the travel route of the railway vehicle RV. The map DB 171 is composed of a plurality of layers. For example, the map DB 171 is composed of a layer in which the basic map data of the 1 / 2500 level of the Geographical Survey Institute is registered, a layer in which the sensing area sections and sensing area ranges are registered, a layer in which the information on civil engineering structures such as tunnels and bridges is registered, and the like. Among the plurality of layers, there is a layer in which information on the branching of the line is registered.

[0103] Figure 8 1 is a schematic diagram showing an example of a data structure of a layer of information related to branching of a route in the map DB 171 according to the embodiment. Figure 8As shown, in the layer of information related to the branch in the map DB 171, the position information of the switch in the track, the open and closed state of the switch in each time period, and the management information are associated with each other.

[0104] The position information of the turnout is the position coordinates of the turnout on the map. The open / closed state of the turnout is the correct open / closed state of the turnout switch in each time period. The management information is information indicating a method of handling when the turnout or the open / closed state of the turnout determined by the determination unit 13 is inconsistent with the above-mentioned registration content of the map DB 171 or when it is inconsistent with the notification content from the central device 200.

[0105] Image saving specifies whether to save the detected image in the storage unit 17 when the switch or the switch opening and closing state determined by the determination unit 13 does not match the above-mentioned registration content of the map DB 171. "On" means to save the image, and "off" means not to save the image.

[0106] The map update is to determine whether to send the update request of the map DB 171 to the central device 200 when the switch or the open / closed state of the switch determined by the determination unit 13 is inconsistent with the above-mentioned registration content of the map DB 171. In the case of "ON", it means that the update request of the map DB 171 is sent to the central device 200, and in the case of "OFF", it means that the update request of the map DB 171 is not sent to the central device 200.

[0107] The fault sensing notification is to determine whether to report a fault when the switch or the switch opening and closing state determined by the determination unit 13 is inconsistent with the notification content from the central device 200. "Open" means reporting a fault, and "Closed" means not reporting a fault.

[0108] Note that the above management information of the map DB 171 is all set to on by default and can be changed by the administrator.

[0109] It should be noted that the information in the map DB 171 is merely an example, and the map DB 171 may not have part of the information and may have other information.

[0110] return Figure 2The comparison unit 15 compares the position information of the turnout detected by the turnout detection unit 132 and the open / closed state of the turnout determined by the branch state determination unit 133 with the position information of the turnout and the open / closed state of the turnout in the time period including the current time in the map DB 171. If the comparison result is inconsistent, the comparison unit 15 records the running position of the railway vehicle RV (own vehicle) measured by the positioning unit 16 and the captured image captured by the camera 10 in the storage unit 17 at the same time when the image storage of the management information of the map DB 171 is set to on.

[0111] When the comparison result is inconsistent, the comparison unit 15 determines whether the open / closed state of the turnout determined by the turnout state determination unit 133 is consistent with the traveling direction of the railway vehicle RV received by the communication unit 19 described later.

[0112] The display processing unit 18 causes the display device 30 to display the position information of the turnout detected by the turnout detection unit 132 , the open / closed state of the turnout determined by the branch state determination unit 133 , and the comparison result of the comparison unit 15 .

[0113] The communication unit 19 transmits and receives various information and notifications to and from the central device 200 through wireless communication.

[0114] Specifically, the communication unit 19 of the present embodiment transmits a correction instruction of the map DB 171 to the central device 200 when the comparison result with the map DB 171 obtained by the comparison unit 15 is inconsistent. In addition, the correction instruction of the map DB 171 by the communication unit 19 is performed when the map update of the management information of the map DB 171 is set to on.

[0115] In addition, as described above, when the comparison result is inconsistent, the comparison unit 15 records the traveling position of the railway vehicle RV (own vehicle) and the captured image captured by the camera 10 in the storage unit 17 at the same time when the image saving of the management information of the map DB 171 is set to on, thereby improving the reliability of the update process or correction process of the map DB 171 in the central device 200.

[0116] In addition, the communication unit 19 receives information on the traveling direction of the railway vehicle RV from the central device 200 .

[0117] When the comparison result of the comparison unit 15 is inconsistent, the notification unit 20 outputs to the display device 30 that the comparison result is inconsistent.

[0118] In addition, when the open / closed state of the turnout determined by the turnout state determination unit 133 is different from the traveling direction of the railway vehicle RV received by the communication unit 19, the reporting unit 20 outputs an alarm indicating that a turnout failure has occurred to the display device 30. When the failure sensing notification of the management information of the map DB 171 is set to on, the alarm output by the reporting unit 20 is performed.

[0119] Furthermore, the reporting unit 20 and the communication unit 19 may be configured to transmit a notification that a failure has occurred in a switch to the central device 200 .

[0120] The notification unit 20 , the display processing unit 18 , and the display device 30 are examples of an output unit.

[0121] Next, an example of a screen display of the comparison processing result will be described.

[0122] Fig. 9 This is a diagram showing an example of a case where the comparison result is normal in the embodiment. The position and opening / closing state of the turnout detected by the determination unit 13 are consistent with the registration information of the map DB 171, and the comparison result is normal. In this case, the image saving function as evidence, the map DB update function, and the fault detection notification function are turned off by default. The image saving function can be changed by the administrator or the like.

[0123] Fig.10 1 is a diagram showing an example of a case where the comparison result is abnormal in the embodiment. Fig.10 , an example is shown in which the position information of the turnout detected by the determination unit 13 is consistent with the registration information of the map DB 171, but the detected switch opening and closing state is different from the switch opening and closing state registered in the map DB 171. When the comparison unit 15 detects an abnormality, since it may be related to a major accident, the preservation function is turned on for the image as evidence. In this case, the image preservation function can be changed (designated to off) by the administrator or the like. In addition, the update function of the map DB 171 and the on / off setting of the fault detection notification function can also be performed.

[0124] Fig.11 1 is a diagram showing an example of a case where a turnout is detected from a location not registered in the map DB 171 in the embodiment. Fig.11 As shown, all of the default settings of the function of saving the image as evidence, the function of updating the map DB 171, and the function of notifying the failure detection are on.

[0125] (Line detection processing)

[0126] Next, the line detection process of the line detection device 100 according to the present embodiment configured as above will be described.

[0127] Fig.12 This is a flowchart showing an example of the sequence of the line detection process according to the embodiment.

[0128] Fig.13 This is a diagram showing an example of an image generated during the line detection process according to the embodiment.

[0129] First, the line detection unit 131 acquires a captured image from the camera 10 ( S101 ). Fig.13 (a) is an example of a captured image. Then, the line detection unit 131 detects a line area from the captured image using the line detection model 1311 and outputs a line detection image (S102). Fig.13 (b) is from Fig.13 An example of a line detection image detected in the captured image of (a).

[0130] Next, the switch detection unit 132 determines the driving lane of the vehicle from one or more links based on the link detection image ( S103 ).

[0131] Here, regarding the determination of the driving lane of the vehicle by the switch detection unit 132, a method of determining the driving wheel by using the internal parameters and external parameters of the camera 10 can be considered. Alternatively, the switch detection unit 132 can be configured to use the result of the GPS signal received by the positioning unit 16. Alternatively, the switch detection unit 132 can also be configured to analyze the line detection image to determine the driving lane.

[0132] Then, as described above, the switch detection unit 132 detects a switch of a switch in front of the determined line of the travel lane (S104). Fig.13 (c) is a diagram showing an example of a switch detecting a turnout.

[0133] When a switch of a turnout is detected ahead of the track of the travel lane (S104: Yes), the turnout detection unit 132 extracts a triangular area serving as the turnout from the track detection image as described above, and outputs a confirmation image (S106). Fig.13 (d) is a diagram showing an example of a determined image.

[0134] Next, the turnout state determination unit 133 uses the turnout state determination model 1331 to determine the open / closed state of the turnout switch based on the determined image (S107). Next, the line information generation unit 14 generates and outputs line information including the turnout (S108). Then, the process returns to S103, and the processes of S106 to S108 are repeatedly performed until the turnout switch is no longer detected in front of the line of the driving lane.

[0135] When the switch of the turnout is not detected in front of the line of the travel lane in S104 (S104: No), the matching unit 15 performs a matching process (S105). When the matching process is completed, the process ends.

[0136] Next, the comparison process in S105 will be described.

[0137] Fig.14 This is a flowchart showing an example of the procedure of the matching process according to the embodiment.

[0138] The comparison unit 15 obtains information such as the position information of the turnout and the opening and closing state of the turnout from the map DB 171 (S201). Next, the comparison unit 15 compares whether the position information of the turnout and the opening and closing state of the turnout detected by the determination unit 13 are consistent with the position information of the turnout and the opening and closing state of the turnout obtained from the map DB 171 in S201 (S202).

[0139] If the comparison result shows that the two are consistent (S202: Yes), the process ends and returns to the calling source.

[0140] On the other hand, if the comparison result shows that the two do not match (S202: No), the communication unit 19 sends an update instruction of the map DB 171 to the central device 200 (S203). In addition, the reporting unit 20 outputs a warning indicating that a failure has been detected to the display device 30 (S204). Then, the process returns to the calling source.

[0141] Fig.15 1 is a diagram showing an example of the determination result of the open / closed state of the turnout switch when the track is branched into two tracks in the embodiment. Fig.15 In the figure, for the sake of convenience, the background other than the track and turnout is omitted.

[0142] exist Fig.15 In (a), the detection results of the track and rails of the line are shown. In the case of bifurcation, the vertices 1701 and 1702 of the triangular area between the rails corresponding to the switch of the turnout are facing downward ( Fig.15 The switch detection unit 132 detects the vertices 1701 and 1702 at the bottom of the paired left and right triangular regions. Since the lower part of the vertex 1701 is a switch indicating a merging switch, the detected switch is set to Fig.15 The bifurcation region 403 of (b).

[0143] Fig.16 1 is a diagram showing an example of the result of determining the state of a turnout when two tracks merge into one track in the embodiment. Fig.16In the figure, for the sake of convenience, the background other than the track and turnout is omitted.

[0144] exist Fig.16 In (a), the detection result of the track and rail space of the line is also shown. In the case of merging, the vertices 1801 and 1802 of the triangular area between the rails corresponding to the switch of the turnout are facing upward ( Fig.16 The switch detection unit 132 detects the top vertex of the paired left and right triangular areas. Since the top of the vertex is the switch indicating the merging switch, the detected switch is set to Fig.16 The bifurcation region 403 of (b).

[0145] Fig.17 1 is a diagram showing an example of the determination result of the open / closed state of a turnout when both a divergence and a merging state exist in one captured image in the embodiment. Fig.17 In FIG. 1 , an example of the result of detecting the switch of the turnout and determining the opening and closing state of the turnout is shown. Fig.17 As shown, even when the switch detection unit 132 detects a plurality of switches having different switch opening and closing states from one captured image, the switch state determination unit 133 can accurately determine the position of each switch and the switch opening and closing state.

[0146] In the above description, a relatively simple example of detecting one or two tracks in one captured image is cited for explanation. However, in this embodiment, through the above processing, even for a captured image in which there are multiple tracks and a complex branching structure, the detection of the turnout and the determination of the open and closed state of the turnout can be accurately performed.

[0147] Fig.18 is a diagram showing an example of complex bifurcation in the implementation mode. Fig.18 The most complex crossover branching structure is shown in Figure 1. Here, the crossover branching structure is a track structure in which two sets of crossovers are set in an X shape between two connected tracks. The crossover consists of four sets of turnouts and one frog. No matter which track a train is running on, it can cross over to the adjacent track without backing up by crossing the crossover.

[0148] Below, with Fig.18 The case of a bifurcated structure of a crossover shown in FIG. 1 is described as an example.

[0149] Fig.19 This is a diagram showing an example of a captured image of a plurality of links including a branch having an intersecting crossover in the embodiment.

[0150] In this embodiment, the line detection unit 131 outputs Fig.19A track detection image in which a plurality of tracks are detected in the captured image shown in FIG. As described above, the turnout detection unit 132 determines the driving lane of the vehicle from the plurality of tracks. Then, the turnout detection unit 132 determines the state of the turnout with the driving lane as the center. Fig.19 In FIG. 1 , the determined travel lane is indicated by arrow 2005. Then, the switch detection unit 132 detects the switch of the switch located in front of the travel lane. The detection method is as described above.

[0151] exist Fig.19 In the example of FIG. 1 , two pairs of triangular regions 2001a and 2001b are detected between a pair of tracks in front of the driving lane. Then, the switch detection unit sets branching regions 403a and 403b for them respectively.

[0152] An example of the determination result of the branching regions 403a and 403b is as follows.

[0153] The state determination result of the turnout in the branch area 403a corresponding to the triangular area 2001a is that the vehicle merges from the left, and the state determination result of the turnout in the branch area 403b corresponding to the triangular area 2001b is that the vehicle branches off and enters the right side.

[0154] Then, the switch detection unit 132 determines the next travel lane based on the next correlation determination result.

[0155] According to the state determination result (branching, entering the right side) of the turnout in the branch area 403b corresponding to the triangular area 2001b, the next driving lane of the driving lane indicated by the arrow 2005 is the portion indicated by the arrow 2006. A pair of triangular areas 2001c are detected from the driving lane indicated by the arrow 2006, and an example of the determination result of the branch area is as follows.

[0156] The state determination result of the branching region 403c corresponding to the triangular region 2001c is merging from the right.

[0157] Then, since the turnout detection unit 132 cannot detect the turnout switch from the front of the further travel lane, the turnout detection process ends.

[0158] In this way, even for captured images of a plurality of tracks having a complex branching structure, it is possible to accurately detect switches and determine the open / closed state of switches.

[0159] Here, the switch detection unit 132 sets the range of the branching area as follows.

[0160] Fig. 20 This is a diagram for explaining an example of setting the range of the branching region in the case of a complex branching structure in the embodiment.

[0161] As described above, the switch detection unit 132 determines the lateral width of the branching region based on the distance between the rails (also referred to as the track width). Fig. 20 In the example of , when the switch detection unit 132 detects four triangular areas 2403a, 2403b, 24043c, and 2403d from four tracks ABCD, the pair of triangular areas is selected according to the specified distance between the tracks. When the specified line width is the width between AB, two groups AB and CD can be detected from the four triangular areas. That is, the pair of triangular area 2403a and triangular area 2403b, and the pair of triangular area 2403c and triangular area 2403d are two groups. In this way, by using the line width to determine the lateral width of the branching area, incorrect combinations such as AC and BD can be excluded.

[0162] In addition, as an example, the switch detection unit 132 may determine the lateral width of the branching area based on 1.5 times the track width. In addition, the switch detection unit 132 may also set the lateral width of the branching area to 1.2 times or 2 times the track width, etc., taking into account calculation cost and accuracy.

[0163] In addition, the longitudinal width of the branching region is determined as described above.

[0164] (Summary)

[0165] In recent years, there is a demand for automatic driving of railway vehicles RV. In order to realize automatic driving of railway vehicles RV, a more accurate and detailed map DB (for example, a database including branching positions and merging positions) is required. In addition, the existing map DB needs to be frequently maintained.

[0166] Here, there are multiple lines in the railway facilities, and switches are provided in the middle of the lines for branching the lines into multiple lines or merging multiple lines. When moving a railway vehicle to another line, it is necessary to switch the opening and closing of a switch, which refers to the part of the switch that actually branches the line, that is, the switch machine. In order to determine whether the opening and closing direction of such a switch is consistent with the traveling direction of the railway vehicle, it is necessary to process the image captured by the on-board camera mounted on the railway vehicle to accurately determine the state of the opening and closing direction of the switch.

[0167] Patent document 1 proposes a technique for detecting a turnout from a captured image and determining whether or not the turnout can be entered. In this prior art, the following method is proposed: a candidate point of the track is detected using a brightness difference with the track as a feature quantity, the continuity of a thinned point rail used in a switch (point) or expansion joint and the track is determined by the brightness value, the open state of the turnout is identified, and when the turnout is open, it is determined whether or not the turnout can be entered.

[0168] In addition, the technology of Patent Document 2 relates to a method for detecting railway tracks through images, which discloses dividing an image into a near area on the image, i.e., a nearby area, and a far area on the image, i.e., a distant area, and performing template matching by multiple coordinates for each area to explore extension lines.

[0169] The technology of Patent Document 3 detects a switch by performing template matching by learning image feature quantities in advance.

[0170] However, in the running environment of the railway vehicle RV, the illumination varies greatly due to the weather, seasons and other environmental and time period changes, and the appearance of the track changes greatly. In addition, due to the shaking or tilting of the railway vehicle RV during running, the shooting angle varies greatly, and it is difficult to always stably detect the track from the shot image. Therefore, it becomes more difficult to detect the switch from the track and automatically determine the open or closed state of the switch.

[0171] For example, in the technology of Patent Document 1, in the running environment of railway vehicles, due to changes in the environment such as weather and seasons and time periods, the illumination will vary greatly, so that the brightness value of the track may not necessarily converge within a certain range. In addition, depending on the running environment of railway vehicles, due to the presence of sleepers, gravel, concrete, snow, etc. between the rails, it becomes more difficult to identify the open and closed state of the switch of the turnout by the difference in brightness between the left and right rails, that is, between the rails.

[0172] In addition, in the techniques of Patent Documents 2 and 3, since the line is detected based on template matching, a simple line can be detected, but when multiple lines intersect, it is difficult to accurately detect multiple lines. In addition, since the traveling direction of the vehicle, the shooting angle, the shape of the branching part, etc. are different, the method using template matching in the techniques of Patent Documents 2 and 3 has a problem that it is difficult to detect a part of the line or the shape pattern of a part of the line.

[0173] In order to solve such a problem, in the present embodiment, the track detection device 100 includes: a track detection unit 131 that detects a track from an image captured by a camera 10 capturing an area in front of a railway vehicle RV and the like, and outputs a track detection image, wherein the track detection model 1311 is a learned model that inputs an image and outputs a track detection image as a detection result of a track composed of a pair of rails; a switch detection unit 132 that performs image processing on the track detection image to detect a pair of triangular regions between the rails from the track detection image, and when a pair of triangular regions is detected, outputs a determination image that determines the vertices of the pair of triangular regions as switches of a switch for branching a track into a plurality of tracks or merging a plurality of tracks; a switch state determination unit 133 that determines the open / closed state of a switch based on the determination image, using a switch state determination model 1331, which is a learned model that inputs an image of a switch including a switch and outputs the open / closed state of the switch; and a display processing unit 18 that displays the determination result of the open / closed state of the switch on the display device 30.

[0174] Thus, according to the present embodiment, the detection of the turnout and the determination of the open / closed state of the turnout are performed by using the learned model and image processing simultaneously. Therefore, according to the present embodiment, regardless of the running environment of the railway vehicle RV, the line consisting of a pair of tracks can be detected relatively stably based on the captured image of the camera 10 that captures the front and other surrounding areas of the railway vehicle RV. Moreover, in the present embodiment, the location of the turnout switch can be detected regardless of the brightness value and texture between the rails, and the open / closed state of the turnout switch can be determined based on the branching area that represents the turnout switch. Therefore, according to the present embodiment, the open / closed state of the turnout existing in the line can be determined more accurately and easily based on the captured image.

[0175] In addition, in the present embodiment, the switch detection unit 132 of the track detection device 100 performs an expansion process on the track detection image to expand the width of each rail in the track detection image, and detects a pair of triangular areas from the image on which the expansion process has been performed. Therefore, according to the present embodiment, a pair of triangular areas can be detected more accurately, and the open / closed state of the switch existing in the track can be determined more accurately and easily based on the captured image.

[0176] In addition, in the present embodiment, the line detection unit 131 of the line detection device 100 detects a plurality of lines from the captured image using the line detection model 1311 and outputs a line detection image, the turnout detection unit 132 determines the line on which the railway vehicle RV is traveling based on the line detection image in which the plurality of lines are detected, detects one or more pairs of triangular areas in front of the determined traveling line, and outputs the determined image in which the vertices of the triangles of the one or more pairs of triangular areas are determined as switches of one or more turns, and the turnout state determination unit 133 determines the opening and closing state of each of the one or more turns based on the determined image in which the switches of the one or more turns are determined using the turnout state determination model 1331. Therefore, according to the present embodiment, when the captured image of the camera 10 includes a plurality of lines at the same time, even when the plurality of lines intersect, that is, when there is a branching portion or a merging portion, the opening and closing state of the turnouts existing in the line can be determined more accurately and easily based on the captured image.

[0177] In addition, in the present embodiment, the turnout detection unit 132 of the track detection device 100 determines the vertex of the triangular area as the end of the turnout switch, and outputs a determination image of the branching area in which the predetermined range including the vertex is set as the range of the turnout switch. Therefore, according to the present embodiment, the turnout switch can be determined more accurately, and as a result, the open / closed state of the turnout existing in the track can be determined more accurately and easily based on the captured image.

[0178] In addition, in the present embodiment, the switch detection unit 132 of the track detection device 100 sets the length of the branch area in the direction intersecting the traveling direction of the railway vehicle RV, that is, the lateral width, to be longer than the interval between the pair of rails. Therefore, according to the present embodiment, the switch of the switch can be more accurately identified, and as a result, the open and closed state of the switch existing in the track can be determined more accurately and easily based on the captured image.

[0179] In addition, in the present embodiment, the turnout detection unit 132 of the track detection device 100 sets the length of the branching area in the traveling direction of the railway vehicle, that is, the longitudinal width, based on the lateral width. Therefore, according to the present embodiment, the switch of the turnout can be more accurately determined from the branching area of ​​the longitudinal width based on the lateral width, and as a result, the open and closed state of the turnout existing in the track can be determined more accurately and easily based on the captured image.

[0180] In addition, in the present embodiment, the turnout detection unit 132 of the track detection device 100 sets the length of the turnout region in the traveling direction of the railway vehicle, that is, the longitudinal width, based on the lateral width and the parameters related to the camera 10. Therefore, according to the present embodiment, the switch of the turnout can be more reliably determined from the turnout region set within the range based on the parameters related to the camera 10, and as a result, the open / closed state of the turnout existing in the track can be determined more accurately and easily based on the captured image.

[0181] In addition, in the present embodiment, the switch detection unit 132 of the track detection device 100 sets the length of the branch area in the traveling direction of the railway vehicle RV so that the longer the distance from the current position of the railway vehicle RV to the switch of the switch, the longer the length. Therefore, in the present embodiment, the longitudinal width of the branch area is determined by taking into account that if the distance from the current position of the railway vehicle RV to the switch of the switch is longer, the switch of the switch is photographed longer in the captured image, so that the open / closed state of the switch existing in the track can be determined more accurately and easily based on the captured image.

[0182] In addition, in the present embodiment, the turnout detection unit 132 of the track detection device 100 sets the length of the branch area in the traveling direction of the railway vehicle RV so that the smaller the angle of the vertex of the detected triangular area, the longer the length. Therefore, in the present embodiment, the longitudinal width of the branch area is determined by taking into account that the smaller the angle of the vertex of the triangular area, the longer the turnout is photographed in the captured image. Therefore, the open / closed state of the turnout existing in the track can be determined more accurately and easily based on the captured image.

[0183] In addition, in the present embodiment, the line detection unit 131 of the line detection device 100 outputs a line detection image that classifies the captured image into pixels of a pair of rails, pixels between the pair of rails, and pixels of an area other than the pair of rails and the rails, using the line detection model 1311. Therefore, according to the present embodiment, line detection can be performed more accurately, and as a result, the open / closed state of a switch existing in the line can be determined more accurately and easily based on the captured image.

[0184] In addition, in the present embodiment, the switch state determination unit 133 of the track detection device 100 determines whether the switch state is at least one of left branching, right branching, left merging, right merging, and straight running using the switch state determination model 133. Therefore, according to the present embodiment, the switch state of the switch existing in the track can be determined more accurately and easily based on the captured image.

[0185] In addition, in the present embodiment, the track detection device 100 includes: a storage unit 17 for storing a map DB 171 in which the position information of the turnout in the track and the opening and closing state of the turnout in each time period are associated; a comparison unit 15 for comparing the position information of the turnout detected by the turnout detection unit 132 and the opening and closing state of the turnout determined by the branching state determination unit 133 with the position information of the turnout in the map DB 171 and the opening and closing state of the turnout in the time period including the current time; and a display processing unit 18 for displaying the comparison result of the comparison unit 15 on the display device 30. Therefore, according to the present embodiment, during the travel of the railway vehicle RV, it is possible to always confirm whether the latest state of the track and the turnout is consistent with the registered information in the map DB 171.

[0186] In addition, in the present embodiment, the track detection device 100 includes the communication unit 19, and the communication unit 19 sends a correction instruction of the map DB 171 to the central device 200 when the comparison result is inconsistent. Therefore, during the travel of the railway vehicle RV, the map DB can be updated on the central device 200 side at the timing when it is determined that the latest status of the track and the switch is inconsistent with the map DB 171, and the map DB can be updated quickly.

[0187] In addition, in the present embodiment, the communication unit 19 of the track detection device 100 further receives information on the traveling direction of the railway from the central device 200, and the comparison unit 15 further determines whether the opening and closing state of the turnout determined by the turnout state determination unit 133 is consistent with the traveling direction of the railway vehicle RV received by the communication unit 19 when the comparison result is inconsistent, and the notification unit 20 outputs an alarm when the opening and closing state of the turnout determined by the turnout state determination unit 133 is different from the traveling direction of the railway vehicle RV received by the communication unit 19. Therefore, according to the present embodiment, it is possible to notify in advance that the current opening and closing state of the turnout is different from the traveling direction of the railway vehicle RV.

[0188] The line detection device 100 of this embodiment has a control device such as a CPU, storage devices such as ROM and RAM, external storage devices such as HDD and CD drive devices, display devices such as a display device, and input devices such as a keyboard and mouse, and uses the hardware structure of a normal computer.

[0189] The line detection program executed by the line detection device 100 of the present embodiment is provided by being pre-installed in a ROM or the like.

[0190] The line detection program executed by the line detection device 100 of this embodiment can also be configured as an installable or executable file recorded on a computer-readable recording medium such as a CD-ROM, floppy disk (FD), CD-R, DVD (Digital Versatile Disk) and provided.

[0191] Furthermore, the line detection program executed by the line detection device 100 of the present embodiment may be stored in a computer connected to a network such as the Internet and provided by downloading it via the network. In addition, the line detection program executed by the line detection device 100 of the present embodiment may be configured to be provided or distributed via a network such as the Internet.

[0192] The track detection program executed by the track detection device 100 of the present embodiment is a module structure including the above-mentioned functional units (track detection unit 131, switch detection unit 132, switch state determination unit 133, positioning unit 16, track information generation unit 14, comparison unit 15, display processing unit 18, reporting unit 20, communication unit 19). As actual hardware, the above-mentioned parts are loaded onto the main storage device by the CPU (processor) reading the obstacle sensing program from the above-mentioned ROM and executing the obstacle sensing program, and the functional units (track detection unit 131, switch detection unit 132, switch state determination unit 133, positioning unit 16, track information generation unit 14, comparison unit 15, display processing unit 18, reporting unit 20, communication unit 19) are generated on the main storage device.

[0193] Several embodiments of the present invention have been described, but these embodiments are presented as examples and are not intended to limit the scope of the invention. These new embodiments can be implemented in various other ways, and various omissions, substitutions, and changes can be made without departing from the scope of the subject matter of the invention. These embodiments and their variations are included in the scope and subject matter of the invention, and are included in the invention described in the claims and the scope of their equivalents.

Claims

1. A line detection device, mounted on a railway vehicle, wherein: have: a line detection unit that detects a line consisting of a pair of rails from an image captured by a camera that captures an area around the railway vehicle and outputs a line detection image using a line detection model, the line detection model being a learned model that receives an image as input and outputs the line detection image as a detection result of the line; a turnout detection unit that performs image processing on the track detection image to detect a pair of triangular regions between the rails from the track detection image, and, when the pair of triangular regions are detected, outputs a determination image that determines the vertices of the pair of triangular regions as switches for branching the track into a plurality of tracks or merging the plurality of tracks; a turnout state determination unit that determines the open / closed state of the turnout based on the determined image using a turnout state determination model, wherein the turnout state determination model is a learned model that inputs an image of a switch including the turnout and outputs the open / closed state of the turnout; as well as The output unit outputs a determination result of the open / closed state of the turnout.

2. The line detection device according to claim 1, wherein: The switch detection unit performs an expansion process on the track detection image to expand the width of each of the rails in the track detection image, and detects the pair of triangular areas from the image on which the expansion process has been performed.

3. The line detection device according to claim 1, wherein: The line detection unit detects a plurality of lines from the captured image using the line detection model and outputs the line detection image. The turnout detection unit determines the route on which the railway vehicle is traveling based on the route detection image in which the plurality of routes are detected, detects one or more pairs of triangular areas in front of the determined route on which the railway vehicle is traveling, and outputs the determination image in which the vertices of the triangles of the one or more pairs of triangular areas are determined as switches of one or more turnouts. The switch state determination unit determines the opening and closing state of each of the one or more switches based on a determination image that determines the point switches of the one or more switches using the switch state determination model.

4. The line detection device according to claim 1, wherein: The switch detection unit specifies the vertices of the triangular region as ends of a point switch of the switch, and outputs the specified image in which a predetermined range including the vertices is set as a branching region indicating a range of the point switch.

5. The line detection device according to claim 4, wherein: The switch detection unit sets a lateral width, which is a length of the branch region in a direction intersecting the traveling direction of the railway vehicle, to be longer than a distance between the pair of rails.

6. The line detection device according to claim 5, wherein: The switch detection unit sets a longitudinal width, which is a length of the branching region in the traveling direction of the railway vehicle, based on the lateral width.

7. The line detection device according to claim 5, wherein: The switch detection unit sets a longitudinal width which is a length of the branch region in the traveling direction of the railway vehicle based on the lateral width and a parameter related to the imaging device.

8. The line detection device according to claim 5, wherein: The switch detection unit sets the length of the branch area in the traveling direction of the railway vehicle so that the length becomes longer as the distance from the current position of the railway vehicle to the switch of the switch becomes longer.

9. The line detection device according to claim 5, wherein: The switch detection unit sets the length of the branch area in the traveling direction of the railway vehicle so that the length becomes longer as the angle of the detected apex of the triangular area becomes smaller.

10. The line detection device according to claim 1, wherein: The line detection unit outputs the line detection image using the line detection model, in which the captured image is classified into pixels of the pair of tracks, pixels between the pair of tracks, and pixels of an area other than the pair of tracks and the track space.

11. The line detection device according to claim 1, wherein: The switch state determination unit determines whether the switch state is at least one of left branching, right branching, left merging, right merging, and straight traveling using the switch state determination model.

12. The line detection device according to claim 1, wherein: Also available: A storage unit storing map information corresponding to the position information of the turnout in the line and the opening and closing state of the turnout in each time period; as well as a comparing unit that compares the position information of the turnout detected by the turnout detecting unit and the open / closed state of the turnout determined by the turnout state determining unit with the position information of the turnout and the open / closed state of the turnout in a time period including the current time in the map information; The output unit also displays the comparison result of the comparison unit.

13. The line detection device according to claim 12, wherein: The route detection device further includes a communication unit that transmits a correction instruction for the map information to a central device connected to a network when the comparison result is inconsistent.

14. The line detection device according to claim 13, wherein: The communication unit also receives information related to the traveling direction of the railway vehicle from the central device, The comparison unit further determines whether the open / closed state of the turnout determined by the turnout state determination unit is consistent with the traveling direction of the railway vehicle received by the communication unit when the comparison result is inconsistent, The output unit further outputs an alarm when the open / closed state of the switch determined by the switch state determination unit is different from the traveling direction of the railway vehicle received by the communication unit.

15. A line detection method, performed by a line detection device mounted on a railway vehicle, wherein: The following steps are included: A step of detecting a line consisting of a pair of rails from an image captured by a camera that captures an area around the railway vehicle and outputting a line detection image using a line detection model, the line detection model being a learned model that inputs an image and outputs the line detection image as a detection result of the line; The step of performing image processing on the line detection image, detecting a pair of triangular areas between the rails from the line detection image, and, when the pair of triangular areas are detected, outputting a determination image in which the vertices of the pair of triangular areas are determined as switches for branching the line into a plurality of lines or merging the plurality of lines; The step of determining the open / closed state of the turnout according to the determined image using a turnout state determination model, wherein the turnout state determination model is a learned model that inputs an image of a switch including the turnout and outputs the open / closed state of the turnout; as well as The step of outputting a determination result of the open / closed state of the turnout.

Images