Line detection model learning device, line detection model learning method, and line detection device

Abstract

The purpose of the present invention is to achieve high-precision learning and construction of a line detection model dedicated to a low-illumination image, said line detection model enabling accurate line detection even when a captured image is used in a low-illumination environment. The line detection model learning device includes: a low-illuminance image selection unit that selects a low-illuminance image for learning from image correspondence information of a plurality of high-illuminance images and a plurality of low-illuminance images; a high-illumination image selection unit that selects a high-illumination image corresponding to the low-illumination image for learning; a line detection unit that inputs the selected high-illumination image to a high-illumination image line detection model to detect a line region in the high-illumination image; a teacher data generation unit that generates teacher data from information of a line region in the detected high-illumination image; and a low-illuminance image line detection model construction unit that, on the basis of the low-illuminance image for learning and the teacher data, learns and constructs a low-illuminance image line detection model using a region segmentation algorithm for the low-illuminance image for learning.

Description

Technical Field

[0001] Embodiments of the present invention relate to a line detection model learning device, a line detection model learning method, and a line detection device. Background Art

[0002] In order to realize the automatic operation of railway vehicles, it is necessary to monitor the surroundings such as the front of the railway vehicle to be driven, and it is necessary to detect the line area, that is, the left and right track areas, in real time from the captured images obtained in low-light environments such as bad weather during the day and at night. In the images captured in such low-light environments, the track area of ​​the line is not clear, so it is difficult to extract the edge of the track through image processing technology. Therefore, it is necessary to use deep learning (deep learning) area segmentation algorithms to build a line detection model dedicated to low-light images.

[0003] In order to construct a line detection model dedicated to low-light images, it is necessary to teach (Japanese: teach) the line area to correct data through human visual observation of the learning image obtained by photographing in a low-light environment, thereby allowing the line detection model to learn.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: International Publication No. 2022 / 123641

[0007] Patent Document 2: Japanese Patent Application Publication No. 2019-191961

[0008] Patent Document 3: Japanese Patent Application Publication No. 2020-179798 Summary of the invention

[0009] Technical problem to be solved by the invention

[0010] However, even if images captured in a low-light environment are observed with the naked human eye, it is difficult to accurately represent the line area, and teaching becomes incomplete. As a result, it becomes difficult to learn and build a line detection model dedicated to low-light images.

[0011] The purpose of the implementation method is to provide a high-precision learning of a line detection model dedicated to low-illuminance images and a constructed line detection model learning device, a line detection model learning method and a line detection device. The line detection model dedicated to low-illuminance images can realize accurate line detection even when using images taken in a low-illuminance environment.

[0012] Means used to solve problems

[0013] The line detection model learning device of the embodiment comprises: a storage unit, which stores image correspondence information obtained by establishing correspondence between each of a plurality of high-illuminance images and each of a plurality of low-illuminance images, wherein the plurality of high-illuminance images are obtained by photographing the periphery of a traveling railway vehicle under a high-illuminance condition higher than a prescribed illuminance, and the plurality of low-illuminance images are obtained by photographing the periphery of the traveling railway vehicle under a low-illuminance condition lower than the prescribed illuminance at the same location as the respective shooting locations of the plurality of high-illuminance images; a low-illuminance image selection unit, which selects a low-illuminance image for learning from the image correspondence information; and a high-illuminance image selection unit, which selects a low-illuminance image for learning from the image correspondence information. corresponding to the above-mentioned high illumination image; a line detection unit, inputting the selected high illumination image into a line detection model for high illumination image to detect the line area in the above-mentioned high illumination image, wherein the line detection model for high illumination image is a learned model that inputs the above-mentioned high illumination image and outputs the line area in the above-mentioned high illumination image; a teacher data generating unit, generating the information of the line area in the above-mentioned high illumination image detected as teacher data; a line detection model construction unit for low illumination image, learning and constructing a line detection model for low illumination image based on the above-mentioned low illumination image for learning and the above-mentioned teacher data, using a region segmentation algorithm for the above-mentioned low illumination image for learning, wherein the line detection model for low illumination image is a learned model that inputs the above-mentioned low illumination image and outputs the line area in the above-mentioned low illumination image. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a block diagram showing the schematic structure of a route detection model learning system according to an embodiment of the present invention.

[0015] Figure 2 A diagram showing an example of a functional block of a route detection model learning device according to an embodiment.

[0016] Figure 3 This is an explanatory diagram showing an example of the result of detecting a line region from a captured image using different line detection models for daytime images.

[0017] Figure 4 This is an explanatory diagram showing an example of teaching a route area in a nighttime image.

[0018] Figure 5 A diagram showing an example of an image in which a region segmentation algorithm is applied to a daytime image for learning and a nighttime image for learning according to an embodiment.

[0019] Figure 6 This is a diagram showing an example of association between a daytime image and a nighttime image captured at the same location in the embodiment.

[0020] Figure 7 It is a diagram for explaining an example of a first determination method performed by the same-point determination unit according to the embodiment.

[0021] Figure 8 It is a diagram for explaining an example of a second determination method performed by the same-point determination unit of the embodiment.

[0022] Fig. 9 A diagram showing an example of a tensor used as a feature map generated by the same point identification unit of the embodiment.

[0023] Fig.10 A diagram for explaining an example of similarity calculation of feature maps according to an embodiment.

[0024] Fig.11 It is a diagram for explaining an example of a third determination method performed by the same-point determination unit of the embodiment.

[0025] Fig.12 A diagram showing an example of a daytime image and a nighttime image being associated with each other in the embodiment.

[0026] Fig.13 It is a block diagram showing an example of the functional configuration of the line detection device according to the embodiment.

[0027] Fig.14 This is a flowchart showing an example of the overall processing sequence of the route detection model learning according to the embodiment.

[0028] Fig.15 This is a flowchart showing an example of the procedure of the daytime image and nighttime image association process according to the embodiment.

[0029] Fig.16 This is a flowchart showing an example of the procedure of line detection model learning processing for daytime images according to an embodiment.

[0030] Fig.17 This is a flowchart showing an example of the procedure of the line detection model learning process for nighttime images according to the embodiment.

[0031] Fig.18 This is a flowchart showing an example of the processing sequence of the nighttime image detection circuit according to the embodiment. DETAILED DESCRIPTION

[0032] Hereinafter, embodiments will be described in detail with reference to the drawings.

[0033] (Implementation Method)

[0034] Figure 1This is a block diagram showing the schematic structure of a route detection model learning system according to an embodiment of the present invention.

[0035] The line detection model learning system 10 of this embodiment is mounted on a railway vehicle 1, and mainly includes: an antenna 300, a positioning device 310, a camera 400, a line detection model learning device 100, an inertial sensor 600, a storage device 500, a display device 700, and a line detection device 200. Here, a railway vehicle is sometimes referred to as a vehicle. In addition, in this embodiment, the vehicle is a one-car formation, but it is not limited to this, and it can also be a two-car formation or more.

[0036] The antenna 300 is installed near the center in the width direction of the railway vehicle 1 and receives a radio signal transmitted from an artificial satellite for positioning. The radio signal is a signal including information for measuring the position of the railway vehicle 1 .

[0037] The positioning device 310 is mounted on the railway vehicle 1, and can measure the position of the railway vehicle 1 based on the radio signal from the artificial satellite received by the antenna 300. The position of the railway vehicle 1 is the position information near the center of the left and right tracks of the line. The positioning device 310 is, for example, a GNSS (Global Navigation Satellite System), and performs positioning (satellite positioning) of the railway vehicle 1 by radio received by the antenna 300, and outputs the positioning information as the measurement result to the line detection model learning device 100 and the line detection device 200.

[0038] Here, GNSS is a general term for satellite positioning systems such as the GPS of the United States, the Quasi-Zenith Satellite (QZSS) of Japan, the GLONASS (GLO) of Russia, the Galileo (GAL) of the European Union, and the BeiDou (BDS) of China.

[0039] Therefore, the receiver constituting the antenna 300 and the positioning device 310 is configured to be able to perform positioning under any of these satellite positioning systems.

[0040] The inertial sensor 600 is a self-supporting sensor such as a three-axis acceleration sensor, a three-axis gyro sensor, and a three-axis geomagnetic sensor, for example. The inertial sensor 600 outputs output data of each sensor to the route detection model learning device 100 .

[0041] The camera 400 is installed in the railway vehicle 1 to shoot the surroundings of the railway vehicle 1. Figure 1As shown, the camera 400 is provided at the front side of the railway vehicle 1, and photographs the front of the railway vehicle 1 as an example of the surroundings of the railway vehicle 1. The camera 400 sends the image information obtained by photographing to the line detection model learning device 100 and the line detection device 200. The camera 400 is an example of a photographing device. In addition, the installation position of the camera 400 is not limited to the front side. In addition, the camera 400 is not limited to photographing the front of the railway vehicle 1, and can also be configured to photograph the surroundings such as the side and rear of the railway vehicle 1.

[0042] The storage device 500 is a storage medium such as a HDD (Hard Disc Drive) or a SSD (Solitary State Drive). The storage device 500 stores a route detection model 501 for daytime images and a route detection model 502 for nighttime images. The details of the route detection model 501 for daytime images and the route detection model 502 for nighttime images will be described later. In addition, a map database (map DB) in which the travel route of the railway vehicle 1 is registered may be stored in the storage device 500.

[0043] The display device 700 displays various data from the route detection model learning device 100. The display device 700 is, for example, a monitor or the like.

[0044] The line detection model learning device 100 inputs the current position of the railway vehicle 1 located by the positioning device 310 and the image of the front of the railway vehicle 1 captured by the camera 400, and learns and constructs a line detection model 501 for daytime images and a line detection model 502 for nighttime images. The details of the line detection model learning device 100 will be described later.

[0045] The track detection device 200 inputs the current position of the railway vehicle 1 located by the positioning device 310 and the captured image of the front of the railway vehicle 1 captured by the camera 400, and detects the track area where the railway vehicle 1 is to travel from the captured image using the track detection model 501 for daytime images and the track detection model 502 for nighttime images stored in the storage device 500. Here, the track area is the area where the left and right pairs of rails are located in the captured image. The details of the track detection device 200 will be described later.

[0046] Next, the details of the link detection model learning device 100 will be described.

[0047] Figure 2 It is a block diagram showing an example of the functional configuration of the route detection model learning device 100 according to the embodiment. Figure 2Also shown are an inertial sensor 600 , a positioning device 301 , a camera 400 , and a storage device 500 .

[0048] like Figure 2 As shown, the line detection model learning device 100 of the present embodiment mainly comprises: a location position information acquisition unit 101, a location image information acquisition unit 102, a same location judgment unit 103, an image correspondence establishment unit 104, a daytime image selection unit 105, a line detection unit 106, a nighttime image selection unit 107, a teacher data generation unit 108, a line detection model construction unit for nighttime images 109, a daytime image acquisition unit 110, a line teaching unit 111, a line detection model construction unit for daytime images 112 and a storage unit 150.

[0049] The location-specific image information acquisition unit 102 acquires a plurality of high-illuminance images and low-illuminance images obtained by photographing the front of the railway vehicle 1 by the camera 400 during the travel of the railway vehicle 1. Here, the high-illuminance image is an image obtained by photographing the front of the railway vehicle 1 during the travel under a high-illuminance condition higher than a predetermined illuminance. In addition, the low-illuminance image is an image obtained by photographing the front of the railway vehicle 1 during the travel under a low-illuminance condition lower than a predetermined illuminance.

[0050] In the present embodiment, as the predetermined illumination, an illumination between the illumination during the day and the illumination at night is used as an example. That is, in the present embodiment, the high illumination image is an image captured from the railway vehicle 1 during the daytime period, and hereinafter, the high illumination image is referred to as a daytime image. Here, the daytime period is, for example, the period from sunrise to sunset.

[0051] The low-illuminance image is an image captured from the railway vehicle 1 during the night time period, and is hereinafter referred to as a night time image. Here, the night time period is, for example, the period from sunset to sunrise the next day.

[0052] The location-specific image information acquisition unit 102 stores the acquired plurality of daytime images and the acquired plurality of nighttime images in the storage unit 150 .

[0053] The location-based position information acquisition unit 101 sequentially acquires the current position information of the railway vehicle 1 from the positioning device 310 in synchronization with the acquisition of the daytime image and the acquisition of the nighttime image, and stores the acquired plurality of position information in the storage unit 150. In addition, the location-based position information acquisition unit 101 may further correct the position information based on the sensing signal from the inertial sensor 600.

[0054] The storage unit 150 is a storage medium such as an HDD or an SSD. In the storage unit 150 of the present embodiment, a plurality of daytime images, a plurality of nighttime images, and a plurality of location information are stored. And, as Figure 2 shown, the daytime image, the nighttime image, and the location information of the images determined to be at the same location are associated as image correspondence information 151 (in other words, are associated). In addition, in the example of the image correspondence information 151 of the present embodiment, the location information is associated with the daytime image and the nighttime image of the images determined to be at the same location, but it is sufficient that at least the daytime image and the nighttime image are associated, and it is not necessary to associate the location information.

[0055] Here, the line detection model 501 for daytime images stored in the storage device 500 will be described.

[0056] The line detection model 501 for daytime images is a learned model that inputs a daytime image and outputs the line region where a pair of left and right tracks are located in the daytime image (Japanese: 学習済みモデル). The line detection model 501 for daytime images is constituted by, for example, a neural network and is learned and constructed by machine learning such as deep learning. Specifically, the line detection model 501 for daytime images is pre-generated by a daytime image acquisition unit 110, a line teaching unit 111, and a line detection model construction unit 112 for daytime images, which will be described below.

[0057] The daytime image acquisition unit 110 acquires a plurality of daytime images from the camera 400. The plurality of daytime images are used for learning.

[0058] The line teaching unit 111 receives an input of teaching of the line region to be accepted by the user in each of the plurality of learning daytime images acquired by the daytime image acquisition unit 110. Specifically, the line teaching unit 111 displays each of the acquired plurality of daytime images on the display device 700, allows the user to teach and specify the line region from each of the displayed daytime images, and receives the input of the teaching.

[0059] The line detection model construction unit 112 for daytime images learns and constructs the line detection model 501 for daytime images by machine learning such as deep learning using a region segmentation algorithm for the learning daytime images based on each of the plurality of learning daytime images and the teaching of the line region received by the line teaching unit 111. Here, in the present embodiment, as an example of the region segmentation algorithm, a semantic segmentation algorithm is used. However, the region segmentation algorithm is not limited to this.

[0060] If the line detection model 501 for daytime images is used, the line area detection from the daytime images can be performed with high accuracy. However, in the case of images captured under low illumination conditions such as nighttime images, it is difficult to detect line areas from nighttime images with high accuracy even if the line detection model 501 for daytime images is used. If the illumination conditions vary in this way, the visibility of the line in the image changes greatly, making it difficult to learn the line area at night.

[0061] Figure 3 This is an explanatory diagram showing an example of the result of detecting a line region from a captured image using a different line detection model 501 for daytime images. Figure 3 (a), (b), and (c) show examples of using different line detection models 501 for daytime images. Figure 3 The images in the upper rows of (a), (b), and (c) represent daytime images, and the images in the middle and lower rows represent nighttime images.

[0062] like Figure 3 As shown in the figure, in the daytime image in the upper section, the line area is well detected. However, in the case of the nighttime images taken in a low-light environment shown in the middle and lower sections, the line area is over-detected or under-detected, making it difficult to accurately detect the line area.

[0063] Figure 4 FIG. 1 is an explanatory diagram showing an example of teaching a line area in a night image. Figure 4 As shown, in the night image taken in a low-light environment, for example, the distant track portion such as the portion indicated by symbol 5001 is unclear, and it is difficult to accurately teach the track area using the track detection model 501 for the daytime image.

[0064] Therefore, in this embodiment, such a problem is solved by adopting the following configuration.

[0065] Return to Figure 2 The same location determination unit 103 determines whether each of the plurality of daytime images and each of the plurality of nighttime images stored in the storage unit 150 are images taken at the same location based on the position information stored in the storage unit 150 and the image features of the daytime images and the nighttime images stored in the storage unit 150. The details of this determination will be described later.

[0066] The image association unit 104 associates the daytime image and the nighttime image determined by the same location determination unit 103 to be photographed at the same location with each other, and stores the associated images in the storage unit 150 as image association information.

[0067] Next, the nighttime image line detection model 502 stored in the storage device 500 will be described.

[0068] The line detection model 502 for night images is a learned model that inputs a night image and outputs a line area where a pair of left and right tracks in the night image are located. The line detection model 502 for night images is composed of, for example, a neural network, and is learned and constructed by machine learning such as deep learning. Here, the line detection model 502 for night images of this embodiment inputs a daytime image of the same location corresponding to the night image for learning into the line detection model 501 for daytime images and learns the line area in the output daytime image as teacher data. This teacher data is an accurate line area, and by using relevant accurate teacher data, through this embodiment, it is possible to construct a line detection model 502 for night images that can accurately output the line area even for night images.

[0069] Specifically, the line detection model 502 for nighttime images is generated by the nighttime image selection unit 107, the daytime image selection unit 105, the line detection unit 106, the teacher data generation unit 108, and the line detection model construction unit 109 for nighttime images to be described below.

[0070] The night image selection unit 107 selects a night image for learning from the image correspondence information 151 in the storage unit 150. The night image selection unit 107 selects a plurality of night images for learning.

[0071] The daytime image selection unit 105 selects daytime images corresponding to the nighttime images for learning selected by the nighttime image selection unit 107 from the image correspondence information 151 of the storage unit 150 .

[0072] The line detection unit 106 inputs the daytime image selected by the daytime image selection unit 105 to the line detection model for daytime images 501 , and detects the line region output from the line detection model for daytime images 501 as the line region in the daytime image.

[0073] The training data generating unit 108 sets (or generates) information on the track area in the daytime image detected by the track detecting unit 106 as training data.

[0074] The line detection model construction unit 109 for night images learns and constructs a line detection model for night images by machine learning such as deep learning, using a region segmentation algorithm for the night images for learning, based on each of the multiple night images for learning selected by the night image selection unit 107 and each of the teacher data generated by the teacher data generation unit 108 (i.e., the line area of ​​the day image output from the line detection model for day images 501 by inputting the day image corresponding to the night image for learning). Here, in this embodiment, as an example of the region segmentation algorithm, a semantic segmentation algorithm is used.

[0075] Figure 5 FIG. 1 is a diagram showing an example of an image in which a region segmentation algorithm is applied to a daytime image for learning and a nighttime image for learning according to an embodiment. Figure 5 (a) and (b) are examples of daytime images to which the region segmentation algorithm of the daytime image line detection model construction unit 112 is applied. Figure 5 (c) is an example of a nighttime image to which the region segmentation algorithm of the nighttime image line detection model construction unit 109 is applied.

[0076] like Figure 5 As shown in (a), the daytime image line detection model construction unit 112 adds labels such as line tracks and areas outside the line to the learning images. Figure 5 The nighttime image used for learning in (c) is labeled with track and off-track area, and then a region segmentation algorithm such as semantic segmentation is executed.

[0077] Next, the details of the determination performed by the same-location determination unit 103 as to whether the daytime image and the nighttime image are images captured at the same location will be described.

[0078] Figure 6 FIG. 1 is a diagram showing an example of establishing a correspondence between a daytime image and a nighttime image captured at the same location in the embodiment. The same location identification unit 103 identifies the same location based on the image features. Figure 6 As shown in the example of , a daytime image and a nighttime image taken at the same location are associated with each other.

[0079] As a first judgment method, the same location judgment unit 103 extracts edges as image features from each of the multiple daytime images and each of the multiple nighttime images, and judges whether each of the multiple daytime images and each of the multiple nighttime images are images taken at the same location based on the position information and the similarity between the extracted edges.

[0080] Specifically, the same point determination unit 103 first generates an edge image by performing edge extraction on each of the daytime image and the nighttime image.

[0081] Figure 7 It is a diagram for explaining an example of a first determination method performed by the same-point determination unit 103 according to the embodiment. Figure 7 (a) is an example of an acquired night image (ie, original image). Figure 7 (b) means from Figure 7 The image (a) is obtained by edge extraction. Figure 7 In the example of the edge image in (b), the vertical Sobel operator is applied. The edge image is generated similarly for the daytime image.

[0082] Next, the same location determination unit 103 calculates the similarity between the two edge images. More specifically, the same location determination unit 103 selects a plurality of daytime images that are candidates for establishing correspondence in the vicinity of the position information of the shooting time point with respect to a nighttime image. Then, the same location determination unit 103 calculates the similarity between the edge image of the nighttime image and the edge images of the plurality of daytime images that are candidates for establishing correspondence using the following formula (1).

[0083] [Formula 1]

[0084]

[0085] S i,j : The similarity between two edge images

[0086] The pixel value (0 to 255) at pixel position p of the daytime image collected with frame number i

[0087] I ’ p,j : The pixel value at pixel position p of the night image collected with frame number j (0 to 255)

[0088] E: Edge detection filter (vertical Sobel filter)

[0089] P: total number of pixels in the edge image

[0090] Here, in formula (1), for two edge images, the absolute value of the difference in pixel values ​​between pixels at the same two-dimensional coordinates is calculated. The value obtained by dividing the sum of the absolute values ​​of the differences of all pixels by the total number of pixels in the image is calculated. The more similar the two edge images are, the smaller the value is, and the more dissimilar the two edge images are, the larger the value is.

[0091] Then, the same location determination unit 103 determines that the daytime image of the edge image having the smallest similarity value calculated by equation (1) is an image taken at the same location as the nighttime image. The same location determination unit 103 determines whether each of the plurality of nighttime images and each of the plurality of daytime images are taken at the same location by repeating the relevant processing.

[0092] As a second judgment method, the same location judgment unit 103 calculates a mapping of feature quantities serving as image features, i.e., a feature map, based on each of the multiple daytime images and each of the multiple nighttime images, and judges whether each of the multiple daytime images and each of the multiple nighttime images are images taken at the same location based on the similarity between the position information and the calculated feature maps.

[0093] Specifically, the same point identification unit 103 first calculates a feature map of a nighttime image and a feature map of a daytime image.

[0094] Figure 8 It is a diagram for explaining an example of the second determination method performed by the same-point determination unit 103 according to the embodiment. Figure 8 (a) shows an example of calculating feature maps based on nighttime images. Figure 8 (b) shows an example of calculating a feature map based on a daytime image.

[0095] The same point identification unit 103 can use a tensor as a feature map.

[0096] Fig. 9 : is a diagram showing an example of a tensor used as a feature map generated by the same location identification unit 103 of the embodiment. Fig. 9 As shown, a two-dimensional tensor can be used as a feature map, but it is not limited to this. For example, a feature vector (one-dimensional tensor) and a three-dimensional tensor can be used. However, the method of generating a feature map is not limited to this.

[0097] Fig.10 FIG. 1 is a diagram showing an example of similarity calculation of feature maps in an embodiment of the present invention. Fig.10 As shown, regarding the similarity between two feature maps, the same location identification unit 103 may use precision, recall, or F-value as an evaluation index.

[0098] The similarity of feature maps is not limited to these, and for example, the same point identification unit 103 may be configured to use the Euclidean distance between two feature maps as the similarity between feature maps.

[0099] exist Figure 8: The same location determination unit 103 calculates the similarity of the feature map in this way, and as a result, the similarities between the feature map of the night image and the feature maps of the three day images N-1, N, and N+1 are 60%, 90%, and 80%, respectively. Therefore, the same location determination unit 103 determines that the day image N with a feature map similarity of 90% is a day image of the same location as the night image. According to this second determination method, images of the same location can be determined with higher accuracy than the first determination method.

[0100] As a third judgment method, the same location judgment unit 103 detects a line area from each of the multiple daytime images and each of the multiple nighttime images, uses the detected line area as a feature map, and judges whether each of the multiple daytime images and each of the multiple nighttime images are images taken at the same location based on the position information and the similarity between the calculated feature maps.

[0101] Specifically, the same point identification unit 103 detects a road area from each of a plurality of daytime images and each of a plurality of nighttime images.

[0102] Fig.11 It is a diagram for explaining an example of the third determination method implemented by the same-point determination unit 103 of the embodiment. Fig.11 (a) shows an example of a line area detected from a nighttime image. Fig.11 (b) shows an example of a line area detected from a daytime image.

[0103] Next, the same point determination unit 103 calculates each feature map. The method of calculating the feature map is the same as the second determination method.

[0104] Then, the same location determination unit 103 calculates the similarity between the night image and the day image, that is, the similarity of the feature maps of the two images containing the line detection results, and determines the day image with the highest similarity to a certain night image among the multiple candidate day images as a day image of the same location as the night image.

[0105] Here, in order to compensate for the shaking of the railway vehicle 1, the same point determination unit 103 also compares the nighttime image and the daytime image created by correcting the lateral deviation and the rotation.

[0106] Furthermore, in order to cope with the motion blur of the camera 400 caused by the acceleration of the railway vehicle 1 , the same point identification unit 103 also compares blurred images of the nighttime image and the daytime image.

[0107] Furthermore, in order to compensate for the influence of weather, lighting, etc., the same-point determination unit 103 also sets images obtained by adjusting the brightness value, contrast, etc. of the nighttime image and the daytime image as comparison objects.

[0108] Furthermore, the same-point determination unit 103 also sets images generated by adjustment methods other than those described above as comparison targets.

[0109] As described above, the daytime image and the nighttime image determined to be taken at the same location by the same location determination unit 103 are associated with each other as image association information 151 by the image association unit 104 and stored in the storage unit 150 .

[0110] Fig.12 A diagram showing an example in which a daytime image and a nighttime image are associated with each other in the embodiment. Fig.12 (a) is an example of an image showing a line area as a straight line portion of a track, Fig.12 (b) is an example of an image showing a line area which is a curve portion of a track.

[0111] Next, the details of the line detection device 200 will be described.

[0112] Fig.13 1 is a block diagram showing an example of the functional configuration of the line detection device 200 according to the embodiment. Fig.13 As shown, the line detection device 200 of this embodiment mainly includes an image acquisition unit 201 , a line detection unit 202 , and a control unit 203 .

[0113] The image acquisition unit 201 acquires a daytime image and a nighttime image of the traveling railway vehicle 1 through the camera 400 .

[0114] The line detection unit 202 inputs the daytime image acquired by the image acquisition unit 201 into the line detection model 501 for daytime images, and acquires the line area in the daytime image output from the line detection model 501 for daytime images, thereby detecting the line area in the daytime image.

[0115] In addition, in this embodiment, the line detection unit 202 inputs the night image acquired by the image acquisition unit 201 into the line detection model 502 for night images, and acquires the line area in the night image output from the line detection model 502 for night images, thereby detecting the line area in the night image.

[0116] The control unit 203 performs various controls such as travel control of the railway vehicle 1 based on the track area detected by the track detection unit 202 .

[0117] Next, a description will be given of a link detection model learning process performed by the link detection model learning system 10 of the present embodiment having the above configuration.

[0118] Fig.14 This is a flowchart showing an example of the overall processing sequence of the route detection model learning according to the embodiment.

[0119] First, the location-by-location position information acquisition unit 101 , location-by-location image information acquisition unit 102 , same-location identification unit 103 , and image correspondence establishment unit 104 of the route detection model learning device 100 perform a daytime image and nighttime image correspondence establishment process ( S101 ).

[0120] Next, the daytime image acquisition unit 110 , the route teaching unit 111 , and the daytime image route detection model construction unit 112 execute a learning process of the daytime image route detection model 501 ( S102 ).

[0121] Then, the daytime image selection unit 105, the nighttime image selection unit 107, the line detection unit 106, the training data generation unit 108, and the line detection model construction unit for nighttime images 109 perform a learning process of the line detection model for nighttime images 502 (S103).

[0122] Through the above processing, the line detection model 502 for nighttime images is constructed and learned in the storage device 500 .

[0123] Next, the details of the process of associating the daytime image with the nighttime image in S101 will be described.

[0124] Fig.15 This is a flowchart showing an example of the procedure of the daytime image and nighttime image association process according to the embodiment.

[0125] First, the following processing is performed while the railway vehicle 1 is traveling during the daytime period. The location-specific image information acquisition unit 102 acquires a daytime image from the camera 400 (S201). In synchronization with this, the location-specific position information acquisition unit 101 acquires position information of the current position of the railway vehicle 1 from the positioning device 310 (S202). Then, the location-specific image information acquisition unit 102 stores the acquired daytime image in the storage unit 150, and the location-specific position information acquisition unit 101 stores the acquired position information in the storage unit 150 (S203).

[0126] The processes from S201 to S203 are repeatedly executed while the railway vehicle 1 is traveling during the daytime.

[0127] Next, the following processing is performed while the railway vehicle 1 is traveling in the night time period. The location-specific image information acquisition unit 102 acquires a nighttime image from the camera 400 (S204). In synchronization with this, the location-specific position information acquisition unit 101 acquires position information of the current position of the railway vehicle 1 from the positioning device 310 (S205). Then, the location-specific image information acquisition unit 102 stores the acquired nighttime image in the storage unit 150, and the location-specific position information acquisition unit 101 stores the acquired position information in the storage unit 150 (S206).

[0128] The processes from S204 to S206 are repeatedly executed while the railway vehicle 1 is traveling at night.

[0129] Next, the same location determination unit 103 determines whether each of the plurality of daytime images and each of the plurality of nighttime images stored in the storage unit 150 are images taken at the same location based on the image features and position information of each image using the above method ( S207 ).

[0130] Next, the image association unit 104 associates the daytime image and the nighttime image determined to be taken at the same location by the same location determination unit 103 (S208). Then, the image association unit 104 further associates the position information with the associated daytime image and nighttime image, and stores the associated position information in the storage unit 150 as image association information 151 (S209).

[0131] Afterwards, processing returns to the calling source.

[0132] Next, Fig.14 The details of the line detection model learning process for the daytime image in S102 of the overall process are explained.

[0133] Fig.16 This is a flowchart showing an example of the procedure of line detection model learning processing for daytime images according to an embodiment.

[0134] First, the daytime image acquisition unit 110 acquires a daytime image for learning from the camera 400 (S401). Next, the line teaching unit 111 teaches the line area in the daytime image for learning by the user or the like (S402). Then, the daytime image line detection model construction unit 112 inputs the taught line area as correct data, i.e., teacher data, together with the daytime image for learning, into the daytime image line detection model 501, and uses the semantic segmentation algorithm to learn and construct the daytime image line detection model 501 (S403).

[0135] The processing from S401 to S403 is repeatedly executed (S404, S404: No) until the predetermined end condition is met. Then, if the predetermined end condition is met in S404 (S404: Yes), the processing returns to the calling source.

[0136] Next, Fig.14 The details of the line detection model learning process for night images in S103 of the overall process are explained.

[0137] Fig.17 This is a flowchart showing an example of the procedure of the line detection model learning process for nighttime images according to the embodiment.

[0138] First, the night image selection unit 107 selects a night image for learning from the storage unit 150 (S501). Next, the day image selection unit 105 selects a day image of the same location associated with the night image selected in S501 from the image correspondence information 151 of the storage unit 150 (S502).

[0139] Next, the line detection unit 106 inputs the daytime image selected in S502 to the line detection model 501 for daytime images, and obtains the line area output from the line detection model 501 for daytime images, thereby detecting the line area (S503). Next, the teacher data generation unit 108 sets the line area in the daytime image detected in S503 as the teacher data (S504).

[0140] Then, the nighttime image line detection model construction unit 109 learns and constructs the nighttime image line detection model 502 based on the nighttime image for learning acquired in S501 and the line area set in S504 as correct data, that is, the teacher data (S505).

[0141] The night image selection unit 107 determines whether the processing of all night images in the image correspondence information 151 of the storage unit 150 has been completed (S506). Then, when the night image selection unit 107 determines that the processing of all night images has not been completed (S506: No), the processing from S501 to S505 is repeatedly executed.

[0142] On the other hand, when the nighttime image selection unit 107 determines in S506 that the processing of all nighttime images has been completed (S506: No), the process returns to the calling source.

[0143] In the above-described processing, the line detection model 502 for nighttime images is learned and constructed.

[0144] Next, a description will be given of a process of detecting a line from a nighttime image performed by the line detection device 200.

[0145] Fig.18 This is a flowchart showing an example of the processing sequence of the nighttime image detection circuit according to the embodiment.

[0146] First, the image acquisition unit 201 acquires a nighttime image of the front of the railway vehicle 1 running at night from the camera 400 (S601). Next, the track detection unit 202 inputs the nighttime image acquired in S601 to the track detection model 502 for nighttime images stored in the storage device 500 (S602). Then, the track detection unit 202 acquires the track area output from the track detection model 502 for nighttime images, thereby detecting the track area (S603). Next, the control unit 203 performs various travel controls based on the track area detected in S603 (S604).

[0147] Thus, in the present embodiment, the line detection model learning device 100 comprises: a storage unit 150 for storing image correspondence information 151 obtained by establishing correspondence between each of a plurality of daytime images and a nighttime image, wherein the plurality of daytime images are obtained by photographing the front of the railway vehicle 1 traveling during a daytime period with a high illumination higher than a predetermined illumination by the camera 400, and the nighttime image is obtained by photographing the front of the railway vehicle 1 traveling during a nighttime period with a low illumination lower than a predetermined illumination by the camera 400 at the same location as the respective photographing locations of the plurality of daytime images and in the same photographing area as the photographing area of ​​the daytime image; a nighttime image selection unit 107 for selecting a nighttime image for learning from the image correspondence information 151; and a daytime image selection unit 105 for storing the image correspondence information 151; A daytime image corresponding to the nighttime image for learning is selected from the image corresponding information 151; the line detection unit 106 inputs the selected daytime image into the line detection model 501 for daytime images to detect the line area in the daytime image, and the line detection model 501 for daytime images is a learned model that inputs a daytime image and outputs the line area in the daytime image; the teacher data generation unit 108 generates the information of the line area in the detected daytime image as teacher data; the line detection model construction unit 109 for nighttime images uses a region segmentation algorithm for the nighttime image for learning to learn and construct a line detection model 502 for nighttime images based on the nighttime image for learning and the teacher data, and the line detection model 502 for nighttime images is a learned model that inputs a nighttime image and outputs the line area in the nighttime image.

[0148] That is, in the present embodiment, the accurate line area detected from a clear daytime image obtained by shooting under high illumination conditions using the line detection model 501 for daytime images is used as teaching data, and the line detection model 502 for nighttime images is learned and constructed based on the accurate line area, i.e., the teaching data, and the nighttime image obtained by shooting at the same location as the daytime image. Therefore, according to the present embodiment, even in the case of shooting under low illumination conditions and using a nighttime image as a low illumination image in which the line area is unclear and difficult to teach, it is possible to learn and construct the line detection model 502 for nighttime images for detecting the accurate line area from the nighttime images with high accuracy. Furthermore, in the present embodiment, by using the line detection model 502 for nighttime images learned in this way, it is possible to accurately perform line detection even in a nighttime image as a low illumination image.

[0149] In addition, in the present embodiment, the line detection model learning device 100 further includes: a location-based image information acquisition unit 102 that acquires a plurality of daytime images and a plurality of nighttime images from the camera 400 and stores the acquired plurality of daytime images and the acquired plurality of nighttime images in the storage unit 150; a location-based position information acquisition unit 101 that acquires position information of the railway vehicle 1 from a positioning device 310 that measures the current position of the railway vehicle 1 in synchronization with the acquisition of the daytime images and the acquisition of the nighttime images, and stores the acquired plurality of position information in the storage unit 150; a same-location determination unit 103 that determines whether each of the plurality of daytime images stored in the storage unit 150 and each of the plurality of nighttime images stored in the storage unit are images taken at the same location as the nighttime images based on the position information and image features of the daytime images and the nighttime images; and an image correspondence establishment unit 104 that establishes correspondence between the daytime images and the nighttime images that are determined to be taken at the same location and stores them in the storage unit 150 as image correspondence information 151. Therefore, according to this embodiment, the daytime image as a high-illuminance image and the nighttime image as a low-illuminance image taken at the same location are associated and maintained, and the daytime image of the same location can be easily obtained from the nighttime image for learning. Therefore, according to this embodiment, the line detection model 502 for nighttime images for detecting accurate line areas from nighttime images can be learned and constructed with high accuracy.

[0150] In addition, in the present embodiment, the same location determination unit 103 of the line detection model learning device 100 extracts edges as image features from each of the plurality of daytime images and each of the plurality of nighttime images, and determines whether each of the plurality of daytime images and each of the plurality of nighttime images are images taken at the same location based on the position information and the similarity between the extracted edges. Therefore, according to the present embodiment, the daytime image and the nighttime image of the same location can be matched with high accuracy. Therefore, according to the present embodiment, the line detection model 502 for nighttime images for detecting accurate line areas from nighttime images can be learned and constructed with higher accuracy.

[0151] In addition, in the present embodiment, the same location determination unit 103 of the line detection model learning device 100 calculates a mapping of feature quantities as image features, i.e., a feature map, based on each of the daytime images of the plurality of high-illuminance images and each of the plurality of nighttime images, and determines whether each of the plurality of daytime images and each of the plurality of nighttime images are images taken at the same location based on the similarity between the location information and the calculated feature map. Therefore, according to the present embodiment, the daytime image and the nighttime image of the same location can be associated with each other with higher accuracy. Therefore, according to the present embodiment, the line detection model 502 for nighttime images for detecting accurate line areas from nighttime images can be learned and constructed with higher accuracy.

[0152] In addition, in the present embodiment, the same location determination unit 103 of the line detection model learning device 100 detects a line area from each of the plurality of daytime images and each of the plurality of nighttime images, uses the detected line area as a feature map, and determines whether each of the plurality of daytime images and each of the plurality of nighttime images are images taken at the same location based on the position information and the similarity between the calculated feature maps. Therefore, according to the present embodiment, the daytime image and the nighttime image of the same location can be associated with each other with higher accuracy. Therefore, according to the present embodiment, the line detection model 502 for nighttime images for detecting accurate line areas from nighttime images can be learned and constructed with higher accuracy.

[0153] In addition, in the present embodiment, the line detection model learning device 100 further includes: a daytime image acquisition unit 110 that acquires a daytime image; a line teaching unit 111 that accepts input of a line area teaching in the acquired daytime image; and a line detection model construction unit 112 for daytime images that learns and constructs a line detection model for daytime images 501 based on the acquired daytime image and the accepted teaching, using a region segmentation algorithm for the acquired daytime image. Therefore, according to the present embodiment, by using the learned and constructed line detection model for daytime images 501, even in the case of using a nighttime image as a low-light image in which the line area is unclear and difficult to teach, which is captured in a low-light environment, it is possible to learn and construct a line detection model for nighttime images 502 for detecting an accurate line area from the nighttime image with high accuracy.

[0154] In addition, in the present embodiment, the region segmentation algorithm is a semantic segmentation algorithm. Therefore, in the present embodiment, the line detection model 502 for nighttime images can be learned and constructed with higher accuracy.

[0155] In addition, the track detection device 200 of the present embodiment includes: an image acquisition unit 201, which acquires a night image as a low-illuminance image obtained by photographing the front of the railway vehicle 1 running in the night time period with low illumination lower than the prescribed illumination by the camera 400; and a track detection unit 202, which inputs the acquired night image to the track detection model 502 for night images, detects the track area, and the track detection model 502 for night images is a learned model that inputs the night image and outputs the track area in the night image, and uses the track area in the daytime image detected by inputting the daytime image to the track detection model 501 for daytime images as teacher data, and uses the area segmentation algorithm to perform learning based on the night image for learning and the teacher data. The above-mentioned daytime image is obtained by photographing the front of the railway vehicle 1 running in the daytime time period with high illumination higher than the prescribed illumination by the camera 400, and the track detection model 501 for daytime images is a learned model that inputs the daytime image and outputs the track area in the daytime image.

[0156] That is, in this embodiment, even when a night image is used as a low-illuminance image in which the line area is unclear and difficult to teach, it is possible to perform line detection using the line detection model 502 for night images that can detect accurate line areas from the night images. Therefore, according to this embodiment, by using the line detection model 502 for night images that is learned in this way, it is possible to accurately perform line detection even in night images that are low-illuminance images.

[0157] (Variation Example)

[0158] In the above-described embodiment, the line detection model for nighttime images 502 is provided separately from the line detection model for daytime images 501 and the line detection model for nighttime images 502 is learned, but the present invention is not limited thereto.

[0159] For example, the line detection model construction unit 109 for night images may be configured to learn the line detection model for day images 501 using a region segmentation algorithm for the night images for learning, based on the night images for learning and the teacher data of the line area output from the line detection model for day images 501, and to construct the learned line detection model for day images 501 into the line detection model for night images 502. Thus, since the line detection model for day images 501 is learned and constructed into the line detection model for night images 502, the processing efficiency of learning can be improved.

[0160] In addition, in the above-mentioned embodiment, the line detection model 501 for daytime images and the line detection model 502 for nighttime images are stored in the storage device 500 provided separately from the line detection model learning device 100 and the line detection device 200, but the present invention is not limited thereto. For example, a configuration in which the line detection model 501 for daytime images and the line detection model 502 for nighttime images are provided in the storage unit 150 of the line detection model learning device 100 or the like, or a configuration in which the line detection model 501 for daytime images and the line detection model 502 for nighttime images are provided in the line detection device 200 may be adopted. In this way, the device configuration can be simplified.

[0161] In the above embodiment, a daytime image is used as a high-illuminance image, and a nighttime image is used as a low-illuminance image, but the present invention is not limited thereto. For example, it is possible to use an image captured during a sunny daytime period as a high-illuminance image, and an image captured during a daytime period (and a nighttime period) under bad weather as a low-illuminance image. In this case, the same effects as those in the above embodiment are achieved.

[0162] The line detection model learning device 100 of the above-mentioned embodiment has a control device such as a CPU, a storage device such as a ROM (Read Only Memory) or a RAM, an external storage device such as a HDD, SSD, a CD drive device, a display device such as a display device, and an input device such as a keyboard or a mouse, thereby utilizing the hardware structure of a conventional computer.

[0163] The link detection model learning program executed by the link detection model learning device 100 of the above-described embodiment is provided by being embedded in a ROM or the like in advance.

[0164] The line detection model learning program executed by the line detection model learning device 100 of the above-mentioned embodiment can also be configured as an installable or executable file recorded in a recording medium that can be read by a computer, such as a CD-ROM, a floppy disk (FD: flexible disk), a CD-R, or a DVD (Digital Versatile Disk: digital video disc).

[0165] Furthermore, the line detection model learning program executed by the line detection model learning device 100 of the above-mentioned embodiment may be stored in a computer connected to a network such as the Internet, and provided by downloading via the network. Furthermore, the line detection model learning program executed by the line detection model learning device 100 of the above-mentioned embodiment may be provided or distributed via a network such as the Internet.

[0166] The line detection model learning program executed by the line detection model learning device 100 of the above-mentioned embodiment becomes a module structure including the above-mentioned functional units. As actual hardware, the CPU (processor) reads out the line detection model learning program from the above-mentioned ROM and executes it. Thus, the above-mentioned functional units (location position information acquisition unit 101, location image information acquisition unit 102, same location judgment unit 103, image correspondence establishment unit 104, daytime image selection unit 105, line detection unit 106, nighttime image selection unit 107, teacher data generation unit 108, line detection model construction unit 109 for nighttime images, daytime image acquisition unit 110, line teaching unit 111, line detection model construction unit 112 for daytime images) are installed on the main storage device, and each functional unit is generated on the main storage device.

[0167] The line detection device 200 of the above-mentioned embodiment has a control device such as a CPU, a storage device such as a ROM or RAM, an external storage device such as a HDD, SSD, or CD drive, a display device such as a display device, and an input device such as a keyboard or a mouse, and utilizes the hardware configuration of a normal computer.

[0168] The line detection program executed by the line detection device 200 of the above-described embodiment is provided by being embedded in a ROM or the like in advance.

[0169] The line detection program executed by the line detection device 200 of the above-mentioned embodiment may be provided as an installable or executable file recorded on a computer-readable recording medium such as a CD-ROM, a floppy disk (FD), a CD-R, or a DVD.

[0170] Furthermore, the line detection program executed by the line detection device 200 of the above-mentioned embodiment may be stored in a computer connected to a network such as the Internet, and provided by downloading via the network. Furthermore, the line detection program executed by the line detection device 200 of the above-mentioned embodiment may be provided or distributed via a network such as the Internet.

[0171] The line detection program executed by the line detection device 200 of the above-mentioned embodiment becomes a module structure including the above-mentioned functional units. As actual hardware, the CPU (processor) reads out the line detection model learning program from the above-mentioned ROM and executes it. Therefore, the above-mentioned functional units (image acquisition unit 201, line detection unit 202, control unit 203) are installed on the main storage device, and each functional unit is generated on the main storage device.

[0172] Several embodiments of the present invention have been described, but these embodiments are disclosed as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other ways, and various omissions, substitutions, and changes can be made without departing from the scope of the invention. These embodiments or their variations are included in the scope and purpose of the invention, and are included in the invention described in the claims and their equivalents.

[0173] Explanation of symbols

[0174] 1 Railway vehicles

[0175] 10 Line Detection Model Learning System

[0176] 100 Line detection model learning device

[0177] 101 Location information acquisition unit

[0178] 102 Location-based image information acquisition unit

[0179] 103 Same location identification unit

[0180] 104 Image correspondence establishment unit

[0181] 105 Daytime Image Selection Department

[0182] 106 Line Detection Department

[0183] 107 Night Image Selection Unit

[0184] 108 Teacher Data Generation Department

[0185] 109 Line detection model construction for night images

[0186] 110 Daytime Image Acquisition Unit

[0187] 111 Line Teaching Department

[0188] 112Route detection model construction department for daytime images

[0189] 150 Storage

[0190] 151 Image Correspondence Information

[0191] 200 Line detection device

[0192] 201 Image acquisition unit

[0193] 202 Line Detection Department

[0194] 203 Control Department

[0195] 300 Antenna

[0196] 310 Positioning device

[0197] 400 cameras

[0198] 500 Storage Device

[0199] 501 Line Detection Model for Daytime Images

[0200] 502 Line Detection Model for Nighttime Images

[0201] 600 Inertial Sensors

[0202] 700 Display Device

Claims

1. A line detection model learning device, wherein: have: a storage unit storing image correspondence information obtained by establishing correspondence between each of a plurality of high-illuminance images obtained by photographing the periphery of a traveling railway vehicle under a high-illuminance condition higher than a predetermined illuminance by a photographing device and each of a plurality of low-illuminance images obtained by photographing the periphery of the traveling railway vehicle under a low-illuminance condition lower than the predetermined illuminance at the same location as the respective photography locations of the plurality of high-illuminance images; A low-light image selection unit, which selects a low-light image for learning from the above-mentioned image correspondence information; a high-illuminance image selection unit that selects the high-illuminance image corresponding to the low-illuminance image for learning from the image correspondence information; A line detection unit inputs the selected high-illuminance image into a line detection model for high-illuminance image to detect a line area in the high-illuminance image, wherein the line detection model for high-illuminance image is a learned model that inputs the high-illuminance image and outputs a line area in the high-illuminance image; A teacher data generating unit generates the information of the line area detected in the high-illuminance image as teacher data; as well as The line detection model construction unit for low-illuminance images learns and constructs a line detection model for low-illuminance images based on the above-mentioned low-illuminance images for learning and the above-mentioned teacher data, using a region segmentation algorithm for the above-mentioned low-illuminance images for learning. The line detection model for low-illuminance images inputs the above-mentioned low-illuminance images and outputs a learned model of the line area in the above-mentioned low-illuminance images.

2. The line detection model learning device according to claim 1, wherein: Also available: The location-based image information acquisition unit acquires the plurality of high-illuminance images and the plurality of low-illuminance images from the photographing device, and stores the acquired plurality of high-illuminance images and the acquired plurality of low-illuminance images in the storage unit; a location information acquisition unit for acquiring location information of the railway vehicle from a positioning device that measures the current location of the railway vehicle in synchronization with the acquisition of the high illumination image and the acquisition of the low illumination image, and storing the acquired plurality of location information in the storage unit; a same-location determining unit, which determines whether each of the plurality of high-illuminance images stored in the storage unit and each of the plurality of low-illuminance images stored in the storage unit are images taken at the same location based on the position information and image features of the high-illuminance image and the low-illuminance image; as well as The image association unit associates the high-illuminance image and the low-illuminance image determined to be captured at the same location with each other and stores the associated images in the storage unit as the image association information.

3. The line detection model learning device according to claim 2, wherein: The above-mentioned same-location determination unit extracts edges serving as the above-mentioned image features from each of the above-mentioned multiple high-illuminance images and each of the above-mentioned multiple low-illuminance images, and determines whether each of the above-mentioned multiple high-illuminance images and each of the above-mentioned multiple low-illuminance images are images taken at the same location based on the above-mentioned position information and the similarity between the extracted edges.

4. The line detection model learning device according to claim 2, wherein: The above-mentioned same-location determination unit calculates a mapping of the feature quantity serving as the above-mentioned image feature, i.e., a feature map, based on each of the above-mentioned multiple high-illuminance images and each of the above-mentioned multiple low-illuminance images, and determines whether each of the above-mentioned multiple high-illuminance images and each of the above-mentioned multiple low-illuminance images are images taken at the same location based on the similarity between the above-mentioned position information and the calculated feature map.

5. The line detection model learning device according to claim 4, wherein: The above-mentioned same-location determination unit detects a line area from each of the above-mentioned multiple high-illuminance images and each of the above-mentioned multiple low-illuminance images, uses the detected line area as the above-mentioned feature map, and based on the above-mentioned position information and the similarity between the calculated feature maps, determines whether each of the above-mentioned multiple high-illuminance images and each of the above-mentioned multiple low-illuminance images are images taken at the same location.

6. The line detection model learning device according to claim 1, wherein: Also available: A high illumination image acquisition unit, which acquires the high illumination image; A line teaching unit receives input of teaching of a line area in the acquired high-illuminance image; and The high-illuminance image line detection model construction unit learns and constructs the high-illuminance image line detection model based on the acquired high-illuminance image and the received teaching, using the region segmentation algorithm for the acquired high-illuminance image.

7. The line detection model learning device according to claim 1, wherein: The above region segmentation algorithm is a semantic segmentation algorithm.

8. The line detection model learning device according to claim 1, wherein: The line detection model construction unit for low-illuminance images learns the line detection model for high-illuminance images based on the low-illuminance images for learning and the teacher data, using a region segmentation algorithm for the low-illuminance images for learning, and constructs the learned line detection model for high-illuminance images as the line detection model for low-illuminance images.

9. The line detection model learning device according to claim 1, wherein: The high-illuminance image is a daytime image obtained by photographing the periphery of the railway vehicle traveling during the daytime. The low-illuminance image is a nighttime image obtained by photographing the periphery of the railway vehicle traveling at night.

10. A line detection model learning method performed by a line detection model learning device, wherein: The above-mentioned line detection model learning device has: a storage unit storing image correspondence information obtained by establishing correspondence between each of a plurality of high-illuminance images and each of a plurality of low-illuminance images, wherein the plurality of high-illuminance images are obtained by photographing the periphery of a traveling railway vehicle under a high-illuminance condition higher than a prescribed illuminance by a photographing device, and the plurality of low-illuminance images are obtained by photographing the periphery of the traveling railway vehicle under a low-illuminance condition lower than the prescribed illuminance at the same location as the respective photographing locations of the plurality of high-illuminance images, The above line detection model learning method includes the following steps: Selecting a low-light image for learning from the above image correspondence information; selecting the high-illuminance image corresponding to the low-illuminance image for learning from the image correspondence information; Inputting the selected high-illuminance image to a line detection model for high-illuminance image to detect the line area in the high-illuminance image, wherein the line detection model for high-illuminance image is a learned model that inputs the high-illuminance image and outputs the line area in the high-illuminance image; generating information of the detected line area in the high-illuminance image as teacher data; and Based on the above-mentioned low-illumination image for learning and the above-mentioned teacher data, a region segmentation algorithm for the above-mentioned low-illumination image for learning is used to learn and construct a line detection model for the low-illumination image. The line detection model for the low-illumination image is a learned model that inputs the above-mentioned low-illumination image and outputs the line area in the above-mentioned low-illumination image.

11. A line detection device, wherein: have: An image acquisition unit acquires a low-illuminance image obtained by photographing the periphery of a traveling railway vehicle under a low-illuminance condition lower than a predetermined illuminance by a photographing device; as well as The line detection unit uses the line detection model to input the acquired low-light image and detect the line area. The line detection model for low-light images is a learned model that inputs the low-light image and outputs the line area in the low-light image. The line detection model for low illumination images uses the line area in the high illumination image detected by inputting the high illumination image into the line detection model for high illumination images as teacher data, and uses a region segmentation algorithm to perform learning based on the low illumination image for learning and the teacher data. The high illumination image is obtained by photographing the surroundings of the traveling railway vehicle under a high illumination condition higher than the specified illumination. The line detection model for high illumination images is a learned model that inputs the high illumination image and outputs the line area in the high illumination image.

Images