Detection system and detection method

Abstract

To detect abnormalities in railroad tracks with high accuracy. [Solution] The detection system 1 is mounted on a railway vehicle running on a pair of rails R and comprises an imaging unit 21 that outputs an image 50 showing the pair of rails R, a first detection unit 42 that inputs the image 50 into a first trained model to detect the pair of rails R, a generation unit 43 that generates preprocessed images 56, 57 from the image 50 based on the detection results of the first detection unit 42, showing the detection target portions A1, A2 of the pair of rails R, and a second detection unit 44 that inputs the preprocessed images 56, 57 into a second trained model to detect deformations D1, D2 in the detection target portions A1, A2.

Description

[Technical Field]

[0001] The present disclosure relates to detection systems and methods. [Background technology]

[0002] Patent document 1 describes a railway track abnormality detection device that includes an image reading means for reading an image of the railway track, a storage means for storing the read image, a comparison means for comparing the inspection image read by the image reading means with a reference image previously stored in the storage means, and an output means for identifying and outputting abnormal locations on the railway track based on the comparison results. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 3-250103 Summary of the Invention [Problem to be solved by the invention]

[0004] The anomaly detection device described above identifies anomalies in the railway track by comparing the inspection image with a reference image stored in advance in a storage means. It is desirable to detect anomalies in the railway track with high accuracy.

[0005] The present disclosure aims to provide a detection system and a detection method that can detect deformations in rails of railway tracks with high accuracy. [Means for solving the problem]

[0006] The detection system of the present disclosure is [1] "a detection system comprising: an imaging unit mounted on a railway vehicle traveling on a pair of rails, which outputs an image of the pair of rails; a first detection unit which inputs the image to a first trained model to detect the pair of rails; a generation unit which generates a preprocessed image from the image based on the detection result of the first detection unit, which image is a detection target portion that is at least a part of the pair of rails; and a second detection unit which inputs the preprocessed image to a second trained model to detect any abnormalities in the detection target portion."

[0007] In this detection system, an imaging unit mounted on a railway vehicle traveling on the pair of rails can efficiently capture images of the pair of rails. Then, a generation unit generates a preprocessed image based on the detection results of the first detection unit, and a second detection unit inputs the preprocessed image into a second trained model to detect abnormalities in the detection target portion of the pair of rails. This makes it possible to detect abnormalities in the detection target portion with high accuracy from the preprocessed image based on the detection results of the first detection unit. As a result, abnormalities in the rails can be detected with high accuracy.

[0008] The detection system of the present disclosure may be the detection system described in [1], [2] in which "the generation unit generates a first image depicting the detection target portion of a first rail, one of the pair of rails; a second image depicting the detection target portion of a second rail, the other of the pair of rails; and an inverted image in which one of the first image and the second image is inverted left and right; and the second detection unit inputs the inverted image and the other of the first image and the second image as the preprocessed image into the second trained model to detect the deformation in the detection target portion of each of the first rail and the second rail." For example, deformations in a pair of rails tend to occur on the inside of the pair of rails. Therefore, the deformations are located on the same side in the inverted image generated by the generation unit and the other of the first image and the second image, and the deformation detection conditions can be aligned. By inputting images with aligned detection conditions into the second trained model in this manner, deformations can be detected with higher accuracy.

[0009] The detection system of the present disclosure may be [3] "the detection system according to [2], in which the generation unit generates the first image and the second image from the lower region when the captured image is divided into two in the vertical direction." In this case, the first image and the second image can be generated from an image portion that captures an area relatively close to the imaging unit mounted on the railway vehicle. Since the first image and the second image show anomalies relatively clearly, the use of these images allows for more accurate detection of anomalies.

[0010] The detection system of the present disclosure may be [4] "the detection system according to any one of [1] to [3], wherein the generation unit generates the preprocessed image depicting the rectangular detection target portion by performing projective transformation on an image depicting the detection target portion of the pair of rails." In this case, the shape of the detection target portion depicted in the preprocessed image can be made uniform, thereby stabilizing the accuracy of detecting anomalies.

[0011] The detection system of the present disclosure may be [5] "the detection system according to any one of [1] to [4], wherein the generation unit generates the preprocessed image by converting an area other than the detection target portion in an image showing the detection target portion of the pair of rails into a single color." In this case, the range of inference by the second trained model can be narrowed to the detection target portion, thereby enabling more accurate detection of anomalies.

[0012] The detection system of the present disclosure may be [6] "the detection system according to any one of [1] to [5], wherein the generation unit generates a plurality of the preprocessed images by vertically dividing an image showing the detection target portion of the pair of rails, and the second detection unit inputs the plurality of the preprocessed images to the second trained model to detect the deformation in the detection target portion." In this case, the range of inference by the second trained model can be narrowed, thereby enabling deformation to be detected with higher accuracy.

[0013] The detection method of the present disclosure is [7] "a detection method executed by a detection system, comprising: an imaging step of outputting an image of a pair of rails from a railway vehicle traveling on the pair of rails; a first detection step of inputting the image to a first trained model to detect the pair of rails; a generation step of generating, from the image, a preprocessed image of a detection target portion that is at least a part of the pair of rails based on the detection result of the first detection step; and a second detection step of inputting the preprocessed image to a second trained model to detect a defect in the detection target portion." This detection method, for the reasons described above, enables highly accurate detection of defects in rails. [Effects of the Invention]

[0014] According to the present disclosure, it is possible to detect deformations in rails of railway tracks with high accuracy. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a diagram showing the configuration of a detection system. [Figure 2] FIG. 2 is a block diagram showing the configuration of the imaging device. [Figure 3] FIG. 3 is a block diagram showing a general hardware configuration of a processing server. [Figure 4] FIG. 4 is a block diagram showing the configuration of the processing server. [Figure 5] FIG. 5 is a diagram schematically illustrating the positional relationship between the imaging device and the target object in the embodiment. [Figure 6] FIG. 6 is a flowchart illustrating an example of a detection method in the detection system. [Figure 7] FIG. 7 is a diagram illustrating an example of a method for processing a captured image by the detection system. [Figure 8] FIG. 8 is a flowchart showing an example of a learning method for the first learned model. [Figure 9] FIG. 9 is a flowchart showing an example of a learning method for the second learned model. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, preferred embodiments of the detection system according to the present invention will be described in detail with reference to the drawings. In the description of the drawings, the same or corresponding parts are designated by the same reference numerals, and duplicated explanations will be omitted.

[0017] [Detection system configuration] First, the overall configuration of a detection system 1 according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing the configuration of the detection system 1. The detection system 1 is a system that detects defects in a pair of rails of a railway track. The railway track includes, for example, a pair of rails R and sleepers T (see Fig. 5) installed on a trackbed. Defects in the rails R include, for example, spalling (gauge corner spalling, normal joint head spalling), shelling, and corrugation. In this example, the detection system 1 performs the above detection from an image of the rails R.

[0018] The detection system 1 includes an imaging device 2, an intermediate server 3, a processing server 4, a database 5, and a display device 6. The imaging device 2 is connected to the intermediate server 3 via a network N. The intermediate server 3 is connected to the processing server 4 via the network N. Images from the imaging device 2 are transmitted to the processing server 4 via the intermediate server 3. The number of intermediate servers 3 may be any number equal to or greater than one. The processing server 4 processes images from the imaging device 2. The processing server 4 is connected to a database 5 and a display device 6 via the network N. Images from the imaging device 2 may be transmitted to the processing server 4 without passing through the intermediate server 3. Alternatively, images from the imaging device 2 may be transmitted to the database 5 without passing through both the intermediate server 3 and the processing server 4. In this case, the processing server 4 acquires images from the database 5 and processes the images. The database 5 stores images processed by the processing server 4. The display device 6 outputs the processing results of the processing server 4. The processing server 4 may be connected to the database 5 and the display device 6 via a local cable. The imaging device 2 is mounted on a railway vehicle. On the other hand, the intermediate server 3, processing server 4, database 5, and display device 6 do not have to be mounted on a railway vehicle, but may be placed, for example, in facilities (such as stations) along the railway line.

[0019] The imaging device 2 captures an image of a pair of rails R. A railway vehicle is a vehicle that travels on the pair of rails R, such as a locomotive or a train. The imaging device 2 is mounted, for example, on the lead car of the railway vehicle. As shown in FIG. 2, the imaging device 2 includes an imaging unit 21, a computer 22, a sensor 23, a communication device 24, an input device 25, a display device 26, and a power supply 27. FIG. 2 is a block diagram showing the configuration of the imaging device 2. A user inputs operating conditions to the above devices into the input device 25. The display device 26 outputs an image captured by the imaging unit 21. The power supply 27 supplies power to the devices that make up the imaging device 2.

[0020] The imaging unit 21 includes an upward-facing camera 21a, a front-facing camera 21b, and a downward-facing camera 21c. The cameras 21a to 21c are, for example, monocular cameras, stereo cameras, or TOF (Time Of Flight) cameras. When the cameras 21a to 21c are monocular cameras, the portability of the imaging device 2 and the ease of installing the imaging device 2 on a railway vehicle are improved. Each of the cameras 21a to 21c includes an imaging lens 21d, a control unit 21e, and a status display unit 21f.

[0021] Upward-facing camera 21a captures an image diagonally upward with respect to the horizontal direction. Upward-facing camera 21a is mounted on the railway vehicle so that imaging lens 21d faces diagonally upward with respect to the horizontal direction. Front-facing camera 21b captures an image directly ahead in the direction of travel of the railway vehicle. Front-facing camera 21b is mounted on the railway vehicle so that imaging lens 21d faces horizontally. Downward-facing camera 21c captures an image diagonally downward with respect to the horizontal direction. By using cameras 21a to 21c that are appropriate for the position of the object to be imaged, it is possible to preferably image the object.

[0022] The cameras 21a to 21c output a plurality of captured images, including an image showing the rail R, to the calculator 22. The plurality of captured images may include an image not showing the rail R in addition to an image showing the rail R. The cameras 21a to 21c may output only an image showing the rail R, without outputting an image not showing the rail R. Each of the plurality of captured images may be a video or a still image. The frame rate of each of the cameras 21a to 21c is, for example, 5 fps to 60 fps, and is, for example, 10 fps. The shutter speed of each of the cameras 21a to 21c is, for example, 1 ms to 5 ms, and is, for example, 2 ms. The frame rate and shutter speed may be appropriately adjusted by the control unit 21e depending on the traveling speed of the railway vehicle. The imaging unit 21 may include at least one of the cameras 21a to 21c.

[0023] The computer 22 is connected to the imaging unit 21, the sensor 23, the communication device 24, the input device 25, and the display device 26, and controls these devices. The computer 22 also stores the captured images from the imaging unit 21. The sensor 23 acquires position information or acceleration information of the railway vehicle and outputs it to the computer 22. The sensor 23 is, for example, a GPS sensor or an acceleration sensor. The computer 22 transmits the position information or acceleration information of the railway vehicle in addition to the captured images to the intermediate server 3 via the communication device 24.

[0024] 3 is a block diagram showing a general hardware configuration of the processing server 4. The processing server 4 includes a CPU (processor) 101 that executes an operating system, application programs, etc., a main memory unit 102 consisting of ROM and RAM, an auxiliary memory unit 103 consisting of a hard disk, flash memory, etc., a communication control unit 104 consisting of a network card or wireless communication module, input devices 105 such as a keyboard and a mouse, and output devices 106 such as a display and a printer.

[0025] Each functional element of the processing server 4, which will be described later, is realized by loading predetermined software onto the CPU 101 or main memory unit 102, operating the communication control unit 104, input device 105, output device 106, display device 6, etc. under the control of the CPU 101, and reading and writing data from and to the main memory unit 102 or auxiliary memory unit 103. Data and databases required for processing are stored in the main memory unit 102 or auxiliary memory unit 103.

[0026] When a moving image is transmitted from the imaging device 2, the processing server 4 divides the moving image into frames to generate still images. The processing server 4 adds position information and acceleration information of the railway vehicle to the still images, and stores the still images and the information in a predetermined storage device. The processing server 4 performs image processing on the generated still images.

[0027] 4 is a block diagram showing the configuration of the processing server 4. The processing server 4 includes a learning unit 41, a first detection unit 42, a generation unit 43, and a second detection unit 44 as functional components.

[0028] The learning unit 41 trains a learning model by machine learning using teacher data to learn (construct) a first learned model and a second learned model. The learning model is, for example, a convolutional neural network (CNN). The first detection unit 42 inputs a captured image to the first learned model and detects a pair of rails R from the captured image. The generation unit 43 generates a preprocessed image that captures a detection target portion, which is at least a part of the pair of rails R, from the captured image based on the detection result of the first detection unit 42. The second detection unit 44 inputs the preprocessed image to the second learned model and detects a deformation in the detection target portion of the pair of rails R.

[0029] [Detection method] Next, a detection method according to an embodiment using the detection system 1 will be described. Fig. 5 is a diagram schematically illustrating the positional relationship between the imaging device 2 and a pair of rails R. In this example, the imaging device 2 uses a downward-facing camera 21c as the imaging unit 21. The downward-facing camera 21c captures images at an angle of 45 degrees diagonally downward relative to the horizontal, for example.

[0030] Fig. 6 is a flowchart showing, as a flow S1, an example of a detection method in the detection system 1. Fig. 7 is a diagram showing an example of a method of processing a captured image by the detection system 1.

[0031] In step S11 (imaging step), the imaging unit 21 outputs a captured image 50 showing a pair of rails R. The imaging device 2 transmits the captured image 50 to the processing server 4. In this example, the captured image 50 shows a first rail R1, which is one of the pair of rails R, and a second rail R2, which is the other of the pair of rails R. A deformation D1 is located on the first rail R1, and a deformation D2 is located on the second rail R2. The number of deformations on each of the first rail R1 and the second rail R2 may be any number equal to or greater than one.

[0032] In step S12 (first detection step), the first detection unit 42 inputs the captured image 50 into a first trained model and detects a pair of rails R. In this example, the first trained model is used for segmentation. The first trained model classifies each pixel of the captured image 50 as either a pair of rails R or the rest of the background. In this example, the first detection unit 42 detects the pair of rails R, and then detects the first rail R1 and the second rail R2 by separately classifying the first rail R1 and the second rail R2 through a rail classification process. As part of the rail classification process, the first detection unit 42 first identifies the coordinates of the upper ends of each of the two regions detected as the pair of rails R. Then, the first detection unit 42 compares the coordinates of the identified two upper ends to classify the two regions into a region of the first rail R1 and a region of the second rail R2, and detects the first rail R1 and the second rail R2. Similarly, the first detector 42 may detect the first rail R1 and the second rail R2 by identifying the coordinates of the lower ends of the two regions.

[0033] In step S13 (generation step), the generation unit 43 generates an image depicting a detection target portion, which is part of the pair of rails R, from the captured image 50 based on the detection result of the first detection unit 42. The detection target portion is a portion of the pair of rails R depicted in the captured image 50. The detection target portion is a portion that is the subject of inference by a second trained model, which will be described later. In this example, the generation unit 43 generates, from the captured image 50, a first image 51 depicting the detection target portion A1 of the first rail R1 and a second image 52 depicting the detection target portion A2 of the second rail R2. Each of the first image 51 and the second image 52 is also a partial image obtained by cutting out a portion of the captured image 50 into a rectangular shape.

[0034] The generation unit 43 generates the first image 51 and the second image 52 from the lower region when the captured image 50 is divided into two parts in the vertical direction. For example, the generation unit 43 generates the first image 51 and the second image 52 from the lower region when the captured image 50 is divided into two equal parts in the vertical direction. Alternatively, the generation unit 43 may generate the first image 51 and the second image 52 from the lowermost region when the captured image 50 is divided into three equal parts in the vertical direction. In other words, the generation unit 43 generates the first image 51 and the second image 52 from an image portion that captures an area relatively close to the imaging unit 21.

[0035] In step S14, the generator 43 flips the second image 52 left and right to generate an inverted image 53. Generally, the deformations D1 and D2 tend to occur on the inner side of a pair of rails R. For this reason, the deformation D2 in the inverted image 53 tends to be located on the same side as the deformation D1 in the first image 51. In this example, the deformations D1 and D2 are located on the left side of FIG. 7. The generator 43 may flip the first image 51 left and right to generate an inverted image. Note that FIG. 7 illustrates the first image 51 and the inverted image 53 after projective transformation has been performed in step S15, which will be described later.

[0036] In step S15, the generation unit 43 performs projective transformation on the first image 51 and the inverted image 53. The pair of rails R are depicted in the captured image 50 so as to form the sides (hypotenuses) of a trapezoid. Therefore, the detection target portion A1 of the first rail R1 is depicted at an angle in the first image 51, and the detection target portion A2 of the second rail R2 is depicted at an angle in the second image 52 (inverted image 53). By performing projective transformation on the first image 51, the generation unit 43 converts the shape of the detection target portion A1 in the first image 51 from a trapezoid to a rectangle. Furthermore, by performing projective transformation on the inverted image 53, the generation unit 43 converts the shape of the detection target portion A2 in the inverted image 53 from a trapezoid to a rectangle. As a result, the generation unit 43 can generate an image of the first rail R1 and the second rail R2 as viewed from directly above along the vertical direction.

[0037] In step S16, the generation unit 43 generates an image 54 by converting the area of the projection-transformed first image 51 other than the detection target portion A1 into a single color. The generation unit 43 also generates an image 55 by converting the area of the projection-transformed inverted image 53 other than the detection target portion A2 into a single color. In this example, the generation unit 43 converts the area into black. The generation unit 43 may also convert the area into another single color, such as white.

[0038] In step S17, the generation unit 43 generates a plurality of preprocessed images 56 by vertically dividing the image 54. The generation unit 43 also generates a plurality of preprocessed images 57 by vertically dividing the image 55. For example, the generation unit 43 vertically divides each of the images 54 and 55 equally. The number of preprocessed images 56 generated from the image 54 may be any number equal to or greater than two. The number of preprocessed images 57 generated from the image 55 may be any number equal to or greater than two. The generation unit 43 may vertically divide each of the images 54 and 55 so that the size of each of the preprocessed images 56 and 57 is the same as the size of an image used in training the second trained model described below. In FIG. 7, the boundary between adjacent preprocessed images 56 is indicated by a dotted line L1, and the boundary between adjacent preprocessed images 57 is indicated by a dotted line L2.

[0039] In step S18 (second detection step), the second detection unit 44 inputs the preprocessed images 56 and 57 into the second trained model to detect the deformation D1 in the detection target portion A1 and the deformation D2 in the detection target portion A2. The second trained model is used to detect the deformations D1 and D2.

[0040] In step S19, the second detection unit 44 outputs the detection result. The second detection unit 44 may display the processing result on the display device 6, may store the processing result in a predetermined storage device such as a memory or a database, or may transmit the processing result to another computer system.

[0041] [Learning Method] Next, an example of a training method for the first trained model used in the detection method according to the above embodiment will be described. Fig. 8 is a flowchart showing an example of the training method in the detection system 1 as flow S2. Flow S2 corresponds to the training phase.

[0042] In step S21, the generation unit 43 acquires an image (teacher image) showing a pair of rails from the database 5. The teacher image may be an image generated in steps S11 to S17 described above. In step S22, the generation unit 43 adds annotation information indicating the rails to the teacher image, and stores the teacher image and the annotation information as teacher data in a predetermined storage device. This annotation information may be information stored in the database 5, or may be information input by the user. The generation unit 43 may increase the teacher data by adjusting the brightness of the teacher image.

[0043] In step S23, the learning unit 41 trains the learning model using the teacher data. The learning unit 41 adjusts the parameters of the learning model until a predetermined termination condition is met. In step S24, the learning unit 41 trains the learning model at the time when the training is completed as a first trained model for performing rail segmentation. The learning unit 41 stores the first trained model in a predetermined storage device. This first trained model is used in step S12.

[0044] Next, an example of a training method for the second trained model used in the above-described detection method will be described. Fig. 9 is a flowchart showing an example of the training method in the detection system 1 as flow S3. Flow S3 corresponds to the training phase.

[0045] In step S31, the generation unit 43 obtains an image (teacher image) showing a pair of rails and a deformation from the database 5. The teacher image may be an image generated in steps S11 to S17 described above. In step S32, the generation unit 43 adds annotation information about the deformation to the teacher image. The annotation information is information including, for example, the type of deformation and the position (coordinates) of the deformation. The generation unit 43 stores the teacher image and the annotation information as training data in a predetermined storage device. This annotation information may be information stored in the database 5 or information input by the user.

[0046] In step S33, the learning unit 41 trains the learning model using the teacher data. The learning unit 41 adjusts the parameters of the learning model until a predetermined termination condition is met. In step S34, the learning unit 41 trains the learning model at the time when the training is completed as a second trained model for detecting anomalies. The learning unit 41 stores the second trained model in a predetermined storage device. This second trained model is used in step S18.

[0047] [Action and effect] In the detection system 1 and detection method described above, the pair of rails R can be efficiently imaged by the imaging unit 21 mounted on a railway vehicle traveling on the pair of rails R. Then, the generation unit 43 generates preprocessed images 56, 57 based on the detection results of the first detection unit 42, and the second detection unit 44 inputs the preprocessed images 56, 57 into the second trained model to detect deformations D1, D2 in the detection target portions A1, A2 of the pair of rails R. This makes it possible to detect the deformations D1, D2 in the detection target portions A1, A2 with high accuracy from the preprocessed images 56, 57 based on the detection results of the first detection unit 42. As a result, the deformations D1, D2 in the rails R can be detected with high accuracy.

[0048] In the detection system 1 and detection method, the generator 43 generates a first image 51 depicting a detection target portion A1 on the first rail R1, a second image 52 depicting a detection target portion A2 on the second rail R2, and an inverted image 53 in which the left and right sides of the second image 52 are inverted. The generator 43 then generates preprocessed images 56 and 57 based on the first image 51 and the inverted image 53. For example, deformations D1 and D2 in a pair of rails R are likely to occur on the inside of the pair of rails R. Therefore, the deformations D1 and D2 are located on the same side in the inverted image 53 and the first image 51 generated by the generator 43, and the detection conditions for the deformations D1 and D2 can be made uniform. By inputting the preprocessed images 56 and 57 generated based on the inverted image 53 and the first image 51, for which the detection conditions have been made uniform, into the second trained model, the deformations D1 and D2 can be detected with higher accuracy.

[0049] In the detection system 1 and detection method, the generation unit 43 generates a first image 51 and a second image 52 from the lower region when the captured image 50 is divided into two in the vertical direction. This makes it possible to generate the first image 51 and the second image 52 from an image portion that captures an area relatively close to the imaging unit 21 mounted on the railway vehicle. Because the first image 51 and the second image 52 show the deformations D1 and D2 relatively clearly, the use of these images makes it possible to detect the deformations D1 and D2 with higher accuracy.

[0050] In the detection system 1 and detection method, the generation unit 43 generates preprocessed images 56 and 57 that depict the rectangular detection target portions A1 and A2 by performing projective transformation on the first image 51 and the inverted image 53 that depict the detection target portions A1 and A2 of the pair of rails R. This makes it possible to make the shapes of the detection target portions A1 and A2 depicted in the preprocessed images 56 and 57 uniform, and as a result, it is possible to stabilize the detection accuracy of the deformations D1 and D2.

[0051] In the detection system 1 and detection method, the generation unit 43 generates preprocessed images 56 and 57 by converting areas other than the detection target portions A1 and A2 into a single color (black in this example) in the first image 51 and the inverted image 53, which show the detection target portions A1 and A2 of the pair of rails R. This narrows the range of inference by the second trained model to the detection target portions A1 and A2, enabling the detection of deformations D1 and D2 with higher accuracy.

[0052] In the detection system 1 and detection method, the generation unit 43 generates multiple preprocessed images 56, 57 by vertically dividing images 54, 55 showing detection target portions A1, A2 of a pair of rails R, and the second detection unit 44 inputs the multiple preprocessed images 56, 57 into a second trained model to detect deformations in the detection target portions A1, A2. This narrows the range of inference by the second trained model, enabling deformations D1, D2 to be detected with higher accuracy.

[0053] Although the embodiments of the present invention have been described above, the embodiments of the present invention are not limited to the above-described embodiments. In the above-described embodiments, in step S13, the generation unit 43 generates an image that captures a detection target portion, which is a portion of the pair of rails R, from the captured image 50. However, the generation unit 43 may generate an image that captures the entire pair of rails R based on the detection result of the first detection unit 42. For example, the generation unit 43 may generate a first image 51 that captures the entire first rail R1 and a second image 52 that captures the entire second rail R2.

[0054] In the above-described detection method, at least one of steps S14 to S17 may be omitted. The first image 51 and the second image 52 generated in step S13 may be input to a second trained model in step S18 to detect the deformations D1 and D2. Furthermore, steps S13 to S17 may be performed in a different order after steps S15 and S16 are performed in this order. For example, each of steps S15 to S17 may be performed before step S13. That is, in each of steps S15 to S17, predetermined processing may be performed on the first image 51 and the second image 52 (inverted image 53) as in the above embodiment, or the predetermined processing may be performed on the captured image 50. [Explanation of symbols]

[0055] 1...detection system, 21...imaging unit, 42...first detection unit, 43...generation unit, 44...second detection unit, 50...captured image, 51...first image, 52...second image, 53...inverted image, 54, 55...image, 56, 57...preprocessed image, A1, A2...detection target part, D1, D2...deformation, R...rail, R1...first rail, R2...second rail.

Claims

1. an imaging unit mounted on a railway vehicle running on a pair of rails, for outputting a captured image of the pair of rails; a first detection unit that inputs the captured image into a first trained model and detects the pair of rails; a generation unit that generates a preprocessed image from the captured image based on a detection result of the first detection unit, the preprocessed image including a detection target portion that is at least a part of the pair of rails; A second detection unit that inputs the preprocessed image into a second trained model and detects a deformation in the detection target portion, the generating unit generates a first image that shows the detection target portion on a first rail that is one of the pair of rails, a second image that shows the detection target portion on a second rail that is the other of the pair of rails, and an inverted image in which one of the first image and the second image is inverted left and right; The second detection unit inputs the inverted image and the other of the first image and the second image into the second trained model as the preprocessed image, and detects the deformation in the detection target portion of each of the first rail and the second rail. Detection system.

2. the generation unit generates the first image and the second image from a lower region when the captured image is divided into two in a vertical direction. The detection system of claim 1 .

3. the generation unit generates the preprocessed image depicting the rectangular detection target portion by performing a projective transformation on the image depicting the detection target portion of the pair of rails.

3. A detection system according to claim 1 or 2.

4. the generation unit generates the preprocessed image by converting an area other than the detection target portion in an image showing the detection target portion of the pair of rails into a single color.

3. A detection system according to claim 1 or 2.

5. the generation unit generates a plurality of the preprocessed images by dividing an image showing the detection target portions of the pair of rails in a vertical direction; The second detection unit inputs the plurality of preprocessed images into the second trained model to detect the deformation in the detection target portion.

3. A detection system according to claim 1 or 2.

6. A detection method performed by a detection system, comprising: an imaging step of outputting an image of the pair of rails captured from a railway vehicle traveling on the pair of rails; a first detection step of inputting the captured image into a first trained model to detect the pair of rails; a generation step of generating a preprocessed image from the captured image based on the detection result of the first detection step, the preprocessed image including a detection target portion, which is at least a part of the pair of rails; A second detection step of inputting the preprocessed image into a second trained model to detect a defect in the detection target portion, In the generating step, a first image showing the detection target portion on a first rail that is one of the pair of rails, a second image showing the detection target portion on a second rail that is the other of the pair of rails, and an inverted image in which one of the first image and the second image is inverted left and right are generated, In the second detection step, the inverted image and the other of the first image and the second image are input to the second trained model as the preprocessed image to detect the deformation in the detection target portion of each of the first rail and the second rail. Detection method.

Images