Method for detecting abnormalities in trains using artificial intelligence and method for using the same.

Abstract

Maintaining the structural integrity of trains and railway vehicles is of paramount importance, as component defects can lead to derailments. This invention relates to a system equipped with a camera and lighting means for acquiring high-resolution images as a train passes through a portal while in motion, and detects defects by analyzing these images using artificial intelligence. The detected information is stored on a software platform and its accuracy is improved as needed through human-in-the-loop verification. Furthermore, the defect information is transmitted in real time to a remote location for external analysis.

Description

Technical Field

[0001] The present invention relates to an inspection technology for components of trains and railway vehicles, and particularly to a technology for acquiring high-resolution images from a running train using artificial intelligence and detecting abnormalities by analyzing the images.

Background Art

[0002] A train is composed of a plurality of types of railway vehicles connected together, and it is extremely important to maintain the structural integrity of each vehicle. There are various components in railway vehicles, some of which are visible from the outside, but also include components that are difficult to visually recognize. The arrangement of these components is essential for the proper operation of the train. In the prior art, the components of each vehicle were inspected with the train stopped, which increased the so-called "dwell time" and was a factor in reducing productivity. To ensure safety and structural integrity while shortening the dwell time, the development of a technology that can acquire high-resolution images while the train is running is required.

[0003] The components to be inspected include air hoses, axles, bearings, doors, brake components, the state (open / closed) of hopper hatches, knuckle pins, truck springs, etc., covering a wide range. These components need to be inspected regularly, and any failure may jeopardize the safety of the entire train.

[0004] The present invention acquires high-resolution images while the train is running and uses artificial intelligence incorporated in the system to detect whether there are abnormalities in the railway vehicles that make up the train. In recent years, artificial intelligence has become commonly used in various applications.

[0005] In the present invention, by using artificial intelligence, more specifically a supervised learning model, it is possible to complement and improve the working accuracy of mechanical inspectors for inspecting railway vehicles. Also, in order to further improve the accuracy of the detection results and improve the quality of the provided detection information, human intervention can also be carried out.

[0006] While patent applications and technology disclosures related to artificial intelligence (AI) have been filed in various fields for some time, the railway industry faces unique challenges due to its harsh and restrictive image acquisition environment. The following is an example of relevant prior art literature. [Prior art documents] [Patent Documents]

[0007] [Patent Document 1] U.S. Patent No. 8,446,467 (Tilton) relates to a handheld speed detection and video timing device. The technology has the function of measuring the speed of a train and acquiring video footage of the train while it is in motion. [Patent Document 2] U.S. Patent No. 10,296,794 (Ritti) relates to an artificial intelligence-powered monitoring system used for detecting violations on roads. While not directly related to railroads, it is mentioned as an example of an AI application. [Patent Document 3] U.S. Patent No. 10,649,988 (Gold) discloses machine learning technologies in the marine sector, primarily relating to data storage devices and methods. [Patent Document 4] U.S. Patent No. 11,037,443 (Cui) relates to a cooperative warning system that shares warning information among multiple vehicles. [Patent Document 5] U.S. Patent No. 11,052,821 (Petersen) relates to an artificial intelligence expert system for automobiles that detects dangerous driving behavior.

[0008] While these documents all relate to technologies utilizing artificial intelligence, they differ from the technical challenges of the present invention, which involve acquiring high-resolution images from a moving train and using artificial intelligence to detect abnormalities in the railway vehicle based on those images. Furthermore, none of the above documents include components such as portal structures, imaging devices, lighting devices, ID tags, and speed detection devices to accommodate the diverse components and connections unique to railway vehicles, which clearly distinguishes them from the present invention.

[0009] Therefore, these prior art documents, either individually or in combination, do not suggest the structure or technical concept of the present invention. [Overview of the project]

[0010] A train is composed of multiple railway cars coupled together and travels over long distances at high speeds. Each train is made up of a combination of different types of railway cars, and its configuration varies from train to train. The connection points between the cars that make up a train contain components such as chains, hoses, and pins, all of which are essential for the safe operation of the train. If any of these parts fail, it could lead to a serious accident. This invention relates to a technology that automatically detects abnormalities in components by acquiring high-resolution images even while the train is in motion and analyzing those images using artificial intelligence (AI).

[0011] Each railway car is fitted with a unique identification tag. By reading this tag, information specific to that car, such as the location of parts to be inspected, can be obtained. This information is stored within the system and processed in conjunction with the detection results.

[0012] There are various types of railway vehicles. For example, there are boxcars that transport general cargo and tank cars that transport liquids. The system according to the present invention can switch the parts to be inspected according to the type of vehicle, and acquires high-resolution images of the components and connections of each railway vehicle.

[0013] The system according to the present invention performs inspections on specific components of each railway vehicle that makes up a train, in accordance with requests from end users. When the train passes through a portal constructed at a predetermined location, high-resolution images of the vehicle and its components are acquired.

[0014] As a train passes through a portal (gate-type structure) located at a predetermined position, cameras installed on the portal acquire images of each railway vehicle and its components. At this time, a speed detection device that detects the approaching train activates, and the camera's imaging speed (acquisition rate) is controlled according to the train's speed. Line scan cameras or area scan cameras can be used as the cameras. The cameras and lighting equipment are placed on the portal and are automatically activated when the train passes through.

[0015] The train travels along a fixed track, passing through multiple portals. Each portal is a large structure capable of accommodating cameras and lighting equipment while also being designed to allow train passage. The cameras and lighting equipment are installed within the portal structure, enabling the acquisition of high-resolution images of each railway vehicle at predetermined intervals.

[0016] Images of each railway vehicle acquired through this portal (including side, top, bottom, and connection points between vehicles) are analyzed by an AI supervised machine learning model to detect abnormal areas. These machine learning models are pre-trained using labeled images of various railway vehicles, and they detect defects in components and connection points by comparing the acquired images with known abnormal images.

[0017] In order to detect defects in images of railway vehicles, it is first necessary to label a large number of images corresponding to various railway vehicles, and use the labeled images to train a machine learning model of artificial intelligence. Based on such labeled images, an algorithm for defect detection is constructed and trained, and the obtained individual learning model compares the acquired image with known defect images of the same type of vehicle to determine the presence or absence of defects. Furthermore, multiple levels of business logic are implemented in the inference script of this system, and by combining various parameters of the logic with the results of the machine learning model, the area of concern is specified with high accuracy.

[0018] In order to improve the accuracy of real-time anomaly notifications and continuously improve the accuracy of the machine learning model, the present invention also has a "Human in the Loop (HITL)" function. This includes a verification process by a human operator, confirms the validity of the detection results by AI, and at the same time enables model re-learning using the verified images.

[0019] In addition, in the present invention, it is possible to develop and implement a new algorithm according to the parts to be inspected, and it has a modular design that can be flexibly configured according to the application. This facilitates customization according to user needs.

Brief Description of the Drawings

[0020] [Figure 1] Figure 1 is a schematic diagram of the components of this device.

[0021] [Reference Numerals] 15 Railway vehicle model 20 Artificial intelligence 25 Speed detection device 30 Storage device 35 Human in the Loop (HITL) 40 Lighting 45 Camera 50 Identification tag 55 Area scan camera 60-line scan camera

[0022] [Detailed description] A train is composed of an engine or locomotive and a plurality of railway vehicles. In normal operation, an operator (driver) operates the locomotive located at the frontmost part. Due to such a forward arrangement, it is extremely difficult or substantially impossible for the operator to visually recognize the states of the other railway vehicles constituting the train. In particular, the components at the lower part of the train cannot be visually recognized. A train usually travels relatively fast on a defined track from a starting point to a destination point and in a remote area.

[0023] There are various types of railway vehicles that make up a train. Each railway vehicle is attached with an identification tag (identification information tag) 50, and the tag stores characteristic information unique to the vehicle. Examples of such characteristic information include the weight when the vehicle is empty, the configuration of the connection part, the position of the hose or hopper door, the position of the valve or latch, etc. These identification information tags are scanned and stored in the system. Furthermore, model data of various railway vehicles are also stored in this system and are used when comparing with the acquired real vehicle images.

[0024] When a train travels on a track, its speed is detected by a speed detection device. The speed detection device determines the shutter speed of the camera when the train passes through a portal. At a predetermined position where the train enters the portal, the speed detection device is activated, and the speed of the wheels is measured by a plurality of sensors (not shown) provided on the track. The speed detection device 25 calculates the speed of the entire train and determines the shutter speed of the camera according to the speed. This speed detection device is interlocked with the camera and the lighting device and is controlled so that an image is accurately acquired when passing through the portal. [[ID=十六]] [[ID=十七]]

[0025] [[ID=十八]] As the train moves along the track, it passes through a large structure called a portal. This portal is equipped with a camera 45 and lighting equipment 40, which may also be installed on the ground below the train if necessary. The portal is a structure that spans both sides of the track and is installed in a shape that allows the train to pass underneath. An example is shown in Figure 3.

[0026] As the train passes through the portal, multiple cameras 45 acquire high-resolution images of the train's surface. Depending on user requirements, line-scan cameras 60 or area-scan cameras 55 can be selected. A lighting system 40 is also provided to ensure high-resolution image acquisition under all illumination conditions. Depending on the operating environment or user requirements, LED lighting or stadium lighting can be used for the lighting system.

[0027] The acquired images include the underside, top, and sides of the train, as well as the connections between each railway car. These images are stored in the storage device 30 of the present system.

[0028] This system uses supervised machine learning with artificial intelligence to detect areas of concern in railway vehicles. An artificial intelligence model 15, trained on images of railway vehicles, is stored within the system, enabling the detection of defects in both freight and passenger train vehicles.

[0029] As the train travels along the track, the speed detection device 25 activates and calculates the train's speed. Based on this calculated speed, the camera's image acquisition rate is controlled. When the train passes through the portal, a high-resolution image is acquired by the camera and stored in the system.

[0030] In this system, individual algorithms have been built according to various defect use cases, and these algorithms are used to detect defects or areas of concern in acquired images. The algorithms are applied to each component of the railway vehicle and are designed to detect specific areas of concern corresponding to the relevant part. After the acquired images are classified and organized by the system, they are transferred to the "HITL (Human in the Loop)" system 35. The HITL system is activated as needed and used to improve the accuracy of the information provided to railway vehicle operators. Furthermore, defect images validated by HITL can be added to the existing training dataset in the retraining process for specific detection models, thereby continuously improving the accuracy of the machine learning models.

[0031] This stored information can be transmitted to a remote location in real time for analysis and processing. While the embodiments of the present invention are as described above, it goes without saying that various modifications and alterations can be made by those skilled in the art without departing from the spirit of the present invention.

Claims

1. A method for detecting abnormalities in a moving train using artificial intelligence, comprising the following configuration: - A portal consisting of a structure with sides and an upper section, the portal enclosing a train track and configured to allow trains to pass through the portal. - A group of cameras comprising multiple cameras installed on the portal and multiple cameras installed at the lower position of the train, wherein the multiple cameras are for acquiring high-resolution images of the train. - Lighting means installed at the lower position of the portal and the train, which enables the acquisition of high-resolution images under full illumination conditions. - A speed detection device connected to the aforementioned multiple cameras, which detects the speed of the train and controls the shutter speed of the cameras. - An identification tag attached to each railway vehicle, containing information about the railway vehicle. - A storage device for storing the high-resolution image and information relating to the railway vehicle, - Multiple railway vehicle models, wherein the configuration models of the railway vehicles are uploaded to the storage device. - A group of algorithms developed to identify the area to be detected and incorporated into the memory device, - A Human-in-the-Loop (HITL) mechanism that enables the verification of images for augmenting existing training datasets, thereby improving the accuracy of machine learning models. including, An anomaly detection method characterized by transmitting the detection results to a remote location in real time.

2. An anomaly detection method according to claim 1, characterized in that the plurality of cameras are line scan cameras.

3. An anomaly detection method according to claim 1, characterized in that the plurality of cameras are area scan cameras.

4. An anomaly detection method according to claim 1, characterized in that the lighting means is an LED light.

5. An anomaly detection method according to claim 1, characterized in that the lighting means is stadium lighting.

6. An anomaly detection method according to claim 1, characterized in that the plurality of railway vehicle models represent different types of railway vehicles.

Images