Locomotive coupling determination system for identifying whether autonomous locomotive is coupled through ai classification model, and method for training ai classification model

Abstract

A locomotive coupling determination system according to one embodiment of the present invention may comprise: a camera having a photographing area facing coupling pins for coupling or decoupling a plurality of locomotives; and a processor which acquires images from the camera, crops a specific area on the acquired images, and inputs the cropped images into a pre-trained artificial intelligence classification model so as to identify whether a locomotive is coupled by the coupling pins, wherein the pre-trained artificial intelligence classification model may have been trained on the basis of a plurality of cropped images obtained by cropping the specific area of a plurality of images acquired by the camera.

Description

Train coupling determination system and AI classification model training method for identifying whether autonomous trains are coupled using an AI classification model

[0001] The present invention relates to a method for learning an artificial intelligence classification model for determining whether an autonomous train used in a logistics system, etc. is coupled, and to a train coupling determination system that identifies whether an autonomous train is coupled using the learned artificial intelligence classification model.

[0002] Various types of couplers are used as devices for connecting trains in logistics systems and the like. Couplers can also be referred to as train coupling systems, and the status of the train coupling judgment system can be easily determined primarily by human vision based on whether the coupling pin has dropped. However, with the recent increase in autonomous vehicles and trains, it has become difficult to determine whether the coupling pin has dropped within the train coupling judgment system due to the absence of human personnel.

[0003] As a method to solve this problem, an artificial intelligence detection model can be used. An AI detection model is trained to determine the location and whether the fastening pin has fallen by using multiple images having a shooting area that includes the fastening pin. Images containing the fastening pin are then input into the trained AI detection model to estimate the location and whether the fastening pin has fallen, and simultaneously determine whether the train is coupled by the fastening pin.

[0004] (Patent Document 0001) Korean Published Patent Application No. 10-2021-0058875 (Published May 24, 2021)

[0005] (Patent Document 0002) Republic of Korea Published Patent Application No. 10-2024-0001241 (Published Jan. 3, 2024)

[0006] (Patent Document 0003) Korean Patent Publication No. 10-2024-0021146 (Published Feb. 16, 2024)

[0007] Conventionally, artificial intelligence (AI) detection models have been used to determine whether a fastening pin has fallen. These AI detection models can estimate the position of the fastening pin while simultaneously determining whether the train is coupled by the pin. However, because AI detection models perform these two functions simultaneously within a single model, they have disadvantages in terms of speed and accuracy.

[0008] To solve this problem, the present invention provides a train coupling determination system capable of more accurately identifying whether an autonomous train is coupled by a coupling pin through the AI ​​classification model, by using a plurality of images having a shooting area including a coupling pin to lighten the training data compared to a detection model.

[0009] The technical problems to be solved in this document are not limited to those mentioned above, and other unmentioned technical problems will be clearly understood by those skilled in the art to which this invention belongs from the description below.

[0010] A train coupling determination system according to one embodiment of the present invention may include: a camera having a shooting area facing a coupling pin that couples or uncouples a plurality of trains; and a processor that acquires an image from the camera, crops a specific area of ​​the acquired image, and inputs the cropped image into a pre-trained artificial intelligence classification model to identify whether the trains are coupled by the coupling pin, wherein the pre-trained artificial intelligence classification model may be trained based on a plurality of cropped images obtained by cropping the specific area of ​​a plurality of images acquired by the camera.

[0011] A method for training an artificial intelligence classification model used by a train connection determination system including a camera according to an embodiment of the present invention may include: a step of acquiring a plurality of images having a shooting area facing the connection pin by the camera; a step of cropping a specific area of ​​the acquired plurality of images; and a step of training the artificial intelligence classification model based on the plurality of cropped images.

[0012] Unlike conventional AI detection models that acquire images by individually labeling them and train based on such images, according to one embodiment of the present invention, only images of the shooting area containing the fastening pin are acquired, and an AI classification model is trained based on training data that is lighter than that of an AI detection model, thereby enabling the determination of whether the train is coupled. In other words, since the area for acquiring images is used in a fixed manner, a large number of images can be processed more quickly, thereby generating an AI classification model with high accuracy while reducing costs, and using this, it is possible to quickly and accurately identify whether the train is coupled.

[0013] FIG. 1 illustrates a block diagram of a train connection determination system according to one embodiment.

[0014] FIG. 2 illustrates the appearance of a plurality of trains before they are combined according to one embodiment.

[0015] FIG. 3 primarily illustrates a train coupling determination system in the case where a plurality of trains are not coupled according to one embodiment.

[0016] FIG. 4 primarily illustrates a train coupling determination system in the case where a plurality of trains are combined according to one embodiment.

[0017] FIG. 5 illustrates a fastening pin image, a surrounding image, and a cropping image according to one embodiment.

[0018] FIGS. 6 and 7 are flowcharts of a method for training an artificial intelligence classification model according to one embodiment.

[0019] FIG. 8 is a flowchart for identifying whether a train is coupled according to one embodiment.

[0020] FIGS. 9 and 10 illustrate identifying whether a train is coupled using a line laser according to one embodiment.

[0021] FIG. 11 illustrates a train connection determination system in which a background plate having a color contrasting with the connection pin is placed behind the connection pin according to one embodiment.

[0022] FIG. 12 illustrates a fastening pin having a color that contrasts with a structure present within the shooting area according to one embodiment.

[0023] FIG. 13 illustrates that, according to one embodiment, a structure present within the shooting area is painted in a color contrasting with the fastening pin.

[0024] The embodiments described in this document and the configurations illustrated in the drawings are merely preferred examples of the disclosed invention, and various modifications that may replace the embodiments and drawings of this specification may exist at the time of filing this application.

[0025] The terms used in this document are for describing the embodiments and are not intended to limit or restrict the disclosed invention.

[0026] For example, in this specification, singular expressions may include plural expressions unless the context clearly indicates otherwise.

[0027] In this document, each of the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "at least one of A, B, or C" may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof.

[0028] The term "and / or" includes a combination of multiple related described components or any of the multiple related described components. For example, "A and / or B" may include only "A," only "B," or both "A and B."

[0029] Additionally, terms such as “include” or “have” are intended to express the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and do not exclude the additional existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0030] When it is said that a component is "connected," "combined," "supported," or "in contact" with another component, this includes not only cases where the components are directly connected, combined, supported, or in contact, but also cases where they are indirectly connected, combined, supported, or in contact through a third component.

[0031] When it is said that a component is located "on" another component, this includes not only cases where one component is in contact with the other, but also cases where another component exists between the two components.

[0032] Meanwhile, terms such as "front," "rear," "left," "right," "top," and "bottom" used in the following description are defined based on the drawings; however, the shape and position of each component are not limited by these terms. For example, the front side may be defined as the +X side and the rear side as the -X side. For example, based on the drawings, the right side may be defined as the +Y side and the left side as the -Y side. For example, based on the drawings, the top side may be defined as the +Z side and the bottom side as the -Z side.

[0033] In addition, terms including ordinal numbers, such as "first," "second," etc., are used to distinguish one component from another and do not limit the components.

[0034] In addition, terms such as "~part," "~unit," "~block," "~part," and "~module" may refer to a unit that processes at least one function or operation. For example, the terms may refer to at least one piece of hardware such as an FPGA (field-programmable gate array) or an ASIC (application specific integrated circuit), at least one piece of software stored in memory, or at least one process processed by a processor.

[0035] The operating principle and one embodiment of the disclosed invention are described in detail below with reference to the attached drawings. Identical reference numbers or symbols in the attached drawings may indicate parts or components that perform substantially the same function.

[0036] FIG. 1 illustrates a block diagram of a train connection determination system according to one embodiment.

[0037] FIG. 2 illustrates the appearance of a plurality of trains before they are combined according to one embodiment.

[0038] FIG. 3 primarily illustrates a train coupling determination system in the case where a plurality of trains are not coupled according to one embodiment.

[0039] FIG. 4 primarily illustrates a train coupling determination system in the case where a plurality of trains are combined according to one embodiment.

[0040] FIG. 5 illustrates a fastening pin image, a surrounding image, and a cropping image according to one embodiment.

[0041] Referring to FIGS. 1 to 4, a train connection determination system (1) according to one embodiment may be composed of a control unit (200) including a connection pin (110), a camera (120), a line laser (130), a processor (210), and a memory (220), and a display (300).

[0042] The train coupling judgment system (1) exists between multiple trains and can be used when multiple trains approach each other and are coupled.

[0043] A connecting pin (110) located within the train connecting judgment system (1) serves to connect multiple trains. When multiple trains are separated from each other, the connecting pin (110) within the train connecting judgment system (1) does not fall and remains above (Fig. 3). On the other hand, when multiple trains approach each other for connecting, the connecting pin (110) within the train connecting judgment system (1) falls downward to connect the multiple trains (Fig. 4).

[0044] A camera (120) existing within the train coupling determination system may be positioned toward a coupling pin (110) existing within the train coupling determination system (1) and may capture an image having a shooting area including the coupling pin (110). The image captured by the camera (120) is transmitted to a processor (210) of a control unit (200) within the train coupling determination system (1) and may be stored in a memory (220).

[0045] A line laser (130) present in the train coupling judgment system (1) can irradiate a laser toward the coupling pin (110) to determine whether the coupling pin (110) has fallen. If the coupling pin (110) has not fallen, that is, if multiple trains are not coupled, a laser line from the line laser (130) may appear on the surface of the coupling pin (110). On the other hand, if the coupling pin (110) has fallen, that is, if multiple trains are coupled, the laser line from the line laser (130) may be dispersed into the air and not be visible on the surface of the coupling pin (110).

[0046] The control unit (200) present in the train connection judgment system may include a processor (210) and a memory (220).

[0047] The processor (210) controls the overall operation of the train coupling determination system (1). Specifically, at least one processor is connected to each component of the train coupling determination system (1) to control the overall operation of the train coupling determination system (1). For example, the processor (210) may be electrically connected to the memory (220) to control the overall operation of the train coupling determination system (1). The processor (210) may be composed of one or more processors.

[0048] The processor (210) can store an algorithm for processing user input entered through an input device.

[0049] The processor (210) may include one or more of a CPU (Central Processing Unit), GPU (Graphics Processing Unit), APU (Accelerated Processing Unit), MIC (Many Integrated Core), DSP (Digital Signal Processor), NPU (Neural Processing Unit), hardware accelerator, or machine learning accelerator. At least one processor may control one or any combination of other components of the train connection determination system (1) and may perform operations or data processing related to communication. The processor may execute at least one program or instruction stored in memory (220). For example, the processor (210) may perform a method according to at least one embodiment of the present invention by executing at least one instruction stored in memory (220).

[0050] The processor (210) can acquire an image having a shooting area including a fastening pin (110) from the camera (120). The processor (210) can crop a specific area of ​​the acquired image.

[0051] The processor (210) can acquire multiple images from the camera (120) having a shooting area including the fastening pin (110). The part that needs to be observed intensively to determine whether the train is coupled is the area of ​​the fastening pin (110). Due to the fluidity of the fastening pin (110) during train operation, the position of the fastening pin (110) can vary up, down, left, and right from its original position. That is, if only the area where the initial fastening pin (110) existed is acquired as image data (Fig. 5A), the fastening pin (110) may not exist in the data, and the accuracy of the classification of the artificial intelligence classification model learned based on this may decrease. Therefore, the processor (210) can acquire multiple images having a shooting area that includes an area increased by about 50% to the left and right and about 20% to the top and bottom from the area where the initial fastening pin (110) exists (Fig. 5B). However, the multiple surrounding images are not limited to this and may include various surrounding images around the fastening pin (110).

[0052] The processor (210) can crop a specific area of ​​a plurality of acquired images by randomly setting it (C and D of FIG. 5). That is, a plurality of cropped images can be obtained by cropping a specific area. The specific area set randomly may be the size of the area including the initial fastening pin, but is not limited thereto and can be set in various ways. The processor (210) can train an artificial intelligence classification model for determining whether the train is coupled based on the plurality of cropped images. Training the artificial intelligence classification model through the plurality of cropped images can be performed via program code. When training the artificial intelligence classification model using the plurality of cropped images obtained by cropping a specific area of ​​the plurality of images, the fluidity of the fastening pin (110) is taken into account, so even if a part of the fastening pin (110) is cut off and does not appear in the image, the accuracy of identifying whether the train is coupled can be increased.

[0053] The artificial intelligence classification model trained by the processor (210) may be a Se-mobilenet model with SE blocks combined with Mobilenet, an Efficientnet model, a Seresnet model, a Ghostnet model, a Squeezenet model, a Shufflenet model, etc., but is not limited to a specific model.

[0054] Models incorporating AI classification principles are lightweight compared to conventional training models by reducing the number of data parameters and computational load. This enables rapid processing of large volumes of images and offers cost-saving benefits. In other words, since they are trained by emphasizing only important data, they allow for faster and more accurate classification compared to conventional detection models like Yolox, while also providing efficient performance. These models can be widely used for real-time image classification, object recognition, and more.

[0055] The processor (210) can obtain multiple training images by cropping a specific area of ​​a plurality of images having a shooting area including a connecting pin (110) from a camera (120) and then preprocessing the plurality of cropped images. The processor (210) can train an artificial intelligence classification model for determining whether a train is coupled based on the preprocessed plurality of training images. Training of the artificial intelligence classification model using the plurality of training images preprocessed from the plurality of cropped images can be performed in program code. When training the artificial intelligence classification model using the plurality of training images preprocessed from the plurality of cropped images, the accuracy of identifying whether a train is coupled can be further increased.

[0056] As a preprocessing step, the processor (210) can obtain multiple training images by cropping a specific area of ​​a plurality of images having a shooting area including a connecting pin (110) from a camera (120) and then adjusting the contrast ratio of the plurality of cropped images. The processor (210) can train an artificial intelligence classification model for determining whether a train is coupled based on the plurality of training images with adjusted contrast ratios.

[0057] The processor (210) can adjust the brightness value to maximize the difference between the dark and bright parts of the image.

[0058] For example, the processor (210) can automatically perform contrast processing on multiple cropping images. Brightness generally ranges from 0 (completely dark) to 255 (completely bright). To adjust the brightness ratio of multiple cropping images, the processor (210) can find the darkest part of the image and change the brightness value to a value close to 0, and find the brightest part and change the brightness value to a value close to 255. That is, the colors of the intermediate brightness areas are relatively well preserved, and the color range of the image is expanded overall, so that multiple training images with more vivid and distinctly emphasized details can be generated. By performing automatic contrast processing, it is prevented that an artificial intelligence classification model with lower accuracy is trained and used based on cropping images in which the saturation, brightness, and contrast of the images differ depending on the weather, etc.

[0059] However, the contrast adjustment of the cropped image is not limited to automatic contrast processing and can be adjusted in various ways within the range of brightness values ​​(0-255) to be advantageous for generating training images for training an artificial intelligence classification model.

[0060] As a preprocessing step, the processor (210) can obtain multiple training images by cropping a specific area of ​​a plurality of images having a shooting area including a connecting pin (110) from a camera (120) and then blurring the plurality of cropped images. The processor (210) can train an artificial intelligence classification model for determining whether a train is connected based on the multiple blurred training images.

[0061] As a preprocessing step, the processor (210) can obtain multiple training images by cropping a specific area of ​​a plurality of images having a shooting area including a connecting pin (110) from a camera (120) and then flipping the plurality of cropped images left and right. The processor (210) can train an artificial intelligence classification model for determining whether a train is connected based on the multiple training images flipped left and right.

[0062] After cropping a specific area of ​​a plurality of images having a shooting area including a fastening pin (110) from a camera (120), the processor (210) can train an artificial intelligence classification model for determining whether a train is coupled based on the presence or absence of a laser line by a line laser (130) directed toward the fastening pin within the plurality of cropped images.

[0063] The processor (210) can identify whether the train is coupled by inputting an image having a shooting area including a coupling pin (110) or an image cropped from a specific area of ​​the image into a learned artificial intelligence classification model.

[0064] The memory (220) can store data necessary for various embodiments. Depending on the purpose of data storage, the memory may be implemented in the form of a memory embedded in the train coupling judgment system (1) or in the form of a memory that can be attached to and detached from the train coupling judgment system (1). For example, data for operating the train coupling judgment system (1) may be stored in a memory embedded in the train coupling judgment system (1), and data for the expansion function of the train coupling judgment system (1) may be stored in a memory that can be attached to and detached from the train coupling judgment system (1). Meanwhile, the memory embedded in the train coupling determination system (1) may be implemented as at least one of volatile memory (e.g., DRAM (dynamic RAM), SRAM (static RAM), or SDRAM (synchronous dynamic RAM), etc.), non-volatile memory (e.g., OTPROM (one-time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash, etc.), hard drive, or solid state drive (SSD). Additionally, the memory that can be attached to or detached from the train coupling determination system (1) may be implemented in the form of a memory card (e.g., CF (compact flash), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.), or external memory connectable to a USB port (e.g., USB memory). there is.

[0065] In the memory (220), an image having a shooting area including a fastening pin (110) captured through one or multiple operations, an image cropped from a specific area of ​​the image, and multiple training images obtained by pre-processing the cropped image, such as adjusting the contrast ratio, blurring, or flipping left and right, may be stored. Additionally, an artificial intelligence classification model learned from multiple images, multiple cropped images, and multiple training images obtained by the camera (120) may be stored.

[0066] The train connection judgment system (1) may include an output device. The output device may output sensory information (e.g., visual information and / or auditory information). For example, the output device may include a display (300) and / or a speaker.

[0067] The output device can output various information related to the train coupling judgment system (1) (e.g., image, cropped image, training image, artificial intelligence classification model, whether the coupling pin has fallen, whether the train has coupled, etc.).

[0068] The train coupling judgment system (1) may include an input device. The input device may receive user input from a user. Here, user input may include tactile input and / or auditory input. In the present invention, the user is a person using the train coupling judgment system (1) and may be referred to as an operator, a manipulator, etc. The input device may include at least one of various input devices such as a microphone, a touch pad, a button, a jog shuttle, etc. Alternatively, it is possible for the output device and the input device to implement a touch screen. The input device may include a communication interface (e.g., an antenna) for receiving user input remotely through an external device (e.g., a remote control).

[0069] The input device can receive input for controlling the control unit (200) within the train connection judgment system (1). Here, the input for controlling the control unit (200) within the train connection judgment system (1) may include inputs such as the size of a specific area to be cropped, the degree of contrast adjustment, the degree of blur processing, and whether to flip the image left and right.

[0070] FIGS. 6 and 7 are flowcharts of a method for training an artificial intelligence classification model according to one embodiment.

[0071] According to various embodiments, the entity building the artificial intelligence classification model may be a train connection judgment system (1) or an external computing device other than the train connection judgment system (1). The train connection judgment system (1) may transmit image data collected by the camera (120) to an external computing device, and the external computing device may build an artificial intelligence classification model based on the image data received from the train connection judgment system (1).

[0072] Referring to FIG. 6, a method for training an artificial intelligence classification model using multiple cropped images can be seen. In order to obtain data necessary for training an artificial intelligence classification model to determine whether a train is coupled, a camera (120) within a train coupling determination system (1) can acquire multiple surrounding images including a coupling pin (110) (1000). The area that needs to be observed intensively to determine whether a train is coupled is the area of ​​the coupling pin (110). Due to the fluidity of the coupling pin (110) during train operation, the position of the coupling pin (110) can vary up, down, left, and right from its original position. That is, if only the area where the initial coupling pin (110) existed is acquired as image data, the coupling pin (110) may not exist in the data, and the accuracy of the classification of the artificial intelligence classification model trained based on this may decrease. Therefore, a process of acquiring multiple surrounding images including the coupling pin (110) may be required. The multiple surrounding images may be images that include an area that is increased by about 50% to the left and right and about 20% to the right from the area where the initial coupling pin (110) exists. However, multiple surrounding images are not limited to this and may include various surrounding images around the fastening pin (110).

[0073] The train connection determination system (1) can obtain multiple cropped images (1100) by cropping specific areas of multiple images including the connection pin (110). When cropped images are obtained by randomly cropping specific areas of multiple surrounding images including the connection pin (110) and an artificial intelligence classification model is trained based on this, the connection status of the connection pin (110), that is, whether the train is connected, can be determined with high accuracy even if a part of the connection pin (110) on the image is cut off due to the fluidity of the connection pin (110). The randomly set specific area may be the size of the area including the initial connection pin (110), but is not limited to this and can be set in various ways.

[0074] The train coupling determination system (1) can train an artificial intelligence classification model used to determine whether a train is coupled by using multiple cropped images cropped to a specific area (1200). Training the artificial intelligence classification model through multiple cropped images can be performed via program code. When training the artificial intelligence classification model using multiple cropped images obtained by cropping specific areas of multiple images, the fluidity of the coupling pin (110) is taken into account, thereby increasing the accuracy of identifying whether a train is coupled.

[0075] Referring to FIG. 7, a method for training an artificial intelligence classification model using multiple training images that have been preprocessed from multiple cropped images can be seen. In order to obtain data necessary for training an artificial intelligence classification model to determine whether a train is coupled, a camera (120) within a train coupling determination system (1) can acquire multiple surrounding images including a coupling pin (110) (2000). The area that needs to be observed intensively to determine whether a train is coupled is the area of ​​the coupling pin (110). Due to the fluidity of the coupling pin (110) during train operation, the position of the coupling pin (110) can vary up, down, left, and right from its original position. That is, if only the area where the coupling pin (110) was initially located is acquired as image data, the coupling pin (110) may not exist in the data, and the accuracy of the classification of the artificial intelligence classification model trained based on this may decrease. Therefore, a process of acquiring multiple surrounding images including the coupling pin (110) may be required. A plurality of surrounding images may be images that include an area that is increased by about 50% to the left and right and about 20% to the right and from the area where the initial fastening pin (110) is located. However, the plurality of surrounding images are not limited to this and may include various surrounding images around the fastening pin (110).

[0076] The train connection determination system (1) can obtain multiple cropped images (2100) by cropping specific areas of multiple images including the connection pin (110). When cropped images are obtained by randomly cropping specific areas of multiple surrounding images including the connection pin (110) and an artificial intelligence classification model is trained based on this, even if a part of the connection pin (110) on the image is cut off due to the fluidity of the connection pin (110), the connection status of the connection pin (110), that is, the connection status of the train, can be determined with high accuracy. The randomly set specific area may be the size of the area including the initial connection pin (110), but is not limited to this and can be set in various ways.

[0077] The train coupling determination system (1) can obtain multiple training images by preprocessing multiple cropping images (2200). The process of preprocessing multiple cropping images may include, but is not limited to, a step of adjusting the contrast ratio of multiple cropping images, a step of blurring multiple cropping images, or a step of flipping multiple cropping images left and right. When training an artificial intelligence classification model using multiple training images preprocessed from multiple cropping images, the accuracy of identifying whether a train is coupled can be further increased.

[0078] The train coupling determination system (1) can train an artificial intelligence classification model used to determine whether a train is coupled using multiple training images in which multiple cropping images have been preprocessed (2300). Training of the artificial intelligence classification model through multiple training images in which multiple cropping images have been preprocessed can be carried out in program code.

[0079] FIG. 8 is a flowchart for identifying whether a train is coupled according to one embodiment.

[0080] To determine whether the train is coupled, a camera (120) within the train coupling determination system (1) can acquire surrounding images including the coupling pin (110) (3000). Due to the fluidity of the coupling pin (110) during train operation, the position of the coupling pin (110) can vary up, down, left, and right from its original position. Therefore, a process of acquiring multiple surrounding images including the coupling pin (110) may be required. The multiple surrounding images may be images that include an area that is increased by about 50% to the left and right and about 20% to the right from the area where the initial coupling pin (110) is located. However, the multiple surrounding images are not limited to this and may include various surrounding images around the coupling pin (110).

[0081] The train connection determination system (1) can obtain an image (3100) that crops a specific area of ​​an image including a connection pin (110). A cropped image can be obtained by randomly cropping a specific area of ​​a plurality of surrounding images including the connection pin (110). The randomly set specific area may be the size of the area including the initial connection pin (110), but is not limited thereto and can be set in various ways.

[0082] The train connection determination system (1) inputs a cropped image into a pre-trained artificial intelligence classification model to determine whether the connection pin (110) has fallen and can identify whether the train is connected (3200, 3300).

[0083] FIGS. 9 and 10 illustrate identifying whether a train is coupled using a line laser according to one embodiment.

[0084] The train coupling determination system (1) may include a line laser (130) that irradiates a laser toward a coupling pin (110). The laser generated by the line laser (130) appears on the surface of the coupling pin (110) when multiple trains are separated, i.e., when the coupling pin (110) has not fallen (Fig. 9). On the other hand, when multiple trains are coupled, i.e., when the coupling pin (110) has fallen, the laser line from the line laser (130) is dispersed into the air and does not appear on the surface of the coupling pin (110) (Fig. 10). Therefore, it is possible to determine whether the trains are coupled based on whether a laser line from the line laser (130) is detected in an image including the coupling pin (110) obtained by a camera (120) within the train coupling determination system (1). The train coupling determination system (1) may further include a detector for detecting a laser line in an image. The detector can perform image processing to detect a laser line in an image. The train connection determination system (1) can determine whether the connection pin (110) has fallen and whether the train has connected by using at least one of an artificial intelligence classification model and a detector for detecting a laser line.

[0085] In addition, it is possible to train an artificial intelligence classification model for determining whether a train is coupled based on the presence or absence of a laser line by a line laser (130) in a plurality of cropped images in which a specific area of ​​a plurality of images including a coupling pin (110) is cropped.

[0086] FIGS. 11 to 13 illustrate methods for more accurately determining whether a fastening pin has fallen according to one embodiment.

[0087] FIG. 11 illustrates a train connection determination system in which a background plate having a color contrasting with the connection pin is placed behind the connection pin according to one embodiment.

[0088] In order to train the train coupling judgment system (1) more accurately train the artificial intelligence classification model and determine whether the coupling pin (110) has fallen, a separate background plate having a color contrasting with the coupling pin (110) can be placed behind the coupling pin (110). That is, the coupling pin (110) is visually contrasted by the background plate, so that the coupling pin (110) has fallen and is highlighted in the image, thereby increasing the accuracy of the train coupling identification by the artificial intelligence classification model.

[0089] For example, contrasting colors may refer to complementary colors. In the HSV color system, contrasting colors, or complementary colors, have a difference of 180 degrees in hue, so to find the complementary color, 180 degrees can be added to or subtracted from the hue. However, contrasting colors are not limited to complementary colors and may refer to colors that can be highlighted in contrast to the background of the fastening pin (110).

[0090] FIG. 12 illustrates a fastening pin having a color that contrasts with a structure present within the shooting area according to one embodiment.

[0091] The train connection determination system (1) may use a connection pin (110) having a color that contrasts with the structure existing within the shooting area of ​​the camera (120) in order to more accurately train an artificial intelligence classification model and determine whether the connection pin (110) has fallen. That is, since the falling of the connection pin (110) having a color that contrasts with the structure is emphasized and appears on the image, the accuracy of the artificial intelligence classification model in identifying train connections can be increased.

[0092] For example, contrasting colors may refer to complementary colors. In the HSV color system, contrasting colors, or complementary colors, have a difference of 180 degrees in hue, so to find the complementary color, 180 degrees can be added to or subtracted from the hue. However, contrasting colors are not limited to complementary colors and may refer to colors that can be emphasized by contrasting with the structure existing within the shooting area where the connecting pin (110) is located.

[0093] FIG. 13 illustrates that, according to one embodiment, a structure present within the shooting area is painted in a color contrasting with the fastening pin.

[0094] In order to train the train coupling judgment system (1) more accurately train the artificial intelligence classification model and determine whether the coupling pin (110) has fallen, the structure of the train coupling judgment system existing within the train coupling judgment system (1) can be painted with a color that contrasts with the coupling pin (110). That is, since the falling status of the coupling pin (110) having a color that contrasts with the structure is highlighted and appears on the image, the accuracy of the artificial intelligence classification model's train coupling identification can be increased.

[0095] For example, contrasting colors may refer to complementary colors. In the HSV color system, contrasting colors, or complementary colors, have a difference of 180 degrees in hue, so to find the complementary color, 180 degrees can be added to or subtracted from the hue. However, contrasting colors are not limited to complementary colors and may refer to colors that can be emphasized by contrasting with the structure existing within the shooting area where the connecting pin (110) is located.

[0096] A train coupling determination system according to one embodiment of the present invention may include: a camera having a shooting area facing a coupling pin that couples or uncouples a plurality of trains; and a processor that acquires an image from the camera, crops a specific area of ​​the acquired image, and inputs the cropped image into a pre-trained artificial intelligence classification model to identify whether the trains are coupled by the coupling pin, wherein the pre-trained artificial intelligence classification model may be trained based on a plurality of cropped images obtained by cropping the specific area of ​​a plurality of images acquired by the camera.

[0097] In the above-described train connection determination system, the pre-trained artificial intelligence classification model may be trained based on a plurality of training images obtained by adjusting the contrast ratio of the plurality of cropping images.

[0098] In the above-described train connection determination system, the pre-trained artificial intelligence classification model may be trained based on a plurality of training images obtained by blurring the plurality of cropping images.

[0099] In the above-described train connection determination system, the pre-trained artificial intelligence classification model may be trained based on a plurality of training images obtained by flipping the plurality of cropping images left and right.

[0100] The above train coupling determination system further includes a detector for detecting a laser line by a line laser directed toward the coupling pin within the plurality of cropping images; and the processor can identify whether the train is coupled based on whether the laser line is detected by the detector.

[0101] The above train connection determination system may be a train connection determination system in which a background plate having a color contrasting with the connection pin is placed behind the connection pin in the shooting area.

[0102] In the above-described train connection determination system, the connection pin may be painted in a color that contrasts with the structure of the train connection determination system existing within the shooting area.

[0103] In the above-described train coupling determination system, the structure of the train coupling determination system existing within the above-described shooting area may be painted with a color contrasting with the coupling pin.

[0104] In one embodiment of the present invention, a method for training an artificial intelligence classification model used by a train coupling determination system including a camera may include: a step of acquiring a plurality of images having a shooting area facing a coupling pin that combines or uncouples a plurality of trains by the camera; a step of cropping a specific area of ​​the acquired plurality of images; and a step of training the artificial intelligence classification model based on the plurality of cropped images.

[0105] The step of training the artificial intelligence classification model based on the plurality of cropping images may include the step of training based on a plurality of training images obtained by adjusting the contrast ratio of the plurality of cropping images.

[0106] The step of training the artificial intelligence classification model based on the plurality of cropping images may include the step of training based on a plurality of training images obtained by blurring the plurality of cropping images.

[0107] The step of training the artificial intelligence classification model based on the plurality of cropping images may include the step of training based on a plurality of training images obtained by flipping the plurality of cropping images left and right.

[0108] The step of training the artificial intelligence classification model based on the plurality of cropping images may include the step of training based on the plurality of cropping images based on the presence or absence of a laser line by a line laser directed toward the fastening pin within the plurality of cropping images.

[0109] Conventionally, artificial intelligence (AI) detection models have been used to determine whether a fastening pin has fallen. These AI detection models can estimate the position of the fastening pin while simultaneously determining whether the train is coupled by the pin. However, because AI detection models perform these two functions simultaneously within a single model, they have disadvantages in terms of speed and accuracy.

[0110] The present invention acquires only images of the shooting area containing the fastening pin, trains an AI classification model based on training data that is lighter than that of an AI detection model, and enables the determination of whether a train is coupled. In other words, since the area for acquiring images is used in a fixed manner, a large number of images can be processed more quickly, thereby generating an AI classification model with high accuracy while reducing costs; by utilizing this model, it is possible to quickly and accurately identify whether a train is coupled.

[0111] Meanwhile, the disclosed embodiments may be implemented in the form of a recording medium that stores instructions executable by a computer. The instructions may be stored in the form of program code and, when executed by a processor, may generate a program module to perform the operation of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium.

[0112] Computer-readable recording media include all types of recording media that store instructions that can be decoded by a computer. Examples include ROM (read-only memory), RAM (random access memory), magnetic tape, magnetic disk, flash memory, optical data storage devices, etc.

[0113] Additionally, computer-readable recording media may be provided in the form of non-transitory storage media. Here, 'non-transitory storage media' simply means that it is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily. For example, 'non-transitory storage media' may include a buffer in which data is stored temporarily.

[0114] According to one embodiment, the method according to the various embodiments disclosed herein may be provided as included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable recording medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., downloadable app) may be temporarily stored or temporarily created on a device-readable recording medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0115] As described above, the disclosed embodiments have been explained with reference to the attached drawings. Those skilled in the art will understand that the present invention may be practiced in forms different from the disclosed embodiments without changing the technical spirit or essential features of the invention. The disclosed embodiments are illustrative and should not be interpreted restrictively.

Claims

1. A camera having a shooting area facing a connecting pin that connects or unconnects multiple trains; and Acquire an image from the above camera, and Cropping a specific area of ​​the above-mentioned acquired image, and A processor that identifies whether the train is coupled by the fastening pin by inputting the cropped image into a pre-trained artificial intelligence classification model; The above-mentioned pre-trained artificial intelligence classification model is, A train connection determination system learned based on a plurality of cropped images obtained by cropping a specific area of ​​a plurality of images obtained by the camera above.

2. In Paragraph 1, The above-mentioned pre-trained artificial intelligence classification model is, A train connection judgment system learned based on a plurality of learning images obtained by adjusting the contrast ratio of the above-mentioned plurality of cropping images.

3. In Paragraph 1, The above-mentioned pre-trained artificial intelligence classification model is, A train connection judgment system learned based on a plurality of training images obtained by blurring the above plurality of cropping images.

4. In Paragraph 1, The above-mentioned pre-trained artificial intelligence classification model is, A train connection judgment system learned based on a plurality of learning images obtained by flipping the above plurality of cropping images left and right.

5. In Paragraph 1, It further includes a detector for detecting a laser line by a line laser directed toward the fastening pin within the plurality of cropping images. The above processor A train coupling determination system that identifies whether the train is coupled based on whether the laser line is detected by the detector.

6. In Paragraph 1, A train connection determination system in which a background plate having a color contrasting with the connection pin is placed behind the connection pin in the above-mentioned shooting area.

7. In Paragraph 1 The above-mentioned fastening pin is a train fastening judgment system painted in a color contrasting with the structure existing within the above-mentioned shooting area.

8. In Paragraph 1, The structure existing within the above-mentioned shooting area is a train fastening judgment system painted in a color contrasting with the above-mentioned fastening pin.

9. A method for training an artificial intelligence classification model used by a train coupling judgment system including a camera, A step of acquiring a plurality of images having a shooting area facing a fastening pin that combines or uncombines a plurality of trains by the above camera; A step of cropping a specific region of the plurality of images obtained above; A method for training an artificial intelligence classification model comprising the step of training the artificial intelligence classification model based on the plurality of cropping images above.

10. In Paragraph 9, The step of training the artificial intelligence classification model based on the plurality of cropping images is: A method for training an artificial intelligence classification model comprising the step of training based on a plurality of training images obtained by adjusting the contrast ratio of the plurality of cropping images.

11. In Paragraph 9, The step of training the artificial intelligence classification model based on the plurality of cropping images is: An artificial intelligence classification model training method comprising: a step of training based on a plurality of training images obtained by blurring the plurality of cropping images above.

12. In Paragraph 9, The step of training the artificial intelligence classification model based on the plurality of cropping images is: A method for training an artificial intelligence classification model comprising the step of training based on a plurality of training images obtained by flipping the plurality of cropping images horizontally.

13. In Paragraph 9, The step of training the artificial intelligence classification model based on the plurality of cropping images is: A method for training an artificial intelligence classification model, comprising the step of learning based on the presence or absence of a laser line by a line laser directed toward the fastening pin within the plurality of cropping images.

Images