Railway crossing danger determination method and device

Abstract

The present invention provides a technology relating to a railway crossing danger determination method and device, wherein the railway crossing danger determination device installed at a railway crossing: receives railway control information and sensing information from a sensor to determine a railway crossing danger; calculates railway crossing safety facility state information on the basis of at least one of the railway control information and the sensing information; calculates obstacle presence / absence information according to one or more preconfigured obstacle detection algorithms on the basis of the sensing information; and determines whether preconfigured criteria for determination of danger are satisfied, on the basis of the railway crossing safety facility state information and the obstacle presence / absence information so as to determine whether there is a danger.

Description

Method and device for judging danger at crossings

[0001] These embodiments relate to a method and device for judging a risk at a railroad crossing.

[0002] Autonomous driving technology for locomotives (railway vehicles) is a key technology that can significantly contribute to the prevention of safety accidents by automating locomotives for transporting molten iron TLC in steel mills, which were previously driven manually by engineers, through sensor-based control and control-based route input, thereby preventing drowsy driving and carelessness due to long-distance driving.

[0003] A high incidence of safety accidents in autonomous locomotive operation occurs at level crossings where vehicles or pedestrians may pass along the locomotive's path. Therefore, these crossings require a special monitoring system to assess risk factors and implement safety measures, such as temporary stops or the sounding of the locomotive horn.

[0004] When autonomous locomotives drive, they typically utilize cameras and lidar mounted on the front to identify obstacles and hazards ahead. However, when a large object, such as a TLC, is pushed forward, the locomotive's view of the camera and lidar may be obstructed, making it difficult to detect hazards.

[0005] Therefore, there is a need for external devices and methods that enable autonomous locomotives to pass through crossings even when the locomotive's visibility is not secured.

[0006] The present embodiments can provide a method for judging a level crossing risk that detects a risk in advance when an autonomous locomotive is driving at a level crossing.

[0007] In addition, the present embodiments can provide a level crossing risk judgment device that detects risks in advance when an autonomous locomotive is driving at a level crossing.

[0008] In one aspect, the present embodiments may provide a method for judging a level crossing risk, including a receiving step of receiving railway control information and sensing information from a sensor of a locomotive, a status detection step of calculating railway crossing safety facility status information based on at least one of the railway control information and the sensing information, an obstacle detection step of calculating obstacle presence / absence information according to one or more preset obstacle detection algorithms based on the sensing information, and a risk / absence judgment step of determining whether a preset risk / absence criterion is met based on the railway crossing safety facility status information and the obstacle presence / absence information.

[0009] In another aspect, the present embodiments may provide a level crossing risk judgment device including a first body, a second body located on a lower side of the first body, a sensor including a lidar sensor and one or more camera sensors for obtaining sensing information, a control unit for judging whether a level crossing is dangerous using the sensing information, wherein the control unit includes a receiving unit for receiving sensing information from railway control information and a sensor of a locomotive, a status detection unit for calculating level crossing safety facility status information based on at least one of the railway control information and the sensing information, an obstacle detection unit for calculating obstacle presence / absence information according to one or more preset obstacle detection algorithms based on the sensing information, and a risk judgment unit for judging whether a preset risk criterion is satisfied based on the railway crossing safety facility status information and the obstacle presence / absence information.

[0010] According to the present embodiments, a method and device for judging a level crossing risk that detects a risk in advance when an autonomous locomotive is driving at a level crossing can be provided.

[0011] Figure 1 is a flowchart illustrating a method for judging a crossing risk according to one embodiment.

[0012] FIG. 2 is a diagram for explaining a traffic light status detection algorithm according to one embodiment.

[0013] FIG. 3 is a diagram illustrating an image-based obstacle detection algorithm according to one embodiment.

[0014] FIG. 4 is a diagram for explaining a circuit breaker status detection algorithm and a point-based object detection algorithm according to one embodiment.

[0015] Fig. 5 is a drawing for explaining a crossing hazard judgment device according to one embodiment.

[0016] FIG. 6 is a drawing for explaining a crossing hazard judgment device and a control unit of the crossing hazard judgment device according to one embodiment.

[0017] Hereinafter, some embodiments of the present disclosure will be described in detail with reference to exemplary drawings. When adding reference numerals to components in each drawing, identical components may have the same numerals as much as possible even if they are shown in different drawings. In addition, when describing the present embodiments, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the technical idea of ​​the present invention, the detailed description may be omitted. When "includes," "has," "consists of," etc. are used in this specification, other parts may be added unless "only" is used. When a component is expressed in the singular, it may include a case in which the plural is included unless specifically stated otherwise.

[0018] Additionally, terms such as first, second, A, B, (a), (b), etc. may be used to describe components of the present disclosure. These terms are only intended to distinguish the components from other components, and the nature, order, sequence, or number of the components are not limited by the terms.

[0019] In a description of the positional relationship of components, when it is described that two or more components are "connected," "combined," or "connected," it should be understood that the two or more components may be directly "connected," "combined," or "connected," but that the two or more components may also be further "interposed" with another component to be "connected," "combined," or "connected." Here, the other component may be included in one or more of the two or more components that are "connected," "combined," or "connected" to each other.

[0020] In the description of the temporal flow relationship related to components, operation methods, or manufacturing methods, for example, when the temporal or flow relationship is described as “after”, “following”, “next to”, “before”, etc., it may also include cases where it is not continuous, unless “immediately” or “directly” is used.

[0021] Meanwhile, when numerical values ​​or corresponding information (e.g., levels, etc.) for components are mentioned, even without separate explicit description, the numerical values ​​or corresponding information may be interpreted as including an error range that may occur due to various factors (e.g., process factors, internal or external impact, noise, etc.).

[0022] It will be understood by those skilled in the art that the terms “learning,” “learning,” and “algorithm” appearing throughout the detailed description and claims of the present disclosure are terms that refer to algorithms that perform or can perform machine learning or deep learning through procedural computing, and are not intended to refer to mental operations such as human educational activities or models to which mental operations are applied.

[0023] In this specification, a "locomotive" refers to a railway vehicle, a moving vehicle that runs on a railway, driven by a specific source of power. Therefore, a locomotive's power source can be configured in various ways, including an internal combustion engine, an electric motor, or an external power source, and there are no limitations. For ease of understanding, this specification uses the term "locomotive." However, regardless of the source of power, the term should be understood as a railway vehicle that runs on a railway.

[0024] Figure 1 is a flowchart illustrating a method for judging a crossing risk according to one embodiment.

[0025] According to FIG. 1, a method for judging a level crossing risk may include a receiving step of receiving railway control information and sensing information from a sensor, a status detection step of calculating railway crossing safety equipment status information based on at least one of the railway control information and the sensing information, an obstacle detection step of calculating obstacle presence / absence information according to one or more preset obstacle detection algorithms based on the sensing information, and a risk / absence judgment step of determining whether preset risk / absence criteria are met based on the railway crossing safety equipment status information and the obstacle presence / absence information.

[0026] The method for determining a level crossing risk may include a receiving step for receiving railway control information and sensing information from a sensor. (S110)

[0027] For example, railway control information may include at least one of signal light lighting information, level crossing location information, locomotive identification information, and track section information.

[0028] For example, traffic light lighting information may include lighting information for each color of a traffic light, which is used to derive traffic light status information, which will be described later. As another example, level crossing location information may be necessary because it is a factor that allows for determining whether an obstacle is detected at a level crossing, whether a barrier is operating normally, and what color of a traffic light is illuminated when determining danger. As another example, locomotive identification information may include number information for identifying the locomotive, information on the type of locomotive, and information on the color of the locomotive. As another example, track section information may include information on the track on which the locomotive will operate or on which track the locomotive regularly operates. Furthermore, track section information may include information on the location of traffic lights within the track section. Additionally, railway control information may include information on the location of traffic lights. However, the railway control information is not limited to this embodiment and may include various types of information.

[0029] The sensors described in this embodiment refer to various types of sensors, and their number is not limited. For example, sensors may include cameras, radar, lidar, and ultrasonic sensors. Alternatively, sensors may include sensors that generate sensing information about the locomotive's movements, such as speed sensors, motion detection sensors, acceleration sensors, and position sensors.

[0030] In addition, as an example, the sensing information may include traffic light lighting information and color-specific traffic light lighting information received from a camera sensor. As another example, the sensing information may include image information received from a camera sensor. As another example, the sensing information may include point information received from a lidar sensor. As another example, the sensing information may include circuit breaker location information and traffic light location information received from GPS, navigation, etc. In addition, the sensing information may include various information received from sensors configured on the locomotive, such as side information, rear information, structure information, locomotive location information, and locomotive acceleration information.

[0031] The method for assessing a level crossing risk may include a status detection step for calculating status information of a level crossing safety facility based on at least one of railway control information and sensing information. (S120)

[0032] For example, railroad crossing safety equipment status information can be derived based on at least one of railroad control information and sensing information. This railroad crossing safety equipment status information can include signal light status information, sensor-based signal light status information, and barrier direction information, which will be described later. Information contained in the railroad crossing safety equipment status information can be used to determine whether a locomotive is at risk of crossing.

[0033] For example, railroad crossing safety equipment status information includes signal light status information, and the signal light status information can be derived from signal light lighting information included in railroad control information.

[0034] The signal light information included in the railway control information may include color-specific signal light information. For example, the colors of the color-specific signal light information may include green, orange, and red. Green signal light information is a necessary factor for determining conditions under which a locomotive can drive on a railway. That is, when a green signal light is illuminated, the locomotive can be determined to be able to safely drive on the railway. Therefore, green signal light information may be included in the signal light information. Furthermore, when an orange or red signal light is illuminated, the locomotive must slow down or stop on the railway. Therefore, the signal light information may include orange signal light information and red signal light information. Furthermore, railroad crossing safety equipment status information may be derived by determining whether a green signal light is illuminated based on the signal light information, which includes color-specific signal light information.

[0035] For example, the state detection step can set a traffic light area based on image information included in the sensing information using a traffic light state detection algorithm, and evaluate whether a green traffic light is turned on based on image information of the traffic light area to produce sensor-based traffic light state information.

[0036] During the status detection phase, status information on railroad crossing safety equipment can be derived. This can be based on railway control information, but it can also be derived based on sensing information. While railway control information is relatively accurate, delays in information transmitted from the control tower can occur due to issues such as communication errors and control center failures. Therefore, determining the status of traffic lights using only railway control information can be risky. Therefore, the status detection phase can derive sensor-based traffic light status information based on image information included in the sensing information. A detailed description of this embodiment will be provided later with reference to FIG. 2.

[0037] For example, a circuit breaker status detection algorithm can be used to set a circuit breaker area based on point information included in sensing information, calculate information on the presence or absence of points in the circuit breaker area, and classify the direction of the circuit breaker based on the point presence or absence information to calculate the railroad crossing safety equipment status information.

[0038] The status detection step can classify the direction of the barrier to derive status information for railroad crossing safety equipment. In this case, control information is not used because information on the barrier direction is not included in the control information. Therefore, sensing information can be used to classify the barrier direction, and based on this, railroad crossing safety equipment status information can be derived. A detailed description of this embodiment will be provided later with reference to FIG. 4.

[0039] The method for determining the risk of a crossing may include an obstacle detection step for calculating information on the presence or absence of an obstacle according to one or more preset obstacle detection algorithms based on sensing information. (S130)

[0040] For example, each obstacle detection algorithm may include at least one image-based obstacle detection algorithm that extracts information on the presence or absence of an obstacle using image information detected based on sensing information, and one point-based obstacle detection algorithm that extracts information on the presence or absence of an obstacle using point information detected based on sensing information.

[0041] For example, an image-based obstacle detection algorithm can extract information on the presence or absence of an obstacle by calculating image feature information from image information detected within a preset distance and calculating a probability value based on the image feature information.

[0042] For example, a point-based object detection algorithm can select a point detection area based on point information included in sensing information, derive point feature information using the point information included in the point detection area, and extract information on the presence or absence of an obstacle if the point feature information satisfies a preset criterion.

[0043] The obstacle detection algorithm for extracting obstacle presence / absence information is described in detail below with reference to FIGS. 3 and 4.

[0044] It may include a risk assessment step for determining whether preset risk criteria are met based on information on the status of railroad crossing safety equipment and the presence or absence of obstacles. (S140)

[0045] For example, the step of determining whether there is a risk may further include a step of synchronizing the coordinate systems of a plurality of pieces of information in order to use information on the status of railroad crossing safety equipment and information on the presence or absence of obstacles.

[0046] The signal light location information included in the control information or the barrier location information included in the railroad crossing safety equipment status information derived through the barrier status detection algorithm can be set in advance as a three-dimensional coordinate system. However, the obstacle presence / absence information extracted based on image information and point information may be set as a mixture of two-dimensional and three-dimensional coordinate systems. Therefore, this embodiment describes in detail the operation of synchronizing the coordinate systems of the obstacle presence / absence information extracted through the image-based obstacle detection algorithm and the obstacle presence / absence information extracted through the point-based obstacle detection algorithm.

[0047] In this embodiment, the obstacle presence / absence information extracted through an image-based obstacle detection algorithm is defined as “first obstacle presence / absence information.” The obstacle presence / absence information extracted through a point-based obstacle detection algorithm is defined as “second obstacle presence / absence information.”

[0048] For example, the first obstacle presence information may be preset in a two-dimensional coordinate system, and the second obstacle presence information may be preset in a three-dimensional coordinate system. In this case, there is a need to unify the coordinate systems of the first obstacle presence information and the second obstacle presence information.

[0049] In this embodiment, an example of changing the coordinate system of the second obstacle presence / absence information from a three-dimensional coordinate system to a two-dimensional coordinate system is described. For example, the coordinate system of the second obstacle presence / absence information can be changed to the same coordinate system as the coordinate system of the first obstacle presence / absence information using the first parameter and the second parameter. The same coordinate system can be set as a three-dimensional coordinate system, a two-dimensional coordinate system, or a two-dimensional BEV's coordinate system. In addition, the first parameter can include a positional difference between one or more sensors and a direction difference between one or more sensors. The second parameter can include a focal length, a principal point, and an asymmetry coefficient.

[0050] According to this embodiment, the risk assessment step may synchronize the coordinate systems of multiple pieces of information to utilize information on the status of railroad crossing safety equipment and the presence or absence of obstacles. Furthermore, the risk assessment step may determine whether the locomotive is in danger based on the synchronized coordinate systems. However, this embodiment is merely an example, and various other embodiments may be included.

[0051] As another example, the preset risk criterion is determined using three factors: the presence or absence of an obstacle, the direction of the barrier, and whether a green signal light is on, and if at least one of the three factors does not satisfy the safety condition, it can be determined that a risk exists.

[0052] If safety conditions are determined based solely on one or two factors, the locomotive may be at risk if sensor failure results in incorrect sensing data or inaccurate railway control information. Therefore, in this embodiment, three factors are used to determine the preset risk criteria. However, the preset risk criteria are not limited to this embodiment and can be determined in various ways.

[0053] As another example, if the absence of an obstacle is determined based on obstacle presence information, the safety condition can be determined to be satisfied. Furthermore, if the presence of an obstacle is determined based on obstacle presence information, the safety condition can be determined to be satisfied. In this case, the obstacle presence information can include obstacle location information and obstacle shape information.

[0054] As another example, the circuit breaker direction can be categorized into upward, downward, upward-going, and downward-going states. For example, if the circuit breaker direction is in one of the downward, upward, or upward-going states, it can be determined that the safety condition is not satisfied. Furthermore, if the circuit breaker direction is in the downward state, it can be determined that the safety condition is satisfied. However, the circuit breaker direction is not limited to the four categorized states and various directions can be preset.

[0055] As another example, the presence of a green traffic light can be used to determine whether a safety condition is satisfied. For example, if a green traffic light is determined to be on, the safety condition can be determined to be satisfied. However, if either a red or orange traffic light is determined to be on, the safety condition cannot be determined to be satisfied.

[0056] According to this embodiment, the presence or absence of a preset risk can be determined using three factors: information on the presence or absence of an obstacle, the direction of the barrier, and whether a green signal light is on. However, this is not limited to this embodiment, and the three factors can be preset in various ways.

[0057] Below, the aforementioned method of assessing the risk of crossings is explained in more detail and in various ways with reference to drawings.

[0058] FIG. 2 is a diagram for explaining a traffic light status detection algorithm according to one embodiment.

[0059] Referring to FIG. 2, a traffic light status detection algorithm (200) is used to set a traffic light area based on image information included in sensing information, and sensor-based traffic light status information (220) can be produced by evaluating whether a green traffic light is turned on based on image information of the traffic light area.

[0060] When a traffic light area is set based on image information, a traffic light status detection algorithm (200) can extract an image (210) of the traffic light area. When the image of the traffic light area is extracted, evaluation data that can evaluate whether a green traffic light is turned on can be generated using a plurality of Residual Blocks (210). The Residual Block (210) can include two or more Convolution Layers (211), a ReLu function (212), and a skip connection (212). Based on the evaluation data that has passed through the plurality of Residual Blocks (210), it is evaluated whether a green traffic light included in the traffic light area is turned on, and sensor-based traffic light status information (220) can be produced.

[0061] In addition, there are no restrictions on traffic light status detection algorithms, as long as they classify and recognize specific information based on a given image. For example, they could be CNN-based AI algorithms. Furthermore, they could be generated based on AlexNet, ResNet, GoogLeNet, VGG, and other such algorithms.

[0062] FIG. 3 is a diagram illustrating an image-based obstacle detection algorithm according to one embodiment.

[0063] Referring to FIG. 3, an image-based obstacle detection algorithm (310) can extract image feature information (312, 313) using a preset operation method (320, 321, 322) based on image information, and extract obstacle presence / absence information using a preset learning method (330) based on the extracted image feature information (312, 313).

[0064] Image information can extract one or more image feature information using a preset calculation method.

[0065] Referring to FIG. 3, one or more image feature information (312, 313) can be extracted using preset operation methods (320, 321, 322) based on image information. For example, image information can be extracted as first image feature information (312) through a first operation method (321). The first operation method (321) can extract image information as first image feature information (312), which is high-resolution information, using a ConvBlock in which a 3x3 Convolution Layer, a Batch Normalization Layer, and a ReLU activation function are arranged in order. One or more ConvBlocks can be used, and there is no limitation on the number of ConvBlocks.

[0066] Additionally, image information can be extracted as second image feature information (313) through a second operation method (322). The second operation method (322) can extract the second image feature (313) by reducing the amount of information of the image information using a 2x2 max pooling method.

[0067] The image-based obstacle detection algorithm (310) can extract obstacle presence / absence information using a preset learning method (330) based on first image feature information (312) and second image feature information (313). The preset learning method (330) can include a first learning method, a second learning method, and a third learning method.

[0068] For example, the first image feature information (312) can be copied through the first learning method to generate the first obstacle presence information. The second image feature information (313) can be generated as the second obstacle presence information by using the second learning method, but doubling the dimensionality of the information and reducing the number of channels by half through a transposed convolution layer. The third learning method can generate the obstacle presence information by combining the first obstacle presence information and the second obstacle presence information and then using the above-described ConvBlock.

[0069] Furthermore, this embodiment provides a brief overview of an image-based obstacle detection algorithm. Therefore, one or more preset computational methods (320) and preset learning methods (330) may be utilized, and there are no restrictions on their order or number. Furthermore, the image-based obstacle detection algorithm can be generated based on Yolox, and various algorithms can be utilized.

[0070] FIG. 4 is a diagram for explaining a circuit breaker status detection algorithm and a point-based object detection algorithm according to one embodiment.

[0071] Referring to FIG. 4, the point-based obstacle detection algorithm selects a point detection area based on point information included in sensing information, calculates point feature information using the point information included in the point detection area, and extracts obstacle presence / absence information when the point feature information satisfies a preset criterion.

[0072] For example, a point-based obstacle detection algorithm may select a point detection area based on point information (410) received through a lidar sensor, and perform a clustering process (420). Here, the point detection area may be defined as a surveillance area in which it is determined whether an obstacle exists. In addition, the clustering process (420) may include an algorithm that groups points of a point cloud with similar characteristics included in the point information (410). That is, the point information (410) may be converted into a clustered point group through the clustering process (420). For example, the described algorithm may use K-means, DBSCAN, and MeanShift algorithms. However, the present invention is not limited to this algorithm, and various algorithms may be utilized.

[0073] In addition, the point-based obstacle detection algorithm can output point feature information by inputting a clustered point group into a feature extraction process (430). The feature extraction process (430) can output point feature information by extracting object characteristics from the clustered point group. The feature extraction process (430) can output point feature information such as the shape, size, direction, height, etc. of the object by using the distance, distribution, and density between points within the clustered point group. When performing the extraction process (430), the PointNet or PointNet++ algorithm can be used. However, the present invention is not limited to the present algorithm and various algorithms can be utilized.

[0074] Additionally, the point-based obstacle detection algorithm can identify and classify objects through a classification process (440) using point feature information. When performing the classification process (440), at least one or more of the PointNet, PointNet++, PointCNN, PointRCNN, KPConv, DGCNN, and FusionNet algorithms may be used. However, various algorithms may be utilized without being limited to the described algorithms.

[0075] In addition, the point-based obstacle detection algorithm can extract information on the presence or absence of an obstacle after determining whether a preset criterion is met by using an object detection process (450) based on point feature information for which a classification process (440) has been performed.

[0076] The object detection process (450) can identify an actual object based on point feature information on which a classification process has been performed. Through the object detection process (450), the location, direction, size, etc. of the object can be determined, and the object can be distinguished from other objects. When the object detection process (450) is performed, at least one of the PointRCNN, KPConv, VoteNet, and FusionNet algorithms can be used. However, the algorithms are not limited to the described algorithms and various algorithms can be utilized. In addition, the obstacle presence / absence information can include obstacle location information and obstacle name information.

[0077] Additionally, the circuit breaker status detection algorithm shares similarities with the point-based obstacle detection algorithm in terms of operation, function, control method, etc. Therefore, the description of the circuit breaker status detection algorithm described below focuses on the classification process (440) that is different from the point-based obstacle detection algorithm.

[0078] Referring to FIG. 4, a circuit breaker status detection algorithm can be used to set a circuit breaker area based on point information (410), and information on the presence or absence of points in the circuit breaker area can be derived through a clustering process (420) and a feature extraction process (430). Furthermore, based on the point presence or absence information, information on the status of railroad crossing safety equipment can be derived through a classification process (440) and an object detection process (450).

[0079] The feature extraction process (430) can produce information such as the position, shape, size, direction, height, etc. of an object. Information on the direction of a circuit breaker can be produced by synthesizing information on the height, shape, direction, height, etc. of the clustered points that passed through the feature extraction process (430). The classification process (440) can identify and classify objects. Therefore, the circuit breaker status can be classified through the classification process (440) based on the produced information on the direction of the circuit breaker. The circuit breaker status can be classified into an upward state, a downward state, an upward state, and a downward state as described above. However, the circuit breaker status is not limited to a total of four and can be set in advance in various ways.

[0080] Fig. 5 is a drawing for explaining a crossing hazard judgment device according to one embodiment.

[0081] Referring to FIG. 5, the crossing hazard judgment device may include a first body (510), a second body (520) located at the lower side of the first body, a lidar sensor (540) for obtaining sensing information, and sensors (530, 531, 532) including one or more camera sensors.

[0082] In addition, the first body (510) and the second body (520) are connected by one or more supports (550, 551, 552, 553), the lidar sensor (540) is disposed on the upper surface of the first body (510), and a rotation bracket (541) is included between the lidar sensor (540) and the upper surface of the first body (510) to rotate the lidar sensor (540), the first camera (530) included in the one or more camera sensors is disposed on the lower surface of the first body (510), and the second camera (531) and the third camera (532) included in the one or more camera sensors can be disposed spaced apart from each other on the upper surface of the second body (520).

[0083] For example, the first camera (530) may be placed on the lower surface of the first body (510) and between the second support (551) and the third support (552). In addition, the second camera (531) may be placed on the upper surface of the second body (520) and between the first support (550) and the second support (551). In addition, the third camera (532) may be placed on the upper surface of the second body (520) and between the third support (552) and the fourth support (553).

[0084] For example, the first camera (530) may be placed on the upper surface of the second body (520) and between the second support (551) and the third support (552). In addition, the second camera (531) may be placed on the lower surface of the first body (520) and between the first support (550) and the second support (551). In addition, the third camera (532) may be placed on the lower surface of the first body (520) and between the third support (552) and the fourth support (553).

[0085] However, one or more camera sensors are not limited to this embodiment and may be positioned in various locations.

[0086] For example, a support (560) for fixing a crossing hazard judgment device may be placed at the bottom of the second body (520). Additionally, wheels and outriggers may be further placed on the lower surface of the support (560). This allows the crossing hazard judgment device to be moved or fixed to the ground depending on the situation. However, the configuration of the support (560) is not limited and various configurations may be placed.

[0087] FIG. 6 is a drawing for explaining a crossing hazard judgment device and a control unit of the crossing hazard judgment device according to one embodiment.

[0088] According to FIG. 6, a level crossing hazard judgment device includes a first body, a second body located below the first body, a sensor including a lidar sensor and one or more camera sensors for obtaining sensing information, and a control unit (600) for judging whether a level crossing is dangerous using the sensing information, wherein the control unit (600) may include a receiving unit (610) for receiving sensing information from railway control information and the sensor, a status detection unit (620) for calculating railway crossing safety facility status information based on at least one of the railway control information and the sensing information, an obstacle detection unit (630) for calculating obstacle presence / absence information according to one or more preset obstacle detection algorithms based on the sensing information, and a risk judgment unit (640) for judging whether a preset risk criterion is satisfied based on the railway crossing safety facility status information and the obstacle presence / absence information.

[0089] In addition, the crossing hazard judgment device includes a first body and a second body connected by one or more supports, a lidar sensor disposed on an upper surface of the first body, and a rotating bracket included between the lidar sensor and the upper surface of the first body to rotate the lidar sensor, a first camera included in one or more camera sensors may be disposed on a lower surface of the first body, and a second camera and a third camera included in one or more camera sensors may be disposed spaced apart from each other on the upper surface of the second body. Since the shape of the crossing hazard judgment device has been described in detail with reference to FIG. 5, the following description will focus on the control unit (600) of the crossing hazard judgment device.

[0090] The control unit (600) of the level crossing risk judgment device may include a receiving unit (610) that receives railway control information and sensing information from a sensor.

[0091] For example, railway control information may include at least one of signal light lighting information, level crossing location information, locomotive identification information, and track section information.

[0092] For example, traffic light lighting information may include lighting information for each color of a traffic light, which is used to derive traffic light status information, which will be described later. As another example, level crossing location information may be necessary because it is a factor that allows for determining whether an obstacle is detected at a level crossing, whether a barrier is operating normally, and what color of a traffic light is illuminated when determining danger. As another example, locomotive identification information may include number information for identifying the locomotive, information on the type of locomotive, and information on the color of the locomotive. As another example, track section information may include information on the track on which the locomotive will operate or on which track the locomotive regularly operates. Furthermore, track section information may include information on the location of traffic lights within the track section. Additionally, railway control information may include information on the location of traffic lights. However, the railway control information is not limited to this embodiment and may include various types of information.

[0093] The sensors described in this embodiment refer to various types of sensors, and their number is not limited. For example, sensors may include cameras, radar, lidar, and ultrasonic sensors. Alternatively, sensors may include sensors that generate sensing information about the locomotive's movements, such as speed sensors, motion detection sensors, acceleration sensors, and position sensors.

[0094] In addition, as an example, the sensing information may include traffic light lighting information and color-specific traffic light lighting information received from a camera sensor. As another example, the sensing information may include image information received from a camera sensor. As another example, the sensing information may include point information received from a lidar sensor. As another example, the sensing information may include circuit breaker location information and traffic light location information received from GPS, navigation, etc. In addition, the sensing information may include various information received from sensors configured on the locomotive, such as side information, rear information, structure information, locomotive location information, and locomotive acceleration information.

[0095] The control unit (600) of the level crossing risk judgment device may include a status detection unit (620) that calculates status information of a level crossing safety facility based on at least one of railway control information and sensing information.

[0096] For example, railroad crossing safety equipment status information can be derived based on at least one of railroad control information and sensing information. This railroad crossing safety equipment status information can include signal light status information, sensor-based signal light status information, and barrier direction information, which will be described later. Information contained in the railroad crossing safety equipment status information can be used to determine whether a locomotive is at risk of crossing.

[0097] For example, railroad crossing safety equipment status information includes signal light status information, and the signal light status information can be derived from signal light lighting information included in railroad control information.

[0098] The signal light information included in the railway control information may include color-specific signal light information. For example, the colors of the color-specific signal light information may include green, orange, and red. Green signal light information is a necessary factor for determining conditions under which a locomotive can drive on a railway. That is, when a green signal light is illuminated, the locomotive can be determined to be able to safely drive on the railway. Therefore, green signal light information may be included in the signal light information. Furthermore, when an orange or red signal light is illuminated, the locomotive must slow down or stop on the railway. Therefore, the signal light information may include orange signal light information and red signal light information. Furthermore, railroad crossing safety equipment status information may be derived by determining whether a green signal light is illuminated based on the signal light information, which includes color-specific signal light information.

[0099] For example, the status detection unit (620) can set a traffic light area based on image information included in sensing information using a traffic light status detection algorithm, and can evaluate whether a green traffic light is turned on based on image information of the traffic light area to produce sensor-based traffic light status information.

[0100] The status detection unit (620) can calculate status information on railroad crossing safety equipment. This can be calculated based on railroad control information, but it can also be calculated based on sensing information. While railroad control information is relatively accurate, delays in information transmitted from the control tower may occur due to issues such as communication errors and control center failures. Therefore, there is a potential risk in determining the status of traffic lights using only railroad control information. Therefore, the status detection unit (620) can calculate sensor-based traffic light status information based on image information included in the sensing information.

[0101] For example, a circuit breaker status detection algorithm can be used to set a circuit breaker area based on point information included in sensing information, calculate information on the presence or absence of points in the circuit breaker area, and classify the direction of the circuit breaker based on the point presence or absence information to calculate the railroad crossing safety equipment status information.

[0102] The status detection unit (620) can classify the direction of the barrier and derive status information for railroad crossing safety equipment. In this case, control information is not used because information on the barrier direction is not included in the control information. Therefore, sensing information can be used to classify the direction of the barrier, and based on this, railroad crossing safety equipment status information can be derived.

[0103] The control unit (600) of the crossing hazard judgment device may include an obstacle detection unit (630) that calculates information on the presence or absence of an obstacle according to one or more preset obstacle detection algorithms based on sensing information.

[0104] For example, each obstacle detection algorithm may include at least one image-based obstacle detection algorithm that extracts information on the presence or absence of an obstacle using image information detected based on sensing information, and one point-based obstacle detection algorithm that extracts information on the presence or absence of an obstacle using point information detected based on sensing information.

[0105] For example, an image-based obstacle detection algorithm can extract information on the presence or absence of an obstacle by calculating image feature information from image information detected within a preset distance and calculating a probability value based on the image feature information.

[0106] For example, a point-based object detection algorithm can select a point detection area based on point information included in sensing information, derive point feature information using the point information included in the point detection area, and extract information on the presence or absence of an obstacle if the point feature information satisfies a preset criterion.

[0107] It may include a risk judgment unit (640) that judges whether preset risk criteria are met based on information on the status of railroad crossing safety equipment and information on the presence or absence of obstacles.

[0108] For example, the risk judgment unit (640) may further include a coordinate synchronization unit that synchronizes the coordinate systems of a plurality of pieces of information in order to use information on the status of railroad crossing safety equipment and information on the presence or absence of obstacles.

[0109] The signal light location information included in the control information or the barrier location information included in the railroad crossing safety equipment status information derived through the barrier status detection algorithm can be set in advance as a three-dimensional coordinate system. However, the obstacle presence / absence information extracted based on image information and point information may be set as a mixture of two-dimensional and three-dimensional coordinate systems. Therefore, this embodiment describes in detail an operation for synchronizing the coordinate systems of the obstacle presence / absence information (first obstacle presence / absence information) extracted through the image-based obstacle detection algorithm and the obstacle presence / absence information (second obstacle presence / absence information) extracted through the point-based obstacle detection algorithm.

[0110] For example, the first obstacle presence information may be preset in a two-dimensional coordinate system, and the second obstacle presence information may be preset in a three-dimensional coordinate system. In this case, there is a need to unify the coordinate systems of the first obstacle presence information and the second obstacle presence information.

[0111] In this embodiment, an example of changing the coordinate system of the second obstacle presence / absence information from a three-dimensional coordinate system to a two-dimensional coordinate system is described. For example, the coordinate system of the second obstacle presence / absence information can be changed to the same coordinate system as the coordinate system of the first obstacle presence / absence information using the first parameter and the second parameter. The same coordinate system can be set as a three-dimensional coordinate system, a two-dimensional coordinate system, or a two-dimensional BEV's coordinate system. In addition, the first parameter can include a positional difference between one or more sensors and a direction difference between one or more sensors. The second parameter can include a focal length, a principal point, and an asymmetry coefficient.

[0112] According to this embodiment, the risk assessment unit (640) can synchronize the coordinate systems of multiple pieces of information to utilize information on the status of railroad crossing safety equipment and the presence or absence of obstacles. Furthermore, the risk assessment unit (640) can determine whether a locomotive is in danger based on the synchronized coordinate system. However, this embodiment is merely an example, and various other embodiments may be included.

[0113] As another example, the preset risk criterion is determined using three factors: the presence or absence of an obstacle, the direction of the barrier, and whether a green signal light is on, and if at least one of the three factors does not satisfy the safety condition, it can be determined that a risk exists.

[0114] If safety conditions are determined based solely on one or two factors, the locomotive may be at risk if sensor failure results in incorrect sensing data or inaccurate railway control information. Therefore, in this embodiment, three factors are used to determine the preset risk criteria. However, the preset risk criteria are not limited to this embodiment and can be determined in various ways.

[0115] As another example, if the absence of an obstacle is determined based on obstacle presence information, the safety condition can be determined to be satisfied. Furthermore, if the presence of an obstacle is determined based on obstacle presence information, the safety condition can be determined to be satisfied. In this case, the obstacle presence information can include obstacle location information and obstacle shape information.

[0116] As another example, the circuit breaker direction can be categorized into upward, downward, upward-going, and downward-going states. For example, if the circuit breaker direction is in one of the downward, upward, or upward-going states, it can be determined that the safety condition is not satisfied. Furthermore, if the circuit breaker direction is in the downward state, it can be determined that the safety condition is satisfied. However, the circuit breaker direction is not limited to the four categorized states and various directions can be preset.

[0117] As another example, the presence of a green traffic light can be used to determine whether a safety condition is satisfied. For example, if a green traffic light is determined to be on, the safety condition can be determined to be satisfied. However, if either a red or orange traffic light is determined to be on, the safety condition cannot be determined to be satisfied.

[0118] According to this embodiment, the presence or absence of a preset risk can be determined using three factors: information on the presence or absence of an obstacle, the direction of the barrier, and whether a green signal light is on. However, this is not limited to this embodiment, and the three factors can be preset in various ways.

[0119]

[0120] As described above, the device and method for judging the risk of a level crossing according to the present disclosure can be used to provide a function for judging the risk of a locomotive in operation and avoiding the risk.

[0121] The algorithms described above can be used in various ways depending on the operator's choice, in addition to the above-described model.

[0122]

[0123] *The control unit (600) of the crossing risk judgment device may be configured as a computing system or as a GPU server equipped with a GPU processor and general memory, but the present embodiments are not limited thereto.

[0124] The control unit (600) of the crossing hazard judgment device may be implemented by a computing device including at least some of a processor, a memory, a user input device, and a presentation device. The memory is a medium that stores computer-readable software, applications, program modules, routines, instructions, and / or data, which are coded to perform a specific task when executed by the processor. The processor can read and execute the computer-readable software, applications, program modules, routines, instructions, and / or data stored in the memory. The user input device may be a means for allowing a user to input a command to cause the processor to perform a specific task or to input data necessary for the execution of a specific task. The user input device may include a physical or virtual keyboard or keypad, key buttons, a mouse, a joystick, a trackball, a touch-sensitive input device, or a microphone. The presentation device may include a display, a printer, a speaker, or a vibration device.

[0125] Computing devices can include a variety of devices, including smartphones, tablets, laptops, desktops, servers, and clients. A computing device may be a single, standalone device, or it may include multiple computing devices operating in a distributed environment, each of which collaborates with another through a communications network.

[0126] In addition, the control unit (600) of the aforementioned crossing hazard judgment device can be executed by a computing device having a processor and a memory storing computer-readable software, applications, program modules, routines, instructions, and / or data structures coded to perform an image classification method utilizing a deep learning model when executed by the processor.

[0127] The artificial intelligence model described in these embodiments may be a current or future machine learning model, such as a model that performs algorithm-based machine learning operating on the aforementioned computing device or a model that performs artificial neural network-based learning.

[0128] Models that perform algorithm-based machine learning can be classical machine learning models such as tree-based models, k-Nearest Neighbors, k-Means Clustering, Principal Component Analysis (PCA), and support vector machines (SVM).

[0129] A tree-based model can be, for example, a decision tree model, a regression model, or a random tree model.

[0130] Meanwhile, an artificial intelligence model can be an ensemble model that solves problems by training and combining multiple models rather than using just one trained model.

[0131] Ensemble models combine multiple individually trained models to prevent overfitting and improve generalization performance. Ensemble models can be helpful in improving performance when the performance of individual models is not sufficient.

[0132] Ensemble models can be broadly divided into voting and boosting methods.

[0133] Voting methods derive a final result through voting on the results generated by multiple models. Examples include bagging, which combines algorithms of the same type but trains them on different data sets, and voting, which combines different types of algorithms.

[0134] Boosting is a method of combining weak machine learning models to create a more accurate and powerful model. Boosting involves sequentially performing tasks on each weak machine learning model, with subsequent models exploring additional areas missed by the previous models. Examples of boosting methods include random forests, gradient boosting, and XGBoost (eXtra Gradient Boost).

[0135] An artificial neural network (ANN) is a machine learning algorithm that analyzes and learns complex data based on a large number of interconnected artificial neurons, mimicking the operating principles of the human brain. An ANN can be any type of ANN, including the multilayer perceptron (MLP), the most basic ANN structure consisting of an input layer, a hidden layer, and an output layer; a convolutional neural network (CNN), which performs convolution operations to extract image features and reduces dimensionality through pooling operations; and a recurrent neural network (RNN), an ANN structure used to process ordered data. These ANNs can be modified in various ways depending on the complexity and diversity of the data.

[0136] A model that has undergone learning based on an artificial neural network can also be an ensemble model that solves problems by learning multiple models and combining them rather than learning just one model.

[0137] Meanwhile, algorithm-based machine learning models and models trained using artificial neural networks can be used complementarily. For example, an algorithm-based machine learning model can use the results of an artificial neural network-based model, and vice versa. An ensemble model combining algorithm-based machine learning models and artificial neural network-based models can also be used.

[0138] The embodiments described above may be implemented through various means. For example, the embodiments may be implemented through hardware, firmware, software, or a combination thereof.

[0139] In the case of hardware implementation, the method for determining the risk of a crossing according to the present embodiments may be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), processors, controllers, microcontrollers, or microprocessors.

[0140] For example, the method for determining the risk of crossing according to the embodiments can be implemented using an artificial intelligence semiconductor device in which the neurons and synapses of a deep neural network are implemented using semiconductor devices. The semiconductor devices may be currently used semiconductor devices, such as SRAM, DRAM, or NAND, or may be next-generation semiconductor devices, such as RRAM, STT MRAM, or PRAM, or may be a combination thereof.

[0141] When implementing a method for judging the risk of crossing a road according to embodiments using an artificial intelligence semiconductor device, the results (weights) of learning a deep learning model using software may be transferred to synapse-mimicking elements arranged in an array, or learning may be performed in the artificial intelligence semiconductor device.

[0142] When implemented using firmware or software, the method for determining the risk of crossing according to the present embodiments may be implemented in the form of a device, procedure, or function that performs the functions or operations described above. The software code may be stored in a memory unit and executed by a processor. The memory unit may be located within or outside the processor and may exchange data with the processor using various known means.

[0143] Additionally, terms such as "system," "processor," "controller," "component," "module," "interface," "model," or "unit" as described above may generally refer to a computer-related entity, such as hardware, a combination of hardware and software, software, or software in execution. For example, the aforementioned components may be, but are not limited to, a process driven by a processor, a processor, a controller, a control processor, an object, a thread of execution, a program, and / or a computer. For example, both an application running on a controller or a processor and the controller or the processor may be components. One or more components may be within a process and / or thread of execution, and the components may be located on a single device (e.g., a system, a computing device, etc.) or distributed across two or more devices.

[0144] The above description is merely an illustrative example of the technical idea of ​​the present disclosure, and those skilled in the art to which the present disclosure pertains will appreciate that various modifications and variations can be made without departing from the essential characteristics of the technical idea of ​​the present disclosure. In addition, the present embodiments are not intended to limit the technical idea of ​​the present disclosure but rather to explain it, and therefore the scope of the technical idea of ​​the present disclosure is not limited by these embodiments. The scope of protection of the present disclosure should be interpreted by the claims below, and all technical ideas within a scope equivalent thereto should be interpreted as being included within the scope of the rights of the present disclosure.

[0145]

[0146] CROSS-REFERENCE TO RELATED APPLICATION

[0147] This patent application claims priority under 35 USC § 119(a) to Korean Patent Application No. 10-2023-0183654, filed December 15, 2023, the entire contents of which are incorporated herein by reference. Furthermore, this patent application claims priority in countries other than the United States for the same reasons, the entire contents of which are incorporated herein by reference.

Claims

1. In the method for determining the risk of a crossing by a crossing risk assessment device installed at a crossing, A receiving step for receiving railway control information and sensing information from sensors; A status detection step for calculating status information of railroad crossing safety equipment based on at least one of the above railroad control information and the above sensing information; An obstacle detection step for calculating obstacle presence / absence information according to one or more preset obstacle detection algorithms based on the sensing information; and A method for judging a level crossing risk, comprising a risk judgment step of judging whether preset risk criteria are met based on the above-mentioned railroad crossing safety equipment status information and the above-mentioned obstacle presence / absence information.

2. In paragraph 1, The above railway control information is, A method for judging a level crossing hazard, comprising at least one of signal light lighting information, level crossing location information, locomotive identification information, and track section information.

3. In paragraph 1, The above railroad crossing safety equipment status information is: A method for judging the risk of a level crossing, which includes signal light status information, wherein the signal light status information is calculated by signal light lighting information included in the railway control information.

4. In paragraph 3, The above state detection step is, A method for determining the risk of a crossing, which sets a traffic light area based on image information included in the sensing information using the above traffic light status detection algorithm, evaluates whether a green traffic light is lit based on image information of the traffic light area, and derives sensor-based traffic light status information.

5. In paragraph 1, The above state detection step is, A method for judging the risk of a level crossing, comprising: setting a level crossing breaker area based on point information included in the sensing information using a level crossing breaker status detection algorithm; calculating information on the presence or absence of points existing in the level crossing breaker area; and classifying the level crossing breaker direction based on the point presence or absence information to calculate the level crossing safety equipment status information.

6. In paragraph 1, Each of the above obstacle detection algorithms, A method for determining a level crossing risk, comprising at least one image-based obstacle detection algorithm that extracts information on the presence or absence of an obstacle using image information detected based on the sensing information, and at least one point-based obstacle detection algorithm that extracts information on the presence or absence of an obstacle using point information detected based on the sensing information.

7. In paragraph 6 The above image-based obstacle detection algorithm, A method for determining a risk of a crossing, wherein image feature information is derived from the detected image information within a preset distance, and information on the presence or absence of an obstacle is extracted using a preset learning method based on the image feature information.

8. In paragraph 6 The above point-based obstacle detection algorithm, A method for determining the risk of a crossing, comprising: selecting a point detection area based on point information included in the above sensing information, calculating point feature information using the point information included in the point detection area, and extracting information on the presence or absence of an obstacle if the point feature information satisfies a preset criterion.

9. In paragraph 1, The above preset risk criteria are: A method for judging the risk of a crossing, wherein the method is determined using three factors: the presence or absence of an obstacle, the direction of the barrier, and whether a green signal light is on, and the risk is judged to exist if at least one of the three factors does not satisfy a safety condition.

10. First body; A second body located below the first body; A sensor including a lidar sensor and one or more camera sensors for obtaining sensing information; and Including a control unit that determines whether a crossing is dangerous using the above sensing information, The above control unit, A receiving unit for receiving railway control information and the sensing information; A status detection unit that calculates status information of a railroad crossing safety facility based on at least one of the above railroad control information and the above sensing information; An obstacle detection unit that calculates obstacle presence / absence information according to one or more preset obstacle detection algorithms based on the above sensing information; and A level crossing risk judgment device further comprising a risk judgment unit that determines whether preset risk criteria are met based on the above-mentioned railroad crossing safety equipment status information and the above-mentioned obstacle presence / absence information.

11. In Article 10, The above railway control information is, A level crossing hazard judgment device including at least one of signal light lighting information, level crossing location information, locomotive identification information, and track section information.

12. In paragraph 10, The above railroad crossing safety equipment status information is: A level crossing hazard judgment device, wherein the signal light status information is calculated by the signal light on / off information included in the railway control information, including the signal light status information.

13. In paragraph 12, The above status detection unit, A crossing hazard judgment device that sets a traffic light area based on image information included in the sensing information using the above traffic light status detection algorithm, and evaluates whether a green traffic light is lit based on image information of the traffic light area to produce sensor-based traffic light status information.

14. In paragraph 10, The above status detection unit, A level crossing hazard judgment device that sets a circuit breaker area based on point information included in the sensing information using a circuit breaker status detection algorithm, calculates information on the presence or absence of points existing in the circuit breaker area, and classifies the circuit breaker direction based on the point presence or absence information to calculate the railroad crossing safety equipment status information.

15. In paragraph 10, Each of the above obstacle detection algorithms, A crossing hazard judgment device including at least one image-based obstacle detection algorithm that extracts obstacle presence information using image information detected based on the sensing information and at least one point-based obstacle detection algorithm that extracts obstacle presence information using point information detected based on the sensing information.

16. In Article 15 The above image-based obstacle detection algorithm, A crossing hazard judgment device that derives image feature information from the detected image information within a preset distance and extracts information on the presence or absence of an obstacle using a preset learning method based on the image feature information.

17. In Article 15 The above point-based obstacle detection algorithm, A crossing hazard judgment device that selects a point detection area based on point information included in the above sensing information, calculates point feature information using the point information included in the point detection area, and extracts obstacle presence / absence information when the point feature information satisfies a preset criterion.

18. In paragraph 10, The above preset risk criteria are: A device for judging the risk of a crossing, which determines whether there is a risk by using three factors: the presence or absence of an obstacle, the direction of the barrier, and whether a green signal light is on, and determines that there is a risk if at least one of the three factors does not satisfy a safety condition.

19. In paragraph 10, The first body and the second body are connected by one or more supports, The above lidar sensor is placed on the upper surface of the first main body, In order to rotate the lidar sensor, a rotational bracket is included between the lidar sensor and the upper surface of the first body, A first camera included in the one or more camera sensors is arranged on the lower surface of the first body, A crossing hazard judgment device in which the second camera and the third camera included in the one or more camera sensors are spaced apart from each other on the upper surface of the second body.

Images