Apparatus and method for controlling railway vehicle

Abstract

An embodiment can provide an apparatus and a method for controlling a railway vehicle, which receive railway control information, sensing information generated by a sensor, and location information calculate estimated travel route information of the railway vehicle on the basis of the railway control information and the location information, extract track area information according to a preset track area extraction algorithm on the basis of the sensing information, extract obstacle location information according to one or more preset object extraction algorithms on the basis of the sensing information, and determine whether the railway vehicle is in danger by using at least one of the estimated travel route information, the track area information and the obstacle location information.

Description

Railway vehicle control device and method

[0001] These embodiments relate to a railway vehicle control device and method for determining whether there is a danger due to a forward obstacle during autonomous driving.

[0002] Autonomous railway vehicles are a key technology that can significantly contribute to the prevention of safety accidents by automating railway vehicles for transporting molten iron TLC in steel mills, which were previously operated manually by engineers, through sensor-based control and control-based route input, thereby preventing drowsy driving and carelessness due to long-distance driving.

[0003] When operating a railway vehicle, a braking distance of at least 100 meters is required. Therefore, it is essential to detect obstacles ahead, both near and far, in advance and control the vehicle to ensure it stops when it encounters them.

[0004] Even if autonomous railway vehicles use detection and recognition technology that utilizes existing technology, if the actual driving track area is not accurately determined, obstacles incorrectly detected outside the track during the control process can cause the railway vehicle to suddenly stop, making autonomous driving difficult.

[0005] Therefore, there is a need for a control device and method technology for autonomous railway vehicles to prevent sudden stops of railway vehicles due to falsely detected obstacles and to eliminate threats.

[0006] The present embodiments can provide a railway vehicle control device that determines whether there is a risk due to a forward obstacle during autonomous driving.

[0007] In one aspect, the present embodiments may provide a railway vehicle control device including an information receiving unit that receives railway control information, sensing information generated from a sensor, and location information, a driving path prediction unit that calculates expected driving path information of a railway vehicle based on the railway control information and the location information, a track area extraction unit that extracts track area information according to a preset track area extraction algorithm based on the sensing information, an obstacle location extraction unit that extracts obstacle location information according to one or more preset object extraction algorithms based on the sensing information, and a danger determination unit that determines whether the railway vehicle is dangerous by using at least one of the expected driving path information, the track area information, and the obstacle location information.

[0008] In another aspect, the present embodiments may provide a railway vehicle control method including an information receiving step of receiving railway control information, sensing information generated from a sensor, and location information, a driving path prediction step of calculating expected driving path information of a railway vehicle based on the railway control information and the location information, a track area extraction step of extracting track area information according to a preset track area extraction algorithm based on the sensing information, an obstacle location extraction step of extracting obstacle location information according to one or more preset object extraction algorithms based on the sensing information, and a danger determination step of determining whether the railway vehicle is in danger using at least one of the expected driving path information, the track area information, and the obstacle location information.

[0009] According to the present embodiments, a railway vehicle control device and method for determining whether there is a danger due to a forward obstacle during autonomous driving can be provided.

[0010] FIG. 1 is a diagram illustrating an example configuration of a system that can be applied to one embodiment.

[0011] FIG. 2 is a drawing for explaining a railway vehicle control device according to one embodiment.

[0012] FIG. 3 is a diagram for explaining a process of extracting track area information according to one embodiment.

[0013] FIG. 4 is a diagram illustrating an image-based object extraction algorithm according to one embodiment.

[0014] FIG. 5 is a diagram for explaining a point-based object extraction algorithm according to one embodiment.

[0015] Figure 6 is a flowchart for explaining a railway vehicle control method according to one embodiment.

[0016] Hereinafter, some embodiments of the present disclosure will be described in detail with reference to exemplary drawings. When adding reference numerals to components in each drawing, identical components may have the same numerals as much as possible even if they are shown in different drawings. In addition, when describing the present embodiments, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the technical idea of ​​the present invention, the detailed description may be omitted. When "includes," "has," "consists of," etc. are used in this specification, other parts may be added unless "only" is used. When a component is expressed in the singular, it may include a case in which the plural is included unless specifically stated otherwise.

[0017] Additionally, terms such as first, second, A, B, (a), (b), etc. may be used to describe components of the present disclosure. These terms are only intended to distinguish the components from other components, and the nature, order, sequence, or number of the components are not limited by the terms.

[0018] In a description of the positional relationship of components, when it is described that two or more components are "connected," "combined," or "connected," it should be understood that the two or more components may be directly "connected," "combined," or "connected," but that the two or more components may also be further "interposed" with another component to be "connected," "combined," or "connected." Here, the other component may be included in one or more of the two or more components that are "connected," "combined," or "connected" to each other.

[0019] In the description of the temporal flow relationship related to components, operation methods, or manufacturing methods, for example, when the temporal or flow relationship is described as “after”, “following”, “next to”, “before”, etc., it may also include cases where it is not continuous, unless “immediately” or “directly” is used.

[0020] Meanwhile, when numerical values ​​or corresponding information (e.g., levels, etc.) for components are mentioned, even without separate explicit description, the numerical values ​​or corresponding information may be interpreted as including an error range that may occur due to various factors (e.g., process factors, internal or external impact, noise, etc.).

[0021] It will be understood by those skilled in the art that the terms “learning,” “learning,” and “algorithm” appearing throughout the detailed description and claims of the present disclosure are terms that refer to algorithms that perform or can perform machine learning or deep learning through procedural computing, and are not intended to refer to mental operations such as human educational activities or models to which mental operations are applied.

[0022] In the present disclosure, a railway vehicle refers to a moving vehicle that travels on a railway. For example, a railway vehicle may refer to a locomotive. A railway vehicle refers to a moving vehicle that travels along a railway using power, and there are no restrictions on the method of power supply. For example, a railway vehicle according to the present disclosure may be powered not only by an internal combustion engine but also by electricity.

[0023] FIG. 1 is a schematic diagram illustrating a railway vehicle control system according to one embodiment.

[0024] Referring to FIG. 1, one or more railway vehicle control devices (110) can be connected to a control server (100) via a network. The control server (100) receives information from the railway vehicle control devices (110) and performs a function of transmitting related railway control information to the railway vehicle control devices (110). In addition, the control server (100) can perform a function of controlling the operation of the railway vehicle control devices (110) by analyzing and generating information for operation control of the railway vehicle control devices (110) and transmitting the information to the railway vehicle control devices (110).

[0025] The railway vehicle control devices (110) can transmit information detected from various sensors configured in the railway vehicle control devices (110) while running on the track to the control server (100). In addition, the control server (100) can transmit various railway control information, such as railway vehicle identification information, track section information, track section-by-track route information, location information of other railway vehicles, and track section information on which track work is being performed, to the railway vehicle control devices (110). The railway control information is not limited and can include various types of information, and can be used in the same manner for the railway control information described below.

[0026] Information exchange between the railway vehicle control devices (110) and the control server (100) can be performed at preset intervals or when a specific event occurs, and when an emergency situation occurs in the railway vehicle control devices (110), relevant information can be immediately transmitted to the control server (100).

[0027] Wireless communication technologies can be applied to networks. Examples include mobile communication technologies, short-range communication technologies like WiFi and Bluetooth, and beacon technology. Two or more technologies can also be used in conjunction.

[0028] For example, the control server (100) may transmit downlink control information on a physical control channel to transmit information to railway vehicle control devices (110). The communication devices configured in the railway vehicle control devices (110) may specify frequency / time resources of downlink data information including railway control information based on the downlink control information and decode the corresponding frequency / time resources to obtain railway control information. If the railway control information is normally decoded, the communication devices configured in the railway vehicle control devices (110) transmit an Ack message to the control server (100) through the uplink control channel or the uplink data channel. If the Ack message is not received, the control server (100) performs a procedure for retransmitting the railway control information.

[0029] FIG. 2 is a drawing for explaining a railway vehicle control device according to one embodiment.

[0030] Referring to FIG. 2, a railway vehicle control device (110) may include an information receiving unit (210) that receives railway control information, sensing information generated from a sensor, and location information, a driving path prediction unit (220) that calculates expected driving path information of a railway vehicle based on the railway control information and the location information, a track area extraction unit (230) that extracts track area information according to a preset track area extraction algorithm based on the sensing information, an obstacle location extraction unit (240) that extracts obstacle location information according to one or more preset object extraction algorithms based on the sensing information, and a risk determination unit (250) that determines whether a railway vehicle is dangerous by using at least one of the expected driving path information, the track area information, and the obstacle location information.

[0031] The railway vehicle control device (110) may include an information receiving unit (210) that receives railway control information, sensing information generated from a sensor, and location information.

[0032] For example, the railway control information of the railway vehicle control device (110) may include at least one of the following information: identification information of the railway vehicle, track section information, route information for each track section, location information of other railway vehicles, and track section information on which track work is being performed.

[0033] For example, identification information of a railway vehicle may include number information for identifying the railway vehicle, type information of the railway vehicle, and color information of the railway vehicle. Track section information may include information about the track on which the railway vehicle will operate or the track on which the railway vehicle regularly operates. Route information for each track section may include one or more route information for each track section, information about whether the railway vehicle operates in one of the forward or reverse directions, and detour route information for the track section. Location information of other railway vehicles may be received from a control device of another railway vehicle through a control server. The reason why location information of other railway vehicles is necessary is because it may be preset to prevent the railway vehicle from operating on a route on which other railway vehicles operate. Information about a section undergoing track work may include information about a section undergoing repair work due to a breakdown or information about a section undergoing track change work. However, the railway control information is not limited to the present embodiment and may include various types of information.

[0034] For example, a railway vehicle control device (110) may include an information receiving unit (210) that receives sensing information and location information generated from a sensor.

[0035] The sensors described in this embodiment refer to various types of sensors installed in a railway vehicle, and there is no limit to the number of sensors. For example, the sensors may include sensors that detect the exterior of the railway vehicle, such as cameras, radar, lidar, and ultrasonic sensors. Alternatively, the sensors may include sensors installed inside the railway vehicle, such as speed sensors, motion detection sensors, acceleration sensors, and position sensors, that generate sensing information about the vehicle's movements.

[0036] In addition, as an example, the sensing information may include track information received from a camera sensor. As another example, the sensing information may include forward railway vehicle information and forward obstacle object information received from a camera sensor. As another example, the sensing information may include obstacle object information received from a lidar sensor. As another example, the sensing information may include location information received from GPS, navigation, etc. In addition, the sensing information may include various information received from sensors configured in the railway vehicle, such as side information, rear information, structure information, railway vehicle location information, and railway vehicle acceleration information.

[0037] The railway vehicle control device (110) may include a driving path prediction unit (220) that calculates expected driving path information of the railway vehicle based on railway control information and location information.

[0038] The predicted driving path described below refers to the path a railway vehicle should travel when operating with assistance or autonomous driving functions. Therefore, predicting a driving path refers to predicting the path a railway vehicle should travel when operating without and / or with assistance from a driver.

[0039] For example, the driving path prediction unit (220) can predict the expected driving path information of a railway vehicle by mapping location information to one or more pieces of information included in railway control information.

[0040] For example, the driving path prediction unit (220) can predict the expected driving path information of a railway vehicle by mapping location information to track section information among railway control information.

[0041] The coordinate systems of the track section information and the location information included in the railway control information may differ. Therefore, work may be required to synchronize the coordinate systems between the track section information and the location information. Accordingly, the driving path prediction unit (220) can map the location information to the received track section information and then calculate the expected driving path information with the coordinate system modified in advance. Alternatively, the driving path prediction unit (220) can map the location information to the received track section information and calculate the expected driving path information with the coordinate system modified in real time.

[0042] As another example, the driving path prediction unit (220) can calculate the expected driving path information of the railway vehicle by mapping the location information to the track section information and the track section information on which track work is being performed. According to the present disclosure, in order to determine the risk factor due to obstacles during autonomous driving of the railway vehicle, not only the track section information on which the railway vehicle can operate but also the information on the track section information on which the railway vehicle cannot operate may be required. Therefore, the driving path prediction unit (220) can calculate the expected driving path information of the railway vehicle by mapping the location information to the track section information and the track section information on which track work is being performed. In addition, as described above, since the information included in the railway control information has different location information and coordinate systems, an additional process of synchronizing the coordinate systems may be required. In addition, the aforementioned coordinate system may be set as a three-dimensional world coordinate system including the x-axis, y-axis, and z-axis, or a two-dimensional coordinate system including the x-axis and y-axis, or a BEV (Bird's Eyes View) coordinate system.

[0043] However, the driving prediction path information can be calculated and the coordinate system can be set according to various embodiments without being limited to the above-described embodiments.

[0044] The railway vehicle control device (110) may include a track area extraction unit (230) that extracts track area information according to a preset track area extraction algorithm based on sensing information.

[0045] The track area described below refers to the area to be extracted using a track area extraction algorithm. The track area extracted from image information can be used as a reference when assessing the presence of obstacles on the track.

[0046] For example, a track area extraction algorithm can select a track located in front of a railway vehicle based on sensing information, extract image information about the track, and extract track area information based on the image information.

[0047] Additionally, the track area extraction algorithm may include a high-dimensional algorithm that extracts one or more track feature information using a preset computational method based on image information, and extracts the track area information using a preset learning method based on the one or more extracted track feature information. The track area extraction algorithm according to the present embodiment will be described in detail later using FIG. 3.

[0048] The railway vehicle control device (110) may include an obstacle location extraction unit (240) that extracts obstacle location information according to one or more preset object extraction algorithms based on sensing information.

[0049] The obstacle locations described below refer to locations extracted by an object extraction algorithm. Obstacle locations extracted from image and point information can be used to assess the presence of obstacles on a track.

[0050] For example, each object extraction algorithm may include at least one image-based object extraction algorithm that extracts obstacle location information using image information detected based on sensing information, and one point-based object extraction algorithm that extracts obstacle location information using point information detected based on sensing information.

[0051] For example, an image-based object extraction algorithm can extract image information about an obstacle located within a preset distance from a railway vehicle, derive obstacle feature information based on the image information, and calculate a probability value based on the obstacle feature information to extract obstacle location information. The image-based object extraction algorithm will be described in detail later using FIG. 4.

[0052] For example, a point-based object extraction algorithm can select an obstacle located within a preset distance from a railway vehicle, extract point information about the obstacle, derive point feature information based on the point information, and then extract the obstacle location information using the point feature information. The point-based object extraction algorithm will be described in detail later using FIG. 5.

[0053] In addition to these, various object extraction algorithms can be configured. Alternatively, a mixture of object extraction algorithms can be configured.

[0054] The railway vehicle control device (110) may include a risk judgment unit (250) that determines whether the railway vehicle is at risk by using at least one of expected driving path information, track area information, and obstacle location information.

[0055] For example, the risk judgment unit (250) can use at least one of the expected driving path information, the track area information, and the obstacle location information to evaluate whether an obstacle exists within a certain offset range based on the track area included in the expected driving path, thereby judging whether the railway vehicle is at risk.

[0056] For example, the risk judgment unit (250) may additionally perform an operation of synchronizing the coordinate systems of a plurality of pieces of information in order to use the expected driving path information, track area information, and obstacle location information.

[0057] The track area information can be set as a two-dimensional coordinate system, and the obstacle location information extracted based on the image information and point information can be set as a mixture of two-dimensional and three-dimensional coordinate systems. The aforementioned driving prediction path information can be set as a three-dimensional coordinate system or various types of two-dimensional coordinate systems. However, the driving prediction path information can be set based on the synchronized coordinate system after synchronizing the set coordinate systems of the track area information and the obstacle location information. Therefore, this embodiment describes in detail the operation of synchronizing the coordinate systems of the track area information and the obstacle location information.

[0058] In this embodiment, the obstacle location information extracted based on point information is defined as “first obstacle location information.” The obstacle location information extracted based on image information is defined as “second obstacle location information.”

[0059] For example, the first obstacle location information may be set in a three-dimensional coordinate system, and the second obstacle location information and track area information may be pre-set in a two-dimensional coordinate system. In this case, there is a need to change the coordinate system of the first obstacle location information from a three-dimensional coordinate system to a two-dimensional coordinate system.

[0060] In addition, the coordinate system of the first obstacle location information can be changed to the same coordinate system as the coordinate system of the second obstacle location information and the track area information using the first parameter and the second parameter. The same coordinate system can be set as a three-dimensional coordinate system, a two-dimensional coordinate system, or a BEV's coordinate system. In this case, the first parameter can include a position difference between one or more sensors and a direction difference between one or more sensors. The second parameter can include a focal length, a principal point, and an asymmetry coefficient.

[0061] According to this embodiment, the risk assessment unit (250) can synchronize the coordinate systems of multiple pieces of information to utilize the expected driving path information, track area information, and obstacle location information. Furthermore, the risk assessment unit (250) can determine the risk of a railway vehicle based on the synchronized coordinate system. However, this embodiment is merely an example, and various other embodiments may be additionally included.

[0062] As another example, the risk assessment unit (250) can set a predetermined offset relative to the railway vehicle. The predetermined offset can be set by adding a predetermined radius forward or backward relative to the railway vehicle. The predetermined offset can be preset in various ways and can be variably changed depending on the speed of the railway vehicle.

[0063] Below, each operation performed by the aforementioned railway vehicle control device is exemplarily described with reference to drawings. The description of each operation described below is provided as an example to aid understanding and is not limited to the drawings or examples.

[0064] FIG. 3 is a diagram for explaining a process of extracting track area information according to one embodiment.

[0065] Referring to FIG. 3, the track area extraction algorithm (310) can extract one or more track feature information (312, 313) using a preset operation method (320, 321, 322) based on image information, and extract track area information using a preset learning method (330) based on the one or more extracted track feature information (312, 313).

[0066] The image information can extract one or more track feature information using a preset calculation method.

[0067] Referring to FIG. 3, one or more line feature information (312, 313) can be extracted using a preset operation method (320, 321, 322) based on image information. For example, image information can be extracted as first line feature information (312) through a first operation method (321). The first operation method (321) can extract image information as first line feature information (312), which is high-resolution information, using a ConvBlock in which a 3x3 Convolution Layer, a Batch Normalization Layer, and a ReLU activation function are arranged in that order. One or more ConvBlocks can be used, and there is no limitation on the number of ConvBlocks.

[0068] Additionally, image information can be extracted as second line feature information (313) through a second operation method (322). The second operation method (322) can extract the second line feature (313) by reducing the amount of information of image information using a 2x2 max pooling method.

[0069] The track area extraction algorithm (310) can extract track area information using a preset learning method (330) based on first track feature information (312) and second track feature information (313). The preset learning method (330) can include a first learning method, a second learning method, and a third learning method.

[0070] For example, through the first learning method, the first line feature information (312) can be copied to generate the first line area information. The second line feature information (313) can be generated as the second line area information by using the second learning method, but doubling the dimensionality of the information and reducing the number of channels by half through a transposed convolution layer. The third learning method can generate the line area information by combining the first line area information and the second line area information and then using the above-described ConvBlock.

[0071] Furthermore, this embodiment is a schematic description of a track area extraction algorithm. Therefore, one or more preset operation methods (320) and preset learning methods (330) may be used, and there are no restrictions on their order or number.

[0072] FIG. 4 is a diagram illustrating an image-based object extraction algorithm according to one embodiment.

[0073] Referring to FIG. 4, image information (400) regarding an obstacle located within a preset distance from a railway vehicle is extracted, and obstacle feature information regarding the image information (400) can be calculated through the Backbone (410). The extracted obstacle feature information needs to include high-dimensional information for object detection. Therefore, the obstacle feature information calculated through the Backbone (410) can include high-dimensional information for recognizing the shape, size, color, etc. of the object. Networks that can be used as the Backbone (410) include ResNet, Darknet, and CSPDarknet networks based on CNN (Convolutional Neural Networks). However, various networks can be used in addition to the network described in this embodiment.

[0074] Image-based object extraction algorithms can extract obstacle location information by calculating probability values ​​based on obstacle feature information.

[0075] For example, an image-based object extraction algorithm can extract obstacle location information by calculating probability values ​​using an FPN network (Feature Pyramid Network) and a PANet network (Path Aggregation Network) based on obstacle feature information. The FPN network (420) can be configured to calculate one or more probability values ​​(422, 423, 424) using obstacle feature information from one or more networks. Additionally, the PANet network can be configured to accurately detect the location and class of an object by integrating the calculated probability values ​​(422, 423, 424). However, the image-based object extraction algorithm is not limited to the above-described networks, and can extract obstacle location information by calculating probability values ​​based on obstacle feature information.

[0076] FIG. 5 is a diagram for explaining a point-based object extraction algorithm according to one embodiment.

[0077] Referring to FIG. 5, the point-based object extraction algorithm selects an obstacle located within a preset distance from a railway vehicle, extracts point information about the obstacle, calculates point feature information based on the extracted point information, and then extracts obstacle location information using the calculated point feature information.

[0078] For example, a point-based object extraction algorithm may perform a clustering process (520) on point information (510) about obstacles received via a lidar sensor. The clustering process (520) may include an algorithm that groups points of a point cloud with similar characteristics included in the point information (510). That is, the point information (510) may be converted into a clustered point group through the clustering process (520). For example, the described algorithm may use K-means, DBSCAN, and MeanShift algorithms. However, the present invention is not limited to this algorithm and various algorithms may be utilized.

[0079] In addition, the point-based object extraction algorithm can output point feature information by inputting a clustered point group into a feature extraction process (530). The feature extraction process (530) can output point feature information by extracting object characteristics from the clustered point group. The feature extraction process (530) can output point feature information such as the shape, size, direction, height, etc. of the object by using the distance, distribution, and density between points within the clustered point group. When performing the extraction process (530), the PointNet or PointNet++ algorithm can be used. However, the present invention is not limited to the present algorithm and various algorithms can be utilized.

[0080] Additionally, the point-based object extraction algorithm can identify and classify objects through a classification process (540) using point feature information. When the classification process (540) is performed, at least one or more of the PointNet, PointNet++, PointCNN, PointRCNN, KPConv, DGCNN, and FusionNet algorithms may be used. However, various algorithms may be utilized without being limited to the described algorithms.

[0081] In addition, the point-based object extraction algorithm can extract obstacle location information using an object detection process (550) based on point feature information on which a classification process (540) has been performed. The object detection process (550) can identify an actual object based on the point feature information on which the classification process has been performed. Through the object detection process (550), the location, direction, size, etc. of the object can be determined and the object can be distinguished from other objects. When the object detection process (550) is performed, at least one of the PointRCNN, KPConv, VoteNet, and FusionNet algorithms can be used. However, various algorithms can be utilized without being limited to the described algorithms.

[0082] Figure 6 is a flowchart for explaining a railway vehicle control method according to one embodiment.

[0083] Referring to FIG. 6, the railway vehicle control method may include an information receiving step (S610) of receiving railway control information, sensing information generated from a sensor, and location information, a driving path prediction step (S620) of calculating expected driving path information of a railway vehicle based on the railway control information and the location information, a track area extraction step (S630) of extracting track area information according to a preset track area extraction algorithm based on the sensing information, an obstacle location extraction step (S640) of extracting obstacle location information according to one or more preset object extraction algorithms based on the sensing information, and a danger determination step (S650) of determining whether the railway vehicle is dangerous by using at least one of the expected driving path information, the track area information, and the obstacle location information.

[0084] The railway vehicle control method may include an information receiving step of receiving railway control information, sensing information generated from a sensor, and location information. (S610)

[0085] For example, the railway control information of the railway vehicle control device may include at least one of the following information: identification information of the railway vehicle, track section information, route information for each track section, location information of other railway vehicles, and track section information on which track work is being performed.

[0086] For example, identification information of a railway vehicle may include number information for identifying the railway vehicle, type information of the railway vehicle, and color information of the railway vehicle. Track section information may include information about the track on which the railway vehicle will operate or the track on which the railway vehicle regularly operates. Route information for each track section may include one or more route information for each track section, information about whether the railway vehicle operates in one of the forward or reverse directions, and detour route information for the track section. Location information of other railway vehicles may be received from a control device of another railway vehicle through a control server. The reason why location information of other railway vehicles is necessary is because it may be preset to prevent the railway vehicle from operating on a route on which other railway vehicles operate. Information about a section undergoing track work may include information about a section undergoing repair work due to a breakdown or information about a section undergoing track change work. However, the railway control information is not limited to the present embodiment and may include various types of information.

[0087] The sensors described in this embodiment refer to various types of sensors installed in a railway vehicle, and there is no limit to the number of sensors. For example, the sensors may include sensors that detect the exterior of the railway vehicle, such as cameras, radar, lidar, and ultrasonic sensors. Alternatively, the sensors may include sensors installed inside the railway vehicle, such as speed sensors, motion detection sensors, acceleration sensors, and position sensors, that generate sensing information about the vehicle's movements.

[0088] In addition, as an example, the sensing information may include track information received from a camera sensor. As another example, the sensing information may include forward railway vehicle information and forward obstacle object information received from a camera sensor. As another example, the sensing information may include obstacle object information received from a lidar sensor. As another example, the sensing information may include location information received from GPS, navigation, etc. In addition, the sensing information may include various information received from sensors configured in the railway vehicle, such as side information, rear information, structure information, railway vehicle location information, and railway vehicle acceleration information.

[0089] The railway vehicle control device method may include a driving path prediction step of calculating expected driving path information of a railway vehicle based on railway control information and location information. (S620)

[0090] The predicted driving path described below refers to the path a railway vehicle should travel when operating with assistance or autonomous driving functions. Therefore, predicting a driving path refers to predicting the path a railway vehicle should travel when operating without and / or with assistance from a driver.

[0091] For example, the driving path prediction step can predict the expected driving path information of a railway vehicle by mapping location information to one or more pieces of information included in railway control information.

[0092] For example, the driving path prediction step can predict the expected driving path information of a railway vehicle by mapping location information to track section information among railway control information.

[0093] The coordinate systems of the track section information and the location information included in railway control information may differ. Therefore, synchronizing the coordinate systems between the track section information and the location information may be necessary. Therefore, the route prediction step can map the location information to the received track section information and then produce the predicted route information with the coordinate system modified in advance. Alternatively, the route prediction step can map the location information to the received track section information and produce the predicted route information with the coordinate system modified in real time.

[0094] As another example, the driving path prediction step may map location information to track section information and track section information on which track work is being performed to derive expected driving path information of the railway vehicle. According to the present disclosure, in order to determine risk factors due to obstacles during autonomous driving of a railway vehicle, not only track section information on which the railway vehicle can operate but also information on track sections on which the railway vehicle cannot operate may be required. Therefore, the driving path prediction step may map location information to track section information and track section information on which track work is being performed to derive expected driving path information of the railway vehicle. In addition, as described above, since the information included in the railway control information has different location information and coordinate systems, an additional process of synchronizing the coordinate systems may be required. In addition, the aforementioned coordinate system may be set as a three-dimensional world coordinate system including the x-axis, y-axis, and z-axis, or a two-dimensional coordinate system including the x-axis and y-axis, or a BEV (Bird's Eyes View) coordinate system.

[0095] However, the driving prediction path information can be calculated and the coordinate system can be set according to various embodiments without being limited to the above-described embodiments.

[0096]

[0097] The railway vehicle control method may include a track area extraction step of extracting track area information according to a preset track area extraction algorithm based on sensing information (S630).

[0098] The track area described below refers to the area to be extracted using a track area extraction algorithm. The track area extracted from image information can be used as a reference when assessing the presence of obstacles on the track.

[0099] For example, a track area extraction algorithm can select a track located in front of a railway vehicle based on sensing information, extract image information about the track, and extract track area information based on the image information.

[0100] In addition, the track area extraction algorithm may include a high-dimensional algorithm that extracts one or more track feature information using a preset operation method based on image information, and extracts the track area information using a preset learning method based on the one or more extracted track feature information.

[0101] The railway vehicle control method may include an obstacle location extraction step of extracting obstacle location information according to one or more preset object extraction algorithms based on sensing information. (S640)

[0102] The obstacle locations described below refer to locations extracted by an object extraction algorithm. Obstacle locations extracted from image and point information can be used to assess the presence of obstacles on a track.

[0103] For example, each object extraction algorithm may include at least one image-based object extraction algorithm that extracts obstacle location information using image information detected based on sensing information, and one point-based object extraction algorithm that extracts obstacle location information using point information detected based on sensing information.

[0104] For example, an image-based object extraction algorithm can extract image information about an obstacle located within a preset distance from a railway vehicle, derive obstacle feature information for the image information, and extract obstacle location information by calculating a probability value based on the obstacle feature information.

[0105] For example, a point-based object extraction algorithm can select an obstacle located within a preset distance from a railway vehicle, extract point information about the obstacle, derive point feature information based on the point information, and then extract the obstacle location information using the point feature information.

[0106] In addition to these, various object extraction algorithms can be configured. Alternatively, a mixture of object extraction algorithms can be configured.

[0107] The railway vehicle control method may include a risk determination step for determining whether the railway vehicle is in danger by using at least one of expected driving path information, track area information, and obstacle location information. (S650)

[0108] For example, the step of determining whether there is a risk can be used to determine whether there is a risk to the railway vehicle by evaluating whether there is an obstacle within a certain offset range based on the track area included in the expected driving path, using at least one of the expected driving path information, track area information, and obstacle location information.

[0109] For example, the risk determination step may additionally perform an operation of synchronizing the coordinate systems of multiple pieces of information in order to utilize the expected driving path information, track area information, and obstacle location information.

[0110] Track area information can be set in a two-dimensional coordinate system, and obstacle location information extracted based on image information and point information can be set in a mixed two-dimensional and three-dimensional coordinate system. The aforementioned predicted driving path information can be set in a three-dimensional coordinate system or various types of two-dimensional coordinate systems. However, the predicted driving path information can be set based on the synchronized coordinate system after synchronizing the set coordinate systems of the track area information and obstacle location information.

[0111] In this embodiment, the obstacle location information extracted based on point information is defined as “first obstacle location information.” The obstacle location information extracted based on image information is defined as “second obstacle location information.”

[0112] For example, the first obstacle location information may be set in a three-dimensional coordinate system, and the second obstacle location information and track area information may be pre-set in a two-dimensional coordinate system. In this case, there is a need to change the coordinate system of the first obstacle location information from a three-dimensional coordinate system to a two-dimensional coordinate system.

[0113] In addition, the coordinate system of the first obstacle location information can be changed to the same coordinate system as the coordinate system of the second obstacle location information and the track area information using the first parameter and the second parameter. The same coordinate system can be set as a three-dimensional coordinate system, a two-dimensional coordinate system, or a BEV's coordinate system. In this case, the first parameter can include a position difference between one or more sensors and a direction difference between one or more sensors. The second parameter can include a focal length, a principal point, and an asymmetry coefficient.

[0114] According to this embodiment, the risk assessment step may synchronize the coordinate systems of multiple pieces of information to utilize the expected driving path information, track area information, and obstacle location information. Furthermore, the risk assessment step may determine the risk of the railway vehicle based on the synchronized coordinate system. However, this embodiment is merely an example, and various other embodiments may be included.

[0115] As another example, the risk assessment step can set a certain offset relative to the train. This offset can be set by adding a certain radius forward or backward from the train. This offset can be preset in various ways and can be variably adjusted based on the train's speed.

[0116] As described above, by using the railway vehicle control device and method according to the present disclosure, a function can be provided to determine whether a railway vehicle in operation is in danger and to avoid danger.

[0117] The algorithms described above can be used in various ways depending on the operator's choice, in addition to the above-described model.

[0118] The railway vehicle control device (110) may be configured as a computing system or as a GPU server equipped with a GPU processor and general memory, but the present embodiments are not limited thereto.

[0119] The railway vehicle control device (110) may be implemented by a computing device including at least some of a processor, a memory, a user input device, and a presentation device. The memory is a medium that stores computer-readable software, applications, program modules, routines, instructions, and / or data, etc., which are coded to perform a specific task when executed by the processor. The processor can read and execute the computer-readable software, applications, program modules, routines, instructions, and / or data stored in the memory. The user input device may be a means for allowing a user to input a command to cause the processor to perform a specific task or to input data necessary for the execution of a specific task. The user input device may include a physical or virtual keyboard or keypad, key buttons, a mouse, a joystick, a trackball, a touch-sensitive input device, or a microphone. The presentation device may include a display, a printer, a speaker, or a vibration device.

[0120] Computing devices can include a variety of devices, including smartphones, tablets, laptops, desktops, servers, and clients. A computing device may be a single, standalone device, or it may include multiple computing devices operating in a distributed environment, each of which collaborates with another through a communications network.

[0121] In addition, the aforementioned railway vehicle control device (110) can be executed by a computing device having a processor and a memory storing computer-readable software, applications, program modules, routines, instructions, and / or data structures coded to perform an image classification method utilizing a deep learning model when executed by the processor.

[0122] The artificial intelligence model described in these embodiments may be a current or future machine learning model, such as a model that performs algorithm-based machine learning operating on the aforementioned computing device or a model that performs artificial neural network-based learning.

[0123] Models that perform algorithm-based machine learning can be classical machine learning models such as tree-based models, k-Nearest Neighbors, k-Means Clustering, Principal Component Analysis (PCA), and support vector machines (SVM).

[0124] A tree-based model can be, for example, a decision tree model, a regression model, or a random tree model.

[0125] Meanwhile, an artificial intelligence model can be an ensemble model that solves problems by training and combining multiple models rather than using just one trained model.

[0126] Ensemble models combine multiple individually trained models to prevent overfitting and improve generalization performance. Ensemble models can be helpful in improving performance when the performance of individual models is not sufficient.

[0127] Ensemble models can be broadly divided into voting and boosting methods.

[0128] Voting methods derive a final result through voting on the results generated by multiple models. Examples include bagging, which combines algorithms of the same type but trains them on different data sets, and voting, which combines different types of algorithms.

[0129] Boosting is a method of combining weak machine learning models to create a more accurate and powerful model. Boosting involves sequentially performing tasks on each weak machine learning model, with subsequent models exploring additional areas missed by the previous models. Examples of boosting methods include random forests, gradient boosting, and XGBoost (eXtra Gradient Boost).

[0130] An artificial neural network (ANN) is a machine learning algorithm that analyzes and learns complex data based on a large number of interconnected artificial neurons, mimicking the operating principles of the human brain. An ANN can be any type of ANN, including the multilayer perceptron (MLP), the most basic ANN structure consisting of an input layer, a hidden layer, and an output layer; a convolutional neural network (CNN), which performs convolution operations to extract image features and reduces dimensionality through pooling operations; and a recurrent neural network (RNN), an ANN structure used to process ordered data. These ANNs can be modified in various ways depending on the complexity and diversity of the data.

[0131] A model that has undergone learning based on an artificial neural network can also be an ensemble model that solves problems by learning multiple models and combining them rather than learning just one model.

[0132] Meanwhile, algorithm-based machine learning models and models trained using artificial neural networks can be used complementarily. For example, an algorithm-based machine learning model can use the results of an artificial neural network-based model, and vice versa. An ensemble model combining algorithm-based machine learning models and artificial neural network-based models can also be used.

[0133] The embodiments described above may be implemented through various means. For example, the embodiments may be implemented through hardware, firmware, software, or a combination thereof.

[0134] In the case of hardware implementation, the railway vehicle control method according to the present embodiments may be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), processors, controllers, microcontrollers, or microprocessors.

[0135] For example, the railway vehicle control method according to the embodiments can be implemented using an artificial intelligence semiconductor device in which neurons and synapses of a deep neural network are implemented using semiconductor devices. In this case, the semiconductor devices may be currently used semiconductor devices, such as SRAM, DRAM, NAND, etc., or may be next-generation semiconductor devices, such as RRAM, STT MRAM, PRAM, etc., or may be a combination thereof.

[0136] When implementing a railway vehicle control method according to embodiments using an artificial intelligence semiconductor device, the results (weights) of learning a deep learning model using software may be transferred to synapse-mimetic elements arranged in an array, or learning may be performed in the artificial intelligence semiconductor device.

[0137] When implemented using firmware or software, the railway vehicle control method according to the present embodiments may be implemented in the form of a device, procedure, or function that performs the functions or operations described above. The software code may be stored in a memory unit and executed by a processor. The memory unit may be located within or outside the processor and may exchange data with the processor using various known means.

[0138] Additionally, terms such as "system," "processor," "controller," "component," "module," "interface," "model," or "unit" as described above may generally refer to a computer-related entity, such as hardware, a combination of hardware and software, software, or software in execution. For example, the aforementioned components may be, but are not limited to, a process driven by a processor, a processor, a controller, a control processor, an object, a thread of execution, a program, and / or a computer. For example, both an application running on a controller or a processor and the controller or the processor may be components. One or more components may be within a process and / or thread of execution, and the components may be located on a single device (e.g., a system, a computing device, etc.) or distributed across two or more devices.

[0139] The above description is merely an illustrative example of the technical idea of ​​the present disclosure, and those skilled in the art to which the present disclosure pertains will appreciate that various modifications and variations can be made without departing from the essential characteristics of the technical idea of ​​the present disclosure. In addition, the present embodiments are not intended to limit the technical idea of ​​the present disclosure but rather to explain it, and therefore the scope of the technical idea of ​​the present disclosure is not limited by these embodiments. The scope of protection of the present disclosure should be interpreted by the claims below, and all technical ideas within a scope equivalent thereto should be interpreted as being included within the scope of the rights of the present disclosure.

[0140]

[0141] CROSS-REFERENCE TO RELATED APPLICATION

[0142] This patent application claims priority under 35 USC § 119(a) of Korean Patent Application No. 10-2023-0181613, filed in Korea on December 14, 2023, the entire contents of which are incorporated herein by reference. Furthermore, this patent application claims priority in countries other than the United States for the same reasons, the entire contents of which are incorporated herein by reference.

Claims

1. An information receiving unit that receives railway control information, sensing information generated from sensors, and location information; A driving path prediction unit that calculates expected driving path information of a railway vehicle based on the above railway control information and the above location information; A track area extraction unit that extracts track area information according to a preset track area extraction algorithm based on the above sensing information; An obstacle location extraction unit that extracts obstacle location information according to one or more preset object extraction algorithms based on the above sensing information; and A railway vehicle control device including a risk judgment unit that determines whether the railway vehicle is at risk by using at least one of the above-mentioned expected driving path information, rail area information, and obstacle location information.

2. In paragraph 1, The above railway control information is, A railway vehicle control device including at least one of the following information: identification information of a railway vehicle, rail section information, route information for each rail section, location information of other railway vehicles, and information on a rail section during which rail work is being performed.

3. In paragraph 2, The above driving path prediction section A railway vehicle control device that calculates expected driving path information of the railway vehicle by mapping the location information to one or more pieces of information included in the railway control information.

4. In paragraph 1, The above track area extraction algorithm is, A railway vehicle control device that selects a track located in front of the railway vehicle based on the sensing information, extracts image information about the track, and extracts track area information based on the image information.

5. In paragraph 4, The above track area extraction algorithm is, A railway vehicle control device that extracts one or more track feature information using a preset calculation method based on the image information, and extracts track area information using a preset learning method based on the one or more extracted track feature information.

6. In paragraph 1 Each of the above object extraction algorithms, A railway vehicle control device comprising at least one image-based object extraction algorithm that extracts obstacle location information using image information detected based on the sensing information and one point-based object extraction algorithm that extracts obstacle location information using point information detected based on the sensing information.

7. In paragraph 6 The above image-based object extraction algorithm, A railway vehicle control device that extracts image information about an obstacle located within a preset distance from the railway vehicle, calculates obstacle feature information for the image information, and extracts obstacle location information by calculating a probability value based on the obstacle feature information.

8. In paragraph 6 The above point-based object extraction algorithm, A railway vehicle control device that selects an obstacle located within a preset distance from the railway vehicle, extracts point information about the obstacle, calculates point feature information based on the point information, and then extracts obstacle location information using the point feature information.

9. In paragraph 1, The above risk judgment department is, A railway vehicle control device that uses at least one of the above-mentioned expected driving path information, rail area information, and obstacle location information to evaluate whether an obstacle exists within a certain offset range based on a rail area included in the expected driving path, thereby determining whether there is a danger to the railway vehicle.

10. Information receiving step for receiving railway control information, sensing information generated from sensors, and location information; A driving path prediction step for calculating expected driving path information of a railway vehicle based on the above railway control information and the above location information; A track area extraction step for extracting track area information according to a preset track area extraction algorithm based on the above sensing information; An obstacle location extraction step for extracting obstacle location information according to one or more preset object extraction algorithms based on the above sensing information; and A railway vehicle control method including a risk determination step of determining whether the railway vehicle is in danger by using at least one of the above-mentioned expected driving path information, rail area information, and obstacle location information.

11. In Article 10, The above railway control information is, A railway vehicle control method including at least one of the following information: identification information of a railway vehicle, track section information, route information for each track section, location information of another railway vehicle, and track section information on which track work is being performed.

12. In paragraph 11, The above driving path prediction step is A railway vehicle control method for calculating expected driving path information of the railway vehicle by mapping the location information to one or more pieces of information included in the railway control information.

13. In paragraph 1, The above track area extraction algorithm is, A railway vehicle control method comprising: selecting a track located in front of the railway vehicle based on the sensing information; extracting image information about the track; and extracting track area information based on the image information.

14. In paragraph 13, The above track area extraction algorithm is, A railway vehicle control method comprising: extracting one or more track feature information using a preset calculation method based on the image information; and extracting track area information using a preset learning method based on the one or more extracted track feature information.

15. In Article 10 Each of the above object extraction algorithms, A railway vehicle control method comprising at least one image-based object extraction algorithm that extracts obstacle location information using image information detected based on the sensing information and one point-based object extraction algorithm that extracts obstacle location information using point information detected based on the sensing information.

16. In Article 15 The above image-based object extraction algorithm, A railway vehicle control method comprising: extracting image information on an obstacle located within a preset distance from the railway vehicle, calculating obstacle feature information for the image information, and extracting obstacle location information by calculating a probability value based on the obstacle feature information.

17. In Article 15 The above point-based object extraction algorithm, A railway vehicle control method comprising: selecting an obstacle located within a preset distance from the railway vehicle, extracting point information about the obstacle, calculating point feature information based on the point information, and then extracting obstacle location information using the point feature information.

18. In paragraph 10, The above risk assessment steps are: A railway vehicle control method for determining whether the railway vehicle is at risk by evaluating whether an obstacle exists within a certain offset range based on a railway area included in the expected driving path using at least one of the above-mentioned expected driving path information, rail area information, and obstacle location information.

Images