Tram capable of providing driving information to a driver and passengers

Abstract

The present invention provides a tram capable of providing tram operation information that can improve the safety and reliability of tram operation by efficiently providing operation information necessary for a driver and passengers during tram operation, comprising a plurality of sensors, a control unit that generates a driver interface and a passenger interface based on sensor data acquired through the plurality of sensors, a first display unit that displays the driver interface, and a second display unit that displays the passenger interface, wherein the control unit generates tram operation information based on the sensor data and generates the driver interface and the passenger interface based on the tram operation information.

Description

Technology Field

[0001] Embodiments of the present invention relate to a tram capable of providing tram operation information to a driver and passengers, and more specifically, to a tram capable of providing tram operation information that can improve the safety and reliability of tram operation by efficiently providing necessary operation information to a driver and passengers during tram operation. Background Technology

[0002] With the recent expansion of tram adoption as an eco-friendly mode of transportation within cities, various technologies are being researched to ensure safe and efficient operation even in complex driving environments. In particular, the importance of operational information display technology—which provides real-time driving conditions and surrounding information inside the vehicle—is increasing in order to reduce the burden on drivers and enhance passenger confidence in the operation.

[0003] However, conventional tram operation information systems were limited to simply displaying information received from the Central Traffic Control (CTC) system, making it difficult to reflect real-time changes in the surrounding environment or vehicle operating status during actual operation. Furthermore, because they were based on a single display structure designed solely for the driver, passengers found it difficult to intuitively perceive the operating situation, leading to increased anxiety and decreased service satisfaction.

[0004] To address these issues, some systems have attempted to provide operational information by integrating external sensor data or vehicle control data. However, due to limitations in data integration and visualization technologies, it was difficult to provide drivers and passengers with necessary information tailored to the specific situation, and the speed and accuracy of information delivery were also compromised. Therefore, new tram technology is required that can integrate various data acquired during operation to provide intuitive and highly reliable operational information to drivers and passengers. The problem to be solved

[0005] The present invention aims to solve various problems, including those mentioned above, and specifically, to provide a tram capable of providing operational information that can improve the safety and reliability of tram operation by efficiently providing necessary operational information to the driver and passengers during tram operation. However, this objective is exemplary and does not limit the scope of the present invention. means of solving the problem

[0006] To solve the aforementioned problem, a tram capable of providing tram operation information according to one embodiment of the present invention comprises a plurality of sensors, a control unit that generates an engine user interface and a passenger interface based on sensor data obtained through the plurality of sensors, a first display unit that displays the engine user interface, and a second display unit that displays the passenger interface. The control unit generates tram operation information based on the sensor data and can generate the engine user interface and the passenger interface based on the tram operation information.

[0007] The above-described operator interface includes a first information display area that displays information necessary for the operator's tram operation among the tram operation information, and a first image display area arranged around the first information display area. The first information display area may include a first-1 information display area that displays sensor status information including the operating status of each of the plurality of sensors, a first-2 information display area that displays tram signal information including at least one of the current signal, next signal, and remaining time of a tram-dedicated signal, a first-3 information display area that displays operation status information including at least one of the tram's operating section, remaining distance, and target speed, a first-4 information display area that displays operation control information including at least one of the tram's current speed, braking status, and speed limit, and a first-5 information display area that displays operation condition information including at least one of the tram's power supply status, battery charge status, and door opening / closing status.

[0008] The first image display area may include a first-1 image display area representing object recognition information based on the front image of the tram, a first-2 image display area representing object recognition information based on the left image of the tram, and a first-3 image display area representing object recognition information based on the right image of the tram.

[0009] The above passenger interface includes a second information display area that displays guidance information to be provided to passengers among the tram operation information, and a second image display area disposed around the second information display area. The second information display area may include a 2-1 information display area that displays operation control information including at least one of the current speed, braking status, and speed limit of the tram; a 2-2 information display area that displays tram signal information including at least one of the current signal, next signal, and remaining time of a tram-dedicated signal; and a 2-3 information display area that displays station information including at least one of the name of the arrival station, the name of the next station, and the remaining distance.

[0010] The sensor status information is information obtained from the plurality of sensors, the tram signal information is information generated based on signal data obtained from the tram-dedicated signal device, the operation status information is information generated based on preset target path data, the operation control information is information generated based on the driving status data of the tram, and the operation condition information may be information generated based on the vehicle status data of the tram.

[0011] The plurality of sensors includes a front camera, a left camera, and a right camera, and the front image is obtained from the front camera positioned to photograph the front of the tram, the left image is obtained from the left camera positioned to photograph the left side of the tram, and the right image can be obtained from the right camera positioned to photograph the right side of the tram.

[0012] The first information display area may be positioned on the left side of the first image display area, the first-1 image display area may be positioned on the upper side of the first image display area, the first-2 image display area may be positioned on the lower left side of the first image display area, and the first-3 image display area may be positioned on the lower right side of the first image display area.

[0013] The above tram driving control information is information generated based on the driving status information of the tram, the above tram signal information is information generated based on signal data obtained from a signal device, and the above station guidance information may be information generated based on preset station data.

[0014] The second information display area may be positioned vertically on the left side of the second image display area, the second-1 information display area may be positioned on the upper side of the second information display area, the second-2 information display area may be positioned in the middle of the second information display area, and the second-3 information display area may be positioned on the lower side of the second information display area.

[0015] The second image display area above may display a three-dimensional virtual screen of the surrounding image of the tram. Effects of the invention

[0016] According to one embodiment of the present invention as described above, a tram capable of providing tram operation information can be implemented, which can improve the safety and reliability of tram operation by efficiently providing necessary operation information to the driver and passengers during tram operation. Of course, the scope of the present invention is not limited by this effect. Brief explanation of the drawing

[0017] The above-described features of the present invention will be more clearly understood from the following detailed description of exemplary and non-limiting embodiments with reference to the accompanying drawings. FIG. 1 is a conceptual diagram showing a tram capable of providing tram operation information according to the present invention. Figure 2 is a drawing showing the detailed configuration of the control unit illustrated in Figure 1. FIG. 3 is a drawing illustrating an example of an institutional user interface displayed on the first display unit illustrated in FIG. 1. FIG. 4 is a drawing illustrating an example of a passenger interface displayed on the second display unit shown in FIG. 1. Specific details for implementing the invention

[0018] Embodiments of the present invention are described below with reference to the attached drawings so that those skilled in the art can easily implement the invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0019] In describing the embodiments disclosed in this specification, if it is determined that a detailed description of related prior art could obscure the essence of the embodiments disclosed in this specification, such detailed description is omitted.

[0020] Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but said components are not limited by said terms. These terms are used solely for the purpose of distinguishing one component from another.

[0021] Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0022] In this application, terms such as “comprising” or “having” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0023] The attached drawings are intended only to facilitate understanding of the embodiments disclosed in this specification, and the technical concept disclosed in this specification is not limited by the attached drawings; it should be understood that all modifications, equivalents, and substitutions included within the concept and technical scope of the present invention are included.

[0024] Throughout the specification, when a part is described as being “connected” to another part, this includes not only cases where they are “directly connected” but also cases where they are “electrically connected” with other elements interposed between them. Furthermore, when a part is described as “comprising” a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0025] In this specification, the term “and / or” includes any and all combinations of one or more of the associated listed items. For example, the expression “A and / or B” indicates A, B, or A and B. Expressions such as “at least one” may be used to refer to one or more of a plurality of components. For example, expressions such as “at least one of a, b, and c” or “at least one selected from the group consisting of a, b, and c” may indicate “a”, “b”, “c”, “a, b”, “b, c”, “a, c”, or “a, b, c”.

[0026] In this specification, terms such as "substantially," "approximately," and similar terms are used as terms of approximation rather than terms of degree, and may be used to describe inherent variations in measured or calculated values ​​that a person skilled in the art can recognize. For example, terms such as "may" or "may" may be used to mean "one or more embodiments disclosed in this specification."

[0027] In this specification, the term "part" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Additionally, one unit may be realized using two or more pieces of hardware, and two or more units may be realized by one piece of hardware. Meanwhile, "part" is not limited to software or hardware, and "part" may be configured to reside in an addressable storage medium or configured to run on one or more processors. Accordingly, as an example, "part" includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and "parts" may be combined into a smaller number of components and "parts" or further separated into additional components and "parts." In addition, the components and '~parts' may be implemented to regenerate one or more CPUs within the device.

[0028] Hereinafter, based on the above-described contents, a tram capable of providing tram operation information according to a preferred embodiment of the present specification will be described in detail as follows.

[0029] A tram is a vehicle in the form of a streetcar or light rail that runs along a track and can operate using external power or its own battery power as a power source.

[0030] FIG. 1 is a conceptual diagram showing a tram (hereinafter referred to as 'tram') capable of providing tram operation information according to the present invention.

[0031] As illustrated in FIG. 1, a tram (10) according to one embodiment of the present invention may include a driver's cabin (200), a passenger cabin (300), a drive unit (400), and a battery unit (500), and may include a plurality of sensors (100) for recognizing the external environment of the tram (10).

[0032] A plurality of sensors (100) are for acquiring data about the driving environment from the outside or receiving signal information while the tram (10) is driving, and may be placed outside or inside the tram (10). The plurality of sensors (100) may include, but are not limited to, sensors such as a camera (110), LiDAR (Light Detection And Ranging) (120), RADAR (Radio Detection And Ranging) (130), and a position recognition sensor (140), as well as a signal information receiver (150) for receiving signal information from the outside, such as a tram-specific signal light.

[0033] The driver's room (200) is a space where a driver can control the operation of the tram (10), and a control unit (210) and a first display unit (220) may be placed therein. The first display unit (220) can provide the driver with operation information of the tram (10) by displaying a driver interface.

[0034] The passenger cabin (300) is a space where a passenger boards, and a second display unit (310) may be installed therein. The second display unit (310) can provide the passenger with the operating status by displaying a passenger interface.

[0035] The drive unit (400) is positioned at the bottom of the tram (10) or the like to propel the tram (10), and may include a motor, a reduction gear, wheels, etc. The drive unit (400) can generate a driving force or a braking force according to a control signal from the control unit (210) to control the driving speed of the tram (10) and enable stable driving of the tram (10). The drive unit (400) can provide driving state data to the control unit (210), including the rotational speed of the tram (10)'s motor, driving torque, braking pressure, wheel rotation speed, input status of the accelerator pedal and brake pedal, and measurement values ​​of sensors related to the driving control of the tram (10) (e.g., acceleration sensor, braking pressure sensor, etc.).

[0036] The battery unit (500) is for storing and supplying power required for driving the tram (10), and can provide vehicle state data including the tram (10) battery voltage, current, output power, and battery state of charge (SOC) to the control unit (210).

[0037] As explained above, the tram (10) illustrated in FIG. 1 can be configured so that each component is interconnected to ensure smooth operation of the tram (10).

[0038] Hereinafter, the specific operation of the tram (10) of the present invention will be described in detail, focusing on a plurality of sensors (100), a control unit (210), a first display unit (220), and a second display unit (310).

[0039] Multiple sensors (100) can acquire various sensor data and provide it to the control unit (210).

[0040] In detail, the camera (110) is for recognizing the driving environment of the tram (10) and can capture the driving path and surrounding environment of the tram (10). The captured image may include tracks, lanes, intersections, roads, etc., and external objects such as pedestrians and vehicles around the tram (10).

[0041] The camera (110) may include a front camera, a left camera, and a right camera. The front camera is positioned to photograph the front of the tram (10) to obtain a front image. The left camera is positioned to photograph the left rear side of the tram (10) to obtain a left image. The right camera is positioned to photograph the right rear side of the tram (10) to obtain a right image.

[0042] The image acquired by the camera (110) is transmitted to the control unit (210) and can be used to generate object recognition information.

[0043] The lidar (120) is positioned at the front of the tram (10) to acquire distance data to obstacles present on the driving path. The distance data acquired by the lidar (120) is provided to the control unit (210) and can be used to generate object recognition information.

[0044] The radar (130) is positioned at the front of the tram (10) and can acquire speed data by detecting the distance, speed, etc. of objects existing around the tram (10) using reflected signals of electromagnetic waves. The radar (130) can reliably detect objects even at night or in environments with limited visibility, thereby complementing the recognition functions of the camera (110) and lidar (120).

[0045] The position recognition sensor (140) is for recognizing the position and driving section of the tram (10) and may be composed of a GNSS (Global Navigation Satellite System) sensor and an Inertial Navigation System (INS). The position recognition sensor (140) can obtain position data by calculating the position coordinates, distance traveled, and driving direction of the tram (10), and in sections where the GNSS signal is unstable, it can provide corrected position data to the control unit (210) through the INS.

[0046] The signal information receiver (150) is positioned inside the driver's cabin (200) of the tram (10) and is intended to communicate with a tram-specific signal or an external control system. For example, when the tram (10) approaches a specific section, the signal information receiver (150) can communicate with the tram-specific signal in that section to acquire signal data and provide it to the control unit (210). The signal information receiver (150) can receive signal data using V2X (Vehicle-to-Everything) or LTE-based wireless communication.

[0047] The control unit (210) can generate an interface for the engine and an interface for passengers based on sensor data obtained through a plurality of sensors (100).

[0048] More specifically, the control unit (210) can generate tram operation information based on the sensor data and generate the engine user interface and passenger interface based on the tram operation information.

[0049] The control unit (210) can generate the above-mentioned interface for the operator and the interface for the passenger by utilizing information provided from an external server, such as a central control server, in addition to the tram operation information generated based on sensor data.

[0050] Sensor data may include images acquired by a camera (110), distance data acquired by a lidar (120), speed data acquired by a radar (130), location data acquired by a position recognition sensor (140), and signal data acquired by a signal information receiver (150).

[0051] The tram operation information is integrated operation status information of the tram (10) generated by the control unit (210) analyzing sensor data obtained from a plurality of sensors (100), and may include sensor status information, tram signal information, operation situation information, operation control information, operation condition information, station information, etc.

[0052] Figure 2 is a drawing showing the detailed configuration of the control unit illustrated in Figure 1.

[0053] As shown in FIG. 2, the control unit (210) may include a processor (211), memory (212), and a communication module (213).

[0054] The processor (211) can control other configurations by executing instructions stored in memory (212). The processor (211) can execute instructions stored in memory (212).

[0055] A processor (211) is a component capable of performing calculations and controlling other devices. It may primarily refer to a central processing unit (CPU), an application processor (AP), a graphics processing unit (GPU), etc. Additionally, a CPU, AP, or GPU may include one or more cores within it, and the CPU, AP, or GPU may operate using operating voltage and clock signals. However, while a CPU or AP may consist of a few cores optimized for serial processing, a GPU may consist of thousands of smaller and more efficient cores designed for parallel processing.

[0056] The processor (211) can provide or process appropriate information or functions to the user by processing signals, data, information, etc. that are input or output through the components described above, or by running an application stored in memory (212).

[0057] The memory (212) stores data that supports various functions of the computing device. The memory (212) can store a number of applications (application programs or applications) running on the computing device, data for the operation of the computing device, and instructions. At least some of these applications may be downloaded from an external server via wireless communication. Additionally, the applications may be stored in the memory (212), installed on the computing device, and driven by the processor (211) to perform the operation (or function) of the computing device.

[0058] The memory (212) may include at least one type of storage medium among flash memory type, hard disk type, SSD type (Solid State Disk type), SSD type (Silicon Disk Drive type), multimedia card micro type, card type memory (e.g., SD or XD memory, etc.), RAM (random access memory; RAM), SRAM (static random access memory), ROM (read-only memory; ROM), EEPROM (electrically erasable programmable read-only memory), PROM (programmable read-only memory), magnetic memory, magnetic disk, and optical disk. Additionally, the memory (212) may include web storage that performs storage functions over the internet.

[0059] The communication module (213) performs transmission and reception of information with a base station or a camera including communication functions through an antenna. The communication module (213) may include a modulation unit, a demodulation unit, a signal processing unit, etc. Additionally, the communication module (213) may perform wired communication functions.

[0060] Wireless communication can refer to communication using communication facilities installed by telecommunication companies and a wireless communication network that uses the frequency of said communication facilities. In this case, the communication module (230) can be used in various wireless communication systems such as CDMA (code division multiple access), FDMA (frequency division multiple access), TDMA (time division multiple access), OFDMA (orthogonal frequency division multiple access), SC-FDMA (single carrier frequency division multiple access), etc. Furthermore, the communication module (213) can also be used in 3GPP (3rd generation partnership project) LTE (long term evolution), etc. In addition, not only 5G communication currently being commercialized but also 6G, which is scheduled for future commercialization, can be used. However, the present specification can utilize existing communication networks without being restricted to such wireless communication methods.

[0061] In addition, short-range communication technologies such as Bluetooth, BLE (Bluetooth Low Energy), Beacon, RFID (Radio Frequency Identification), NFC (Near Field Communication), Infrared Data Association (IrDA), UWB (Ultra Wideband), and ZigBee can be used.

[0062] The control unit (210) generates tram operation information by calculating sensor data using a processor (211), and can generate an interface for the driver and an interface for the passenger, respectively, based on the generated tram operation information.

[0063] When creating an interface for the operator, the control unit (210) can be configured to include information necessary for the operator to operate the tram (e.g., sensor status information, tram signal information, operation control information, etc.).

[0064] The control unit (210) can be configured to include guidance information (e.g., tram signal information, station information, etc.) to enable passengers to recognize the operating situation or to provide convenience of use when creating a passenger interface.

[0065] The control unit (210) can transmit the engine user interface to the first display unit (220) and the passenger user interface to the second display unit (310) respectively through the communication module (213), and control the engine user interface to be visually displayed on the first display unit (220) and the passenger user interface to be visually displayed on the second display unit (310).

[0066] The first display unit (220) can display the institution user interface transmitted from the control unit (210).

[0067] FIG. 3 is a drawing illustrating an example of an institutional user interface displayed on the first display unit illustrated in FIG. 1.

[0068] As illustrated in FIG. 3, a driver interface may be displayed on the first display unit (220). The driver interface may include a first information display area (2210) that displays information necessary for the driver's tram operation among the tram operation information, and a first image display area (2220) placed around the first information display area (2210).

[0069] In detail, the first information display area (2210) may be placed on the left side of the first image display area (2220).

[0070] The first information display area (2210) may include a first-1 information display area (2211), a first-2 information display area (2212), a first-3 information display area (2213), a first-4 information display area (2214), and a first-5 information display area (2215).

[0071] The first information display area (2211) can display sensor status information including the operating status of each of the plurality of sensors (100).

[0072] The sensor status information may be information obtained from a plurality of sensors (100).

[0073] In detail, the control unit (210) can monitor sensor data received from each of the plurality of sensors (100) to check whether the data is updated normally within a preset update time. The control unit (210) can determine that the sensor is in a normal operating state if the sensor data is updated normally, and determine that the sensor is in an abnormal operating state if the sensor data is not updated continuously for more than a certain number of times.

[0074] The judgment result of the control unit (210) is reflected in the institution user interface, so that the operating status of each sensor can be visually distinguished and displayed in the first-1 information display area (2211).

[0075] In detail, the operating status of each of the multiple sensors (100), such as the camera (110), lidar (120), radar (130), position recognition sensor (140), and signal information receiver (150), can be displayed in the first-1 information display area (2211) by distinguishing them by color. For example, a sensor determined to be in a normal operating state can be displayed in green, and a sensor determined to be in an abnormal operating state can be displayed in red. Through this, the train driver can intuitively recognize the operating status of each sensor.

[0076] The first-second information display area (2212) may display tram signal information including at least one of the current signal, next signal, and remaining time of the tram-only signal.

[0077] The tram signal information may be information generated based on signal data acquired by a signal information receiver (150) from the tram-dedicated signal device closest to the location of the tram (10).

[0078] For example, the first and second information display areas (2212) may display the current display status of the tram-only signal (e.g., stop signal, go signal, etc.), the expected signal status of the next section, and the remaining time until the signal changes to the next signal.

[0079] The first-third information display area (2213) may display operation status information including at least one of the operating section of the tram (10), the remaining distance to the next station, and the target speed.

[0080] The operation status information may be information generated based on the pre-set target route data of the tram (10).

[0081] The target path data may be data containing all coordinate values ​​for the target path that the tram (10) must travel.

[0082] Target route data may be provided to the control unit (210) from the central control server of the tram (10) or may be stored in advance in the memory of the tram (10). The memory may be the memory (212) of the control unit (210).

[0083] In detail, the control unit (210) can calculate the operating section of the tram (10), the remaining distance to the next station, and the target speed by comparing the target path data and the location data obtained through the location recognition sensor (140).

[0084] The operating section is a route section between the stations where the tram (10) is currently traveling, and may refer to a corresponding travel path section between the previous stop and the current stop. For example, if the tram (10) departs from stop A and travels toward stop B, the operating section may be “Stop A → Stop B”.

[0085] The remaining distance may mean the distance remaining from the current location of the tram (10) to the station where the tram (10) is scheduled to stop soon, that is, the current station.

[0086] The target speed may refer to a speed value calculated by the control unit (210) by comprehensively considering the pre-set target route data and the speed limit of the corresponding section to ensure the driving efficiency and punctuality of the tram (10).

[0087] The operation status information calculated by the control unit (210) can be reflected in the engine user interface and displayed in the first-third information display area (2213).

[0088] The first-fourth information display area (2214) may display operation control information including at least one of the current speed, braking status, and speed limit of the tram (10).

[0089] The operation control information may be information generated based on the driving state data of the tram (10).

[0090] The driving state data may include the rotational speed of the motor, driving torque, braking pressure, wheel rotation speed, input status of the accelerator pedal and brake pedal, and measurement values ​​of sensors related to the driving control of the tram (10) (e.g., acceleration sensor, braking pressure sensor, etc.). The driving state data may be data provided to the control unit (210) from the driving unit (400).

[0091] The control unit (210) can analyze driving state data to derive the current speed and braking state of the tram (10) and generate driving control information by referring to the speed limit set for each driving section.

[0092] The speed limit for each operating section may be provided to the control unit (210) from the central control server of the tram (10) or may be pre-stored in the memory (212) of the control unit (210).

[0093] The control unit (210) can reflect the speed limit corresponding to the current operating section of the tram (10) in the operation control information. The generated operation control information can be reflected in the engine user interface and displayed in the first-fourth information display area (2214).

[0094] For example, in the first-fourth information display area (2214), the current speed of the tram (10) can be displayed as a graphic in the form of a speedometer, and the braking status can be displayed as a color change (e.g., green for normal driving, yellow for deceleration, red for emergency braking).

[0095] The first-fifth information display area (2215) can display operating condition information including at least one of the power supply status, battery charge status, and door opening / closing status of the tram (10).

[0096] The operating condition information may be information generated based on the vehicle status data of the tram (10).

[0097] Vehicle status data may include data indicating the propulsion power supply method of the tram (10), battery level, hydrogen storage amount, and whether the doors are open or closed.

[0098] Vehicle status data may be information provided to the control unit (210) from the door control sensor of the battery unit (500) or the tram (10).

[0099] For example, in the first-fifth information display area (2215), the power source currently in use among overhead power, battery power, or hydrogen fuel power may be distinguished and displayed as an icon or color according to the power supply method of the tram (10). In addition, the battery charge status may be displayed in the form of a State of Charge (SOC) value or a charge gauge, and the door opening / closing status may be distinguished and displayed by a color change, such as red when the door is open and green when it is closed.

[0100] The first image display area (2220) may include a first-1 image display area (2221), a first-2 image display area (2222), and a first-3 image display area (2223).

[0101] The first-1 image display area (2221) may be positioned above the first image display area (2220), the first-2 image display area (2222) may be positioned at the lower left side of the first image display area (2220), and the first-3 image display area (2223) may be positioned at the lower right side of the first image display area (2220).

[0102] The first-1 image display area (2221) can display object recognition information based on the front image of the tram (10).

[0103] The front view can be obtained from a front camera positioned to photograph the front of the tram (10).

[0104] Object recognition information refers to information generated by detecting objects (e.g., pedestrians, vehicles, bicycles, motorcycles, static obstacles, etc.) existing around the tram (10) based on images acquired from the camera (110) and sensor data acquired from the lidar (120) and radar (130), and analyzing the location, distance, direction of movement, and relative speed of the objects.

[0105] The object recognition information based on the front image can recognize objects existing on the driving path of the tram (10) using the front image obtained from the front camera, and can visually display the location and type of the recognized objects within the image.

[0106] The first-second image display area (2222) can display object recognition information based on the left image of the tram (10).

[0107] The left image can be obtained from the left camera positioned to photograph the left side of the tram (10).

[0108] The object recognition information based on the left image may include location and movement direction information of objects detected within the left side area of ​​the tram (10), such as pedestrians on the left, adjacent vehicles, and vehicles that are stopped.

[0109] The control unit (210) can generate object recognition information based on the left image by fusing the image obtained from the left camera and sensor data obtained from the lidar (120) to determine the relative distance and approach status of the object.

[0110] The generated object recognition information can be displayed in the first-2 image display area (2222), and objects with a high risk of collision can have their outlines highlighted in red or be displayed with a warning icon within the image.

[0111] The first-third image display area (2223) can display object recognition information based on the right image of the tram (10).

[0112] The right side of the image can be obtained from the right camera positioned to photograph the right side of the tram (10).

[0113] Object recognition information based on the right image can be generated in the same way as the object recognition information based on the left image above.

[0114] That is, the control unit (210) can generate right-image-based object recognition information by fusing the image acquired from the right camera and the sensor data acquired from the lidar (120) to determine the location, relative distance, and approach status of an object detected within the right side area of ​​the tram (10).

[0115] The generated object recognition information can be displayed in the first-third image display area (2223), and objects with a high risk of collision can have their outlines highlighted in red or be displayed with a warning icon within the image.

[0116] The second display unit (310) can display the passenger interface transmitted from the control unit (210).

[0117] FIG. 4 is a drawing illustrating an example of a passenger interface displayed on the second display unit shown in FIG. 1.

[0118] As illustrated in FIG. 4, a passenger interface may be displayed on the second display unit (310). The passenger interface may include a second information display area (3110) that displays guidance information to be provided to passengers among the tram operation information, and a second image display area (3120) placed around the second information display area.

[0119] In detail, the second information display area (3110) can be placed on the left side of the second image display area (3120).

[0120] The second information display area (3110) may include a second-1 information display area (3111), a second-2 information display area (3112), and a second-3 information display area (3113).

[0121] The second-1 information display area (3111) may be placed on the upper part of the second information display area (3110), the second-2 information display area (3112) may be placed on the middle part of the second information display area (3110), and the second-3 information display area (3113) may be placed on the lower part of the second information display area (3110).

[0122] The second-1 information display area (3111) can display operation control information including at least one of the tram's current speed, braking status, and speed limit.

[0123] The operation control information can be generated based on the driving state data of the tram (10) in the same way as the operation control information of the engine user interface described above.

[0124] The generated operation control information can be reflected in the passenger interface and displayed in the 2-1 information display area (3111).

[0125] In the second-1 information display area (3111), the current speed of the tram (10) can be displayed as a graphic in the form of a speedometer, and the braking status can be distinguished and displayed by a color change (e.g., green for normal operation, yellow for deceleration, red for emergency braking). In addition, the speed limit corresponding to the current operating section can be displayed together in the form of a separate numerical value or gauge.

[0126] The second information display area (3112) can display tram signal information including at least one of the current signal, next signal, and remaining time of the tram-only signal.

[0127] The above tram signal information can be generated based on signal data obtained by a signal information receiver (150) from a tram-dedicated signal device closest to the location of the tram (10), in the same manner as the tram signal information of the first and second information display areas (2212) described above.

[0128] For example, the current display status of the tram-only signal (e.g., stop signal, proceed signal, etc.) and the remaining time until the next signal is switched may be displayed together in the second information display area (3112), and the change in the signal status may be visually distinguished and displayed in the form of colors or icons.

[0129] The second-third information display area (3113) may display station information including at least one of the current station name, the next station name, and the remaining distance.

[0130] Station information may be information generated based on pre-configured station data.

[0131] Station data may include the name, location, distance between adjacent stations, stopping order, etc. for each station located on the route of the tram (10).

[0132] Station data may be provided to the control unit (210) from the central control server of the tram (10) or may be pre-stored in the memory of the tram (10). The memory may be the memory (212) of the control unit (210).

[0133] In detail, the control unit (210) can generate station information including the current station name, the next station name, and the remaining distance to the current station by comparing station data and location data obtained through the location recognition sensor (140).

[0134] The current station name refers to the name of the station where the tram (10) is scheduled to stop soon, and the next station name refers to the name of the station where the tram (10) is scheduled to stop in the order following the current station. The remaining distance to the current station refers to the distance from the current position of the tram (10) to the stopping point of the current station, and can be calculated by comparing the station coordinate values ​​included in the station data with the location data obtained by the control unit (210) through the location recognition sensor (140).

[0135] In the second-third information display area (3113), the name of the current station and the name of the next station are displayed in text form, and the remaining distance from the current location to the current station may be displayed together in numerical or graphic form (e.g., in the form of a progress bar).

[0136] For example, if the tram (10) is currently traveling toward station A and the next scheduled stop is station B, the second-third information display area (3113) may display 'This stop: Station A', 'Next stop: Station B', 'Remaining distance to station A: 350m'.

[0137] Through this, passengers can intuitively recognize the current operating location of the tram (10) and the next stop.

[0138] The second image display area (3120) can display a three-dimensional virtual screen of the surrounding image of the tram (10).

[0139] A three-dimensional virtual screen of the surrounding image of the tram (10) refers to a screen that virtually reproduces the actual surrounding environment in a three-dimensional form corresponding to the current driving position and driving direction of the tram (10), based on images obtained from cameras (110) positioned at the front and left and right sides of the tram (10).

[0140] The control unit (210) can fuse the image obtained from the camera (110) and the position data obtained from the position recognition sensor (140) to convert the current driving environment of the tram (10) into a 3D model, render it, and output it in real time to the second image display area (3120).

[0141] Below, a detailed method for implementing a three-dimensional virtual screen displayed in the second image display area (3120) will be described.

[0142] The control unit (210) can generate an integrated image by projecting multiple images acquired from the front camera, left camera, and right camera of the camera (110) onto a preset common coordinate system. Specifically, the control unit (210) can perform lens distortion correction and projection transformation on the images acquired from each camera (110). The projection transformation can be performed using a preset transformation matrix corresponding to the installation position and shooting angle of each camera (110). The transformation matrix can be calculated in advance through the calibration process of the camera (110) and stored in memory (212).

[0143] The control unit (210) can generate an integrated image by aligning a plurality of projected transformed images on a common coordinate system. In this case, if there is an overlapping area between the plurality of images, the control unit (210) can perform blending processing to ensure that the boundary lines between the images are naturally connected. Blending processing can be performed by applying weights to the pixel values ​​of each image in the overlapping area to mix them, and the weights can be adjusted linearly or non-linearly depending on the pixel position within the overlapping area.

[0144] The control unit (210) can display a 3D model of the tram (10) in the integrated image by placing it at a preset position based on the position data and driving direction data obtained from the position recognition sensor (140). The 3D model of the tram (10) can be rendered using model data pre-stored in the memory (212). The 3D model of the tram (10) is placed at the center of the integrated image, and the integrated image can be displayed in a form where the surrounding environment is viewed from above with the tram (10) at the center.

[0145] The control unit (210) can calculate the three-dimensional position of surrounding objects by further utilizing distance data obtained from the lidar (120) and display it by superimposing it on the integrated image. Specifically, the control unit (210) can detect the position and shape of objects (e.g., pedestrians, vehicles, bicycles, static obstacles, etc.) existing around the tram (10) by analyzing the point cloud data of the lidar (120). The detected objects can be displayed as icons, bounding boxes, or simplified three-dimensional shapes within the integrated image, and distance information to the objects can be displayed together as numerical values ​​or colors.

[0146] The control unit (210) can identify objects among the surrounding objects included in the object recognition information whose collision risk level is above a preset threshold and highlight the object within the three-dimensional virtual screen. The collision risk level can be calculated by comprehensively considering the distance between the tram (10) and the object, the direction of movement of the object, and the relative speed. For example, objects with a high collision risk level may be displayed with a red outline, a flashing indicator, or a warning icon, and objects with a medium risk level may be displayed in yellow.

[0147] The control unit (210) can identify objects among surrounding objects included in the object recognition information that have a collision risk level greater than or equal to a preset threshold, and highlight the object within the three-dimensional virtual screen.

[0148] The collision risk can be calculated by comprehensively considering the distance between the tram (10) and the object, the direction of movement of the object, and the relative speed. Specifically, the control unit (210) can calculate the collision risk (CR) using the following mathematical formula.

[0149] [Mathematical Formula 1]

[0150] CR = w1 x (1 / D) + w2 x V_rel + w3 x cos(θ)

[0151] Here,

[0152] - D is the distance between the tram (10) and the object (unit: m)

[0153] - V_rel is the relative speed of the object to the tram (10) (unit: m / s)

[0154] - θ is the angle formed by the direction of travel of the tram (10) and the direction of movement of the object (unit: degrees)

[0155] - w1, w2, and w3 are weights for distance, relative velocity, and direction, respectively, and are positive numbers satisfying w1 + w2 + w3 = 1

[0156] In the above mathematical formula 1, the (1 / D) term is a factor that increases the risk of collision as the distance between the tram (10) and the object becomes closer. The V_rel term is a factor that increases the risk of collision as the relative speed of the object with respect to the tram (10) becomes greater. The cos(θ) term is a factor that increases the risk of collision as the object moves in the direction of approach toward the tram (10) (i.e., as θ becomes closer to 0 degrees).

[0157] For example, weights w1, w2, and w3 can be set to 0.5, 0.3, and 0.2, respectively. This means that the distance factor is considered most important, and the relative speed and direction are considered secondarily. However, the weights can be adjusted variably depending on the operating environment, driving speed, weather conditions, etc. of the tram (10).

[0158] The control unit (210) can determine the risk level by comparing the calculated collision risk (CR) with a plurality of preset threshold values. For example, the control unit (210) can classify the risk level as follows.

[0159] - Low risk: CR < 0.3 - Medium risk: 0.3 ≤ CR < 0.7

[0160] - High risk: CR ≥ 0.7

[0161] The control unit (210) can display the object in a different way within a three-dimensional virtual screen according to the determined risk level. For example, low-risk objects may be displayed with a green outline, medium-risk objects with a yellow outline, and high-risk objects with a red outline and a flashing effect. Additionally, for high-risk objects, a warning icon (e.g., in the form of an exclamation mark) and distance information to the object may be displayed together.

[0162] In another embodiment, the control unit (210) may additionally calculate the Time To Collision (TTC) using the following mathematical formula and reflect it in the collision risk assessment.

[0163] [Mathematical Formula 2]

[0164] TTC D / V_rel (where V_rel > 0)

[0165] Here,

[0166] - TTC is the estimated time to collision if the current relative speed is maintained (in seconds)

[0167] - D is the distance between the tram (10) and the object (unit: m)

[0168] - V_rel is the relative approach speed of the object to the tram (10) (unit: m / s)

[0169] The control unit (210) can determine that the object is a high-risk object if the calculated TTC is less than or equal to a preset threshold time (e.g., 3 seconds, 5 seconds). Since a shorter TTC indicates that a collision is imminent, the control unit (210) may additionally apply a weight inversely proportional to the TTC to the calculation of the collision risk (CR) of the above mathematical formula 1.

[0170] In detail, the control unit (210) can calculate a TTC-based correction coefficient (k) using the following mathematical formula and apply it to the mathematical formula 1 to calculate the final collision risk (CR_final).

[0171] [Mathematical Formula 3]

[0172] k = 1 + α x (1 / TTC) (where TTC > 0)

[0173] Here,

[0174] - k is a TTC-based correction coefficient

[0175] - α is a proportionality constant that controls the influence of TTC (e.g., α = 1.0, 2.0)

[0176] - TTC is the estimated time of collision (in seconds)

[0177] [Mathematical Formula 4]

[0178] CR_final = kx CR = [1 + α x (1 / TTC)] x [w1 x (1 / D) + w2 x V_rel + w3 x cos(θ)]

[0179] In the above mathematical formula 3, as the TTC becomes shorter, the value of (1 / TTC) increases, so the correction factor k becomes larger, and accordingly, the final collision risk (CR_final) increases. For example, when the TTC is 1 second, k = 1 + 1.0 x (1 / 1) = 2.0, so the basic collision risk is amplified by 2 times, and when the TTC is 5 seconds, k = 1 + 1.0 x (1 / 5) = 1.2, so the basic collision risk can be amplified by 1.2 times.

[0180] The control unit (210) may apply the correction factor k only when the TTC is less than or equal to a preset threshold time. For example, if the threshold time is set to 5 seconds, the correction factor is not applied (i.e., k = 1.0) to objects where TTC > 5 seconds, and only the basic collision risk is used. This allows the risk to be selectively amplified only for objects that are imminent to collide.

[0181] At least one of the embodiments described above may also be implemented in the form of a recording medium containing computer-executable instructions, such as program modules executed by a computer. A computer-readable medium may be any available medium accessible by a computer and includes both volatile and non-volatile media, and both removable and non-removable media. Additionally, a computer-readable medium may include a computer storage medium. A computer storage medium includes both volatile and non-volatile, removable and non-removable media implemented by any method or technique for storing information, such as computer-readable instructions, data structures, program modules, or other data.

[0182] Although the apparatus and method of the present invention have been described in relation to specific embodiments, some or all of their components or operations may be implemented using a computer system having a general-purpose hardware architecture.

[0183] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical spirit or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0184] The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted as being included within the scope of the present invention. Explanation of the symbols

[0185] 10: Tram 100: Multiple sensors 110: Camera 120: LiDAR 130: Radar 140: Location recognition sensor 150: Signal information receiver 200: Engine Room 210: Control Unit 211: Processor 212: Memory 213: Communication Module 220: 1st display unit 300: Passenger cabin 310: Second display unit 400: Drive unit 500: Battery section

Claims

Claim 1 A tram capable of providing tram operation information comprises: a plurality of sensors; a control unit that generates an engine user interface and a passenger interface based on sensor data acquired through the plurality of sensors; a first display unit that displays the engine user interface; and a second display unit that displays the passenger interface, wherein the control unit generates tram operation information based on the sensor data and generates the engine user interface and the passenger interface based on the tram operation information, and the passenger interface includes a second information display area that displays guidance information to be provided to passengers among the tram operation information, and a second image display area disposed around the second information display area, wherein the second information display area includes a second-1 information display area that displays operation control information including the current speed, braking status, and speed limit of the tram; and a second-2 information display area that displays tram signal information including the current signal, next signal, and remaining time of a tram-dedicated signal. A tram comprising a second and third information display area that displays station information including the name of the current station, the name of the next station, and the remaining distance to the current station, wherein the second image display area displays a three-dimensional virtual screen of the surrounding image of the tram, and the control unit fuses the sensor data to convert the driving environment of the tram into a three-dimensional model and outputs a three-dimensional virtual screen rendered from the three-dimensional model to the second image display area in real time, wherein for surrounding objects recognized through the plurality of sensors, the collision risk is calculated based on the distance between the tram and the object, the relative speed of the object with respect to the tram, the angle formed by the direction of travel of the tram and the direction of movement of the object, and the estimated time of collision, and the object whose calculated collision risk is greater than or equal to a preset threshold is highlighted within the three-dimensional virtual screen. Claim 2 In claim 1, the above-mentioned interface for the operator includes a first information display area that displays information necessary for the operator's tram operation among the tram operation information, and a first image display area disposed around the first information display area, wherein the first information display area includes: a first-1 information display area that displays sensor status information including the operating status of each of the plurality of sensors; a first-2 information display area that displays tram signal information including at least one of the current signal, next signal, and remaining time of a tram-dedicated signal; a first-3 information display area that displays operation status information including at least one of the tram's operating section, remaining distance, and target speed; a first-4 information display area that displays operation control information including at least one of the tram's current speed, braking status, and speed limit; and a first-5 information display area that displays operation condition information including at least one of the tram's power supply status, battery charge status, and door opening / closing status. Claim 3 In paragraph 2, the first image display area comprises: a first-1 image display area representing object recognition information based on the front image of the tram; a first-2 image display area representing object recognition information based on the left image of the tram; and a first-3 image display area representing object recognition information based on the right image of the tram. Claim 4 In claim 1, the control unit determines the risk level of an object by comparing the collision risk level with a preset first threshold value and a preset second threshold value greater than the first threshold value, wherein if the collision risk level is less than the first threshold value, the risk level is determined to be low risk; if the collision risk level is greater than or equal to the first threshold value and less than the second threshold value, the risk level is determined to be medium risk; if the collision risk level is greater than or equal to the second threshold value, the risk level is determined to be high risk; and according to the determined risk level, the object is displayed in a different display method within the three-dimensional virtual screen. Claim 5 In paragraph 2, the sensor status information is information obtained from the plurality of sensors, the tram signal information is information generated based on signal data obtained from the tram-dedicated signal device, the operation status information is information generated based on preset target path data, the operation control information is information generated based on the driving status data of the tram, and the operation condition information is information generated based on the vehicle status data of the tram. Claim 6 In paragraph 3, the plurality of sensors includes a front camera, a left camera, and a right camera, wherein the front image is obtained from the front camera positioned to photograph the front of the tram, the left image is obtained from the left camera positioned to photograph the left side of the tram, and the right image is obtained from the right camera positioned to photograph the right side of the tram. Claim 7 A tram according to paragraph 3, wherein the first information display area is positioned on the left side of the first image display area, the first-1 image display area is positioned on the upper side of the first image display area, the first-2 image display area is positioned on the lower left side of the first image display area, and the first-3 image display area is positioned on the lower right side of the first image display area. Claim 8 A tram according to claim 1, wherein the operation control information is information generated based on the driving state information of the tram, the tram signal information is information generated based on signal data obtained from the tram-dedicated signal device, and the station information is information generated based on pre-set station data. Claim 9 A tram according to claim 1, wherein the second information display area is positioned on the left side of the second image display area, the second-1 information display area is positioned on the upper side of the second information display area, the second-2 information display area is positioned in the middle of the second information display area, and the second-3 information display area is positioned on the lower side of the second information display area. Claim 10 In paragraph 4, the control unit displays an object determined to have a low risk level with a green outline, displays an object determined to have a medium risk level with a yellow outline, displays an object determined to have a high risk level with at least one of a red outline, a flashing effect, and a warning icon, and further displays distance information to the object determined to have a high risk level.

Images