Cross section integrated information providing system and method based on V2X

Abstract

The intersection integrated information system according to the present invention comprises: a preprocessor that estimates the location and speed of an object from spatial information received from an infrastructure image information integration platform, an autonomous vehicle terminal, and a smart device; an object refinement module that defines objects by analyzing data obtained from the preprocessor and removes information with low relevance; a situation awareness module that defines the relationship between objects determined by the object refinement module; a risk judgment module that determines risk factors within an intersection using information based on information from the object refinement module and the situation awareness module; and an action plan module that establishes a situation judgment and an action plan based on information from at least one of the object refinement module, the situation awareness module, and the risk judgment module.

Description

Technology Field

[0001] The present invention relates to an intersection safety support system and a method thereof. Specifically, it relates to a system and method that supports intersection safety for autonomous vehicles, non-autonomous vehicles, and pedestrians by providing integrated information including information on adjacent intersections in an autonomous driving mixed-flow environment based on V2X communication. Background Technology

[0002] Recently, partially autonomous vehicles (Autonomous Driving L2) have been commercialized and are driving on the roads, and it is expected that commercial vehicles with a high level of autonomous driving capabilities of L4 or higher will be released in 2025.

[0003] However, for a considerable period, autonomous and non-autonomous vehicles will be mixed in actual driving environments, and in the case of intersections, even if autonomous vehicles are 100% adopted, pedestrians will move together with autonomous vehicles.

[0004] Therefore, intersection sections where autonomous vehicles, non-autonomous vehicles, and pedestrians move together are the most difficult areas to analyze and predict within traffic conditions, yet an integrated information provision system that considers the mixed-flow environment in such areas is currently lacking. The problem to be solved

[0005] The problem that the present invention aims to solve is as follows.

[0006] 1) Providing a system that generates integrated urban intersection information in real time by fusion processing infrastructure-based information, vehicle-based information, digital maps, etc., to provide accurate integrated information in autonomous driving mixed-flow environments.

[0007] 2) Provision of integrated information capable of high-precision position correction for multiple objects (autonomous vehicles, non-autonomous vehicles, pedestrians, etc.) in mixed-flow intersection environments

[0008] 3) Supports high-safety autonomous driving by predicting dangerous situations within intersections for vehicles, pedestrians, and bicycles in conjunction with signal display based on fused integrated information.

[0009] 4) Provides smartphone-based danger warning services to non-autonomous vehicles not equipped with the system and to pedestrians. means of solving the problem

[0010] An intersection integrated information system for solving the aforementioned problem comprises: a preprocessor that estimates the location and speed of an object from spatial information received from an infrastructure image information integration platform, an autonomous vehicle terminal, and a smart device; an object refinement module that analyzes data obtained from the preprocessor to define objects and removes information with low relevance; a situation awareness module that defines the relationships between objects determined by the object refinement module; a risk judgment module that determines risk factors within an intersection using information based on information from the object refinement module and the situation awareness module; and an action plan module that establishes a situation judgment and an action plan based on information from at least one of the object refinement module, the situation awareness module, and the risk judgment module.

[0011] According to another aspect of the present invention, a method for operating an intersection integrated information system is provided, comprising: a preprocessing step for estimating the location and speed of an object from spatial information received from an infrastructure image information integration platform, an autonomous vehicle terminal, and a smart device; an object refinement step for defining an object by analyzing data obtained in the preprocessing step and removing information with low relevance; a situation recognition step for defining the relationship between objects determined in the object refinement step; a risk judgment step for determining a risk factor within an intersection using information based on the information generated in the object refinement step and the situation recognition step; and an action planning step for establishing a situation judgment and an action plan based on information from at least one of the object refinement step, the situation recognition step, and the risk judgment step.

[0012] According to another aspect of the present invention, an intersection integrated information system is provided, comprising an infrastructure sensor that generates type and location information of multiple objects, such as vehicles and pedestrians, within an intersection, and an intersection integrated information server that receives object information and location information of said objects from said infrastructure sensor and an autonomous vehicle near the intersection, wherein the infrastructure sensor performs object recognition through a classification algorithm using machine learning to distinguish between vehicles, pedestrians, and backgrounds, and is configured to enable double verification with two classifiers to prepare for misrecognition situations.

[0013] According to another aspect of the present invention, an intersection integrated information system is provided comprising an infrastructure sensor that generates type and location information of multiple objects, such as vehicles and pedestrians, within an intersection, and an intersection integrated information server that receives object information and location information of said objects from the infrastructure sensor and smartphones of autonomous vehicles, non-autonomous vehicle drivers, and pedestrians near the intersection, wherein the intersection integrated information server converts the location recognition information of the infrastructure sensor, the location information of the autonomous vehicle, and the location information of the smartphone into a single coordinate system and unifies the units, and then fuses each data by applying the location variance ratio of the data from the infrastructure sensor, the autonomous vehicle, and the smartphone as a weight, and provides services such as collision risk warning between each object, visibility poor collision risk warning, fuel efficiency optimal speed recommendation, concealed sudden pedestrian protection, and pedestrian crossing support based on said fused data. Effects of the invention

[0014] Through the present invention, it is possible to support fuel-efficient driving by estimating real-time traffic conditions and linking signal display. Additionally, by securing an open digital field of view through perception sensors installed in the infrastructure, it is possible to recognize the movements of pedestrians and motorcycles concealed around the intersection, thereby more effectively preventing pedestrian accidents at intersections.

[0015] Ultimately, coordinating the traffic of autonomous vehicles, non-autonomous vehicles, and pedestrians can lead to the early adoption of autonomous vehicles and the prevention of intersection accidents. Brief explanation of the drawing

[0017] FIG. 1 is a data flow diagram illustrating the data flow for each subject on an integrated information providing system according to a preferred embodiment of the present invention. FIG. 2 is a schematic installation specification diagram of an infrastructure sensor according to the present invention. FIG. 3 is a data flow diagram illustrating data collection and processing of an autonomous vehicle according to the present invention. FIG. 4 is a data flow diagram illustrating the types of data from the main data collection sources of the intersection integrated information provision system according to the present invention. FIG. 5 is a logical data model of an intersection integrated information providing system according to the present invention. FIG. 6 is a structural diagram of an intersection integrated information server according to the present invention. FIG. 7 is a correlation diagram between the time required for the first step and the magnitude of the steady-state speed for determining the steady-state speed of a pedestrian according to the present invention. FIG. 8 is an exemplary diagram illustrating a pedestrian recognition area within an acquired image according to the present invention. FIG. 9 is an exemplary diagram for explaining the linear segment-based collision risk calculation of the present invention. FIG. 10 is a conceptual diagram of a clock malfunction collision risk warning service according to the present invention. FIG. 11 is a conceptual diagram of a fuel efficiency optimal speed recommendation scenario service according to the present invention. FIG. 12 is a conceptual diagram of a concealed pedestrian protection service according to the present invention. FIG. 13 is a conceptual diagram of a pedestrian crossing support service according to the present invention. FIG. 14 is a flowchart illustrating the operation method of an intersection integrated information provision system according to the present invention. Specific details for implementing the invention

[0018] The objectives and effects of the present invention are not limited to those mentioned above, and the objectives and effects of the present invention, as well as the technical configurations for achieving them, will become clear by referring to the embodiments described in detail below together with the accompanying drawings.

[0019] In describing the present invention, if it is determined that a detailed description of known functions or configurations may unnecessarily obscure the essence of the invention, such detailed description is omitted. Furthermore, the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms. Each of the following embodiments is provided to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and is not intended to limit the scope of the invention.

[0020] Throughout the specification, when a part is described as "comprising" or "equipped" with a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Furthermore, terms such as "...unit," "...device," "...part," or "...module" as used in the specification refer to a unit that processes at least one function or operation, and this may be implemented in hardware, software, or a combination of hardware and software.

[0021] Meanwhile, in each embodiment of the present invention, each component, functional block, or means may be composed of one or more sub-components, and the electrical, electronic, and mechanical functions performed by each component may be implemented by various known devices or mechanical elements such as electronic circuits, integrated circuits, and ASICs (Application Specific Integrated Circuits), and may be implemented separately or two or more may be integrated into one.

[0023] According to a preferred embodiment of the present invention, the configuration and method of an integrated information provision system are disclosed, which collects accurate location information of vehicles and pedestrians within an intersection using infrastructure and onboard sensors, fuses the collected information to generate real-time integrated information, predicts dangerous situations based thereon, provides the generated information to vehicles and pedestrians, supports economical driving such as carbon dioxide reduction by linking with the signal systems of the current intersection and adjacent intersections, and provides overall situational information within the intersection in real time by implementing a connected environment between the infrastructure and the vehicle by communicating with each collection module.

[0024] Below, information collection, integrated information generation, risk situation prediction, and operation support are each described in detail with reference to the attached drawings.

[0026] [Collection of Vehicle and Pedestrian Location Information]

[0027] FIG. 1 illustrates the data flow for each subject on an integrated information provision system according to a preferred embodiment of the present invention.

[0028] As described above, location and status information of multiple objects within an intersection is collected and exchanged using infrastructure (intersection cameras, radar, etc.), sensors within autonomous vehicles (cameras, GPS, UWB tags, IMUs, etc.), and smartphones of non-autonomous vehicle drivers or pedestrians.

[0029] Intersection situation information is collected and provided through a standard information exchange interface because there are various sources of information, such as infrastructure camera-based image recognition information, information from vehicles, digital maps, and information from pedestrians.

[0030] The efficiency of data processing is secured through a separate data processing unit for each data collector, and intersection safety support and economic driving services are provided to vehicles and pedestrians based on integrated situational information through the future intersection integrated information server (100), which is a data fusion platform.

[0031] - Infrastructure Sensor

[0032] The infrastructure sensor module adopts an integrated configuration comprising an HD-resolution camera, traffic radar, object recognition, and communication unit, generating type and location information of multiple objects such as vehicles and pedestrians within the intersection from the sensor terminal and transmitting it via wireless communication (WIFI) to an intersection integrated information edge computer on the local network within the intersection.

[0033] It is installed on a standard traffic light pole or street light pole embedded in the corner of an intersection. A minimum of one and a maximum of two units are installed per corner, and it is preferable to install them as shown in Fig. 2 so that each camera sensor can recognize objects within the intersection and objects in the direction of entry.

[0034] The desirable field of view (FOV) of the camera is 60 degrees horizontally and 33 degrees vertically, and has a spatial resolution of 4 cm (based on a relative distance of 40 m) required to acquire dynamic information of an object by applying HD quality (1280x720).

[0035] A single infrastructure sensor module enables simultaneous recognition of the intersection interior and the direction of entry (redundant recognition for each corner and direction of entry).

[0036] As shown in Fig. 2, the installation position should be at a height of 5m from the ground, and the angle should be such that the center of the camera forms a 73.5° vertical direction with the ground. When installing a camera at an intersection corner, it is recommended to cover all critical points (CPs) with a horizontal FOV of 60° and to set the camera installation position so that it is perpendicular to the pedestrian's traversing direction to facilitate pedestrian motion measurement.

[0037] The infrastructure sensor performs object recognition using a machine learning-based classification algorithm to distinguish between vehicles, pedestrians, and backgrounds, and is configured to enable double verification with two classifiers to prevent misidentification.

[0038] Since classification algorithms based on machine learning require a large amount of computation, the present invention applies a separate candidate extraction algorithm to reduce the number of targets to be classified.

[0039] Since the first classification algorithm requires a high-speed processing algorithm due to the large number of classification targets, it is designed based on HOG+SVM (Histogram of Gradient + Support Vector Machine), and the second classification algorithm performs second classification by finally excluding results misidentified by the first classifier.

[0040] The object recognition algorithm of the present invention applies a Kalman filter as a basic tracking algorithm, estimates the relative speed based on the distance from the infrastructure sensor to the object and the amount of movement within the image, and extracts the absolute and relative positions of multiple objects using the ground position of traffic light poles and corner points of road signs (stop lines, crosswalks), which are fixed facility information of a high-precision road map.

[0041] - Sensors inside autonomous vehicles

[0042] As shown in FIG. 3, autonomous vehicles commonly employ a front camera, a positioning sensor (GPS, IMU), and a V2X communication unit, but in the present invention, a UWB tag is additionally used to enable the acquisition of high-precision location information even in areas with poor GPS reception within the city.

[0043] The UWB unit generates high-precision location information through UWB tags installed on vehicles by installing APs at intersections based on wireless synchronization-based OTDoA (Observed Time Difference Of Arrival) technology. Since there is no limit to the number of UWB tags mounted on vehicles that can be simultaneously recognized, the locations of multiple vehicles within the intersection can be recognized.

[0044] Positioning from positioning sensors (GPS / UWB / IMU) is achieved by acquiring positioning signals from various types of sensors as shown in Fig. 3, and statistically analyzing the signal quality of GPS for each surrounding environment of the vehicle through a segment switching algorithm using signal quality, and positioning using the best signal.

[0045] Specifically, the signal quality of the GPS is determined by measuring the number of satellites, HDOP, FIX value, and reliability parameters in the NMEA Message of the GPS. It operates as GPS / INS in areas where UWB infrastructure is not installed, and as UWB / INS in GPS blind spots such as urban intersections. Depending on the performance of GPS and UWB, the vehicle driving environment is classified into a GPS Zone (GPS / INS-based position estimation), a UWB Zone (UWB / INS-based position estimation), and a Dead Zone (INS-based position estimation) to perform position estimation.

[0046] The generated high-precision location information is embedded in a BSM message via V2X communication (WAVE) and transmitted to the intersection integrated information edge computer.

[0047] The front camera of an autonomous vehicle acquires driving lane recognition information, recognition information of the vehicle ahead (type and location information), and recognition information of pedestrians jaywalking across the road, and transmits this information to the intersection integrated information edge computer by embedding it into a BSM message via V2X communication (WAVE).

[0048] - Smartphones of non-autonomous vehicle drivers or pedestrians

[0049] Drivers of non-autonomous vehicles or pedestrians who want the service can receive the service through a smartphone app.

[0050] In order for a smartphone to participate in intersection V2X communication, a WAVE communication module is provided at a designated location in the intersection, and the smartphone and the WAVE communication module use Bluetooth communication.

[0051] It uses smartphone-embedded MEMS (Microelectromechanical Systems) sensors and implements a dead reckoning system using accelerometers, gyroscopes, geomagnetic sensors, etc.

[0052] An Attitude Heading Reference System is constructed using a combination of geomagnetic sensors and gyroscopes to measure the direction of the Earth's magnetic field and estimate the direction of travel by measuring rotation.

[0053] Satellite navigation systems suffer from reduced positioning accuracy in urban shadow areas, while dead reckoning systems face the problem of error accumulation during prolonged operation. Therefore, to compensate for the shortcomings of each system, a fusion of satellite navigation and dead reckoning is used. It is recommended to use a Kalman filter to fuse the two systems.

[0054] Location and time information from the smartphone are transmitted to the intersection integrated information edge computer as PSM messages via a WAVE terminal using Bluetooth communication, and service information such as danger alerts and economical driving information is received according to the smartphone user type (non-autonomous vehicle, pedestrian).

[0055] - V2X Communication Department

[0056] In this invention, an IEEE 1609.X-based communication protocol is used, and a SAE J 2735-based message set is defined.

[0057] Extension areas (VehicleSafetyExtensions, SpecialVehicleExtensions, SupplementalVehicleExtensions) were added to the basic message sets for vehicle-targeted BSM and pedestrian-targeted PSM.

[0058] Information received through the WAVE communication module of autonomous vehicles and the WAVE communication modules of smartphone users, such as non-autonomous vehicles and pedestrians, as well as transmitted service information, is encoded and decoded as messages by the WAVE controller via the WAVE antenna of the intersection infrastructure environment and transmitted to and from the intersection integrated information edge computer configured in the local network.

[0059] The local network is synchronized based on time information synchronized with an NTP-Time Server.

[0060] The information transmission items by information collection entity for the convergence of intersection integrated information are as follows.

[0061] Infrastructure Cameras (Vehicles, Pedestrians, Incidents): Transmission time, location (x,y,z), heading, speed, recognition time, width, height, species reliability, location reliability (x,y,z), behavioral patterns, etc.

[0062] Autonomous Vehicle Terminal UWB (Autonomous Vehicle): Transmission time, latitude, longitude, altitude, accuracy, speed, direction, acceleration, etc.

[0063] Autonomous Vehicle Terminal Front Camera (Front Vehicle, Sudden Pedestrian): Transmission time, relative distance to front vehicle (x,y), relative speed, front vehicle duplication, reliability by object type, etc.

[0064] Smartphone Information: Transmission time, type (vehicle / pedestrian), longitude, latitude, altitude, positioning reliability, heading, speed, satellite status information, positioning time, behavioral pattern, etc.

[0065] The above explanation is summarized in Table 1 below.

[0066] Information collecting entity Perception range Cognitive information V2I communication method Infrastructure Camera Inside the intersection -Vehicle-Pedestrian-Sudden Incident Wireless (TCP / IP) UWB autonomous vehicle terminal Full range of intersections BSM - Autonomous Vehicle WAVE Autonomous vehicle terminal front camera driving lane BSM Extension - Vehicle Ahead - Sudden Pedestrian WAVE smartphone Full range of intersections BSM - Non-autonomous vehicles PSM - Pedestrians BLE+WAVE

[0068] - Logical data model for cloud-linked information collection and provision

[0069] The intersection situation information standard logical data model based on dynamic information data items corresponding to the intersection situation information used in the present invention has the MAP and Dynamic Information classes inherited by the subclasses of the intersection situation information Class.

[0070] A logical data model is exemplified in Fig. 5, and the MAP and Dynamic Information classes are classes for expressing intersection situation information, consisting of a set of static information provided on the map, such as speed information, traffic restriction information, lane information, traffic sign information, and section information, and a set of dynamic information provided, such as traffic condition information, road surface information, weather information, vehicle surrounding information, and danger situation information.

[0072] Below, based on the collected information, the future intersection integrated information server (100) of the data fusion framework This document explains in detail the methods for generating real-time integrated information, predicting dangerous situations, and supporting safe and economical driving services.

[0073] The bidirectional data flow diagram was defined as a sequential level relationship, and the schematic diagram is as shown in Fig. 6.

[0074] As described, the future intersection integrated information server (100) of the data fusion framework, which serves as the backbone of the data flow, includes a preprocessor (110), an object refinement module (120), a situation awareness module (130), a risk judgment module (140), and an action planning module (150).

[0075] The preprocessor (110) estimates the position and speed of an object from spatial information received from an infrastructure image information integration platform, an autonomous vehicle terminal, and a smart device.

[0076] The object refinement module (Level 1) (120) determines the location, parameter information, and identification information of the object, and defines the object by analyzing the data obtained from preprocessing through tasks such as estimation, target tracking, and pattern recognition, and removes information with low relevance. The functions performed at this stage are classified into four categories.

[0077] * Data alignment: Coordinate system transformation, unit unification

[0078] * Data / Object Correlation: Nearest Neighbor Method

[0079] * Position / kinematics & attribute estimation: Using Kalman filter

[0080] Object identity estimation: Using pattern recognition

[0082] The situation awareness module (Level 2) (130) defines the relationships between objects determined at Level 1. It describes the current situation using object information.

[0083] Object aggregation

[0084] Event / activity interpretation

[0085] Contextual interpretation

[0087] The risk judgment module (Level 3) (140) is a module that determines future risk information from the current situation and processes the Level 3 stage, which makes decisions regarding the impact each object may have on traffic accidents and the provision of safety information using information obtained from Levels 1 and 2. If there is insufficient basis for predicting the situation at Level 3, the judgment is withheld and feedback is requested from Levels 2 and 1 regarding the collection of information for predicting the situation.

[0089] The action planning module (Level 4) (150) processes a Level 4 step of planning an action for decision-making by determining the situation from the information provided by the Level 3 prediction results processed by the risk judgment module (130).

[0091] The database (160) includes an object database and a fusion database. The object database stores objects that have been interpreted by a preprocessor from raw data collected from various sources. The fusion database stores lists related to risks and countermeasures.

[0092] In addition, the server (100) according to the present invention generates real-time LDM for intersections and builds a database. The intersection autonomous driving mixed flow integrated information (LDM) is classified into basic data, fusion data, and service data according to the data collection stage and stored in the database, and is distinguished by assigning keywords for each stage to the table name.

[0093] Collection system Collected data Communication base station server Autonomous vehicle transmission data, general vehicle transmission data, pedestrian transmission data Infrastructure video server Infrastructure Vehicle Recognition Infrastructure Pedestrian Recognition Queue Recognition database server Economical driving traffic volume data Map data Vehicle information fusion data Pedestrian information fusion data Queue fusion data Signal display data (communication)

[0094] In addition, a guidance module (not shown) is included to provide action plans, such as identified situations, avoidance information, and guidance information.

[0095] The processing performed by each module is explained in detail below.

[0097] [Real-time Integrated Information Generation]

[0098] Local Dynamic Map (LDM) is dynamic map information that displays detailed information about the area around an intersection by integrating a static digital map with the continuously changing locations of moving objects, such as vehicles and pedestrians.

[0099] The object refinement module (120) fuses previously collected information to generate a region map and performs high-precision position correction for multiple objects.

[0100] Object IDs are assigned and locations are corrected through data fusion of infrastructure camera object location recognition information, autonomous vehicle location information (UWB), and smartphone (GPS-PDR) location information.

[0101] First, location recognition information from infrastructure cameras, location information from autonomous vehicles, and location information from smartphones acquired from different sources are converted into a single coordinate system, and the units are unified.

[0102] Next, the location variance ratios of the data from infrastructure cameras, autonomous vehicles, and smartphones are applied as weights to fuse each data set.

[0103] The situation awareness model (130) predicts pedestrian paths and vehicle paths based on the above fusion data.

[0104] Pedestrian path prediction is performed by generating an RLS-based linear path prediction model based on the history of collected data. However, due to the behavioral characteristics of pedestrians within intersections, it is much more common for them to stop and then move rather than to walk continuously at a constant speed. Therefore, since considering only the displacement within each video frame can lead to significant errors, it is necessary to improve the statistical-based behavior prediction algorithm by taking into account the uncertainty of pedestrian behavior patterns to ensure the reliability of the intersection pedestrian model.

[0105] To this end, the time parameter (Prediction) required to reach a steady state, from when a pedestrian starts moving from a standing position to when they enter a stable walking state, is estimated. That is, to increase the reliability of pedestrian path prediction, a parameter for the time required to reach the pedestrian's steady state speed (acceleration section) is needed, and this can be derived by regressing about 120 Ground Truths (GT) obtained from actual intersections to derive a model equation.

[0106] As shown in Fig. 7, the predicted time required to reach a steady state is linearly related to the time required for the first step for steady state speeds of 2.5 m / s or less, and for values ​​greater than that, it was applied by assuming it is 1.55 seconds.

[0107] The time required to reach a steady state for pedestrian path prediction is defined as follows:

[0108]

[0109] The time required to reach steady-state speed tends to be proportional to the magnitude of the steady-state speed, and there is no correlation above a certain speed.

[0110] In addition, the improvement of the pedestrian behavior model considering real urban intersection sensor perception characteristics based on actual intersection sensor data characteristics (algorithm tuning and false-positive performance improvement) is performed as follows.

[0111] Unlike reference sensors, pedestrian motion information is extracted based on the pedestrian blind spot recognition area (ROI) rather than precise measurement of body parts (see Fig. 8), the size of the pedestrian stride can be acquired based on the width of the recognition area (Box width), and the lower body speed is calculated by calculating the video fps for the increase in stride length.

[0112] The magnitude of upper body flexion in pedestrians is classified into 1 to 3 stages, and is categorized into the standing posture, the medium flexion that occurs during fast walking, and the large flexion that occurs when suddenly running out.

[0113] A pedestrian is determined to be stationary if there is no change in the horizontal length of the perception area, and when the pedestrian begins to move, the size of the perception area box width increases, and if it exceeds a certain threshold, the pedestrian can be determined to be in motion.

[0114] When the pedestrian's first step is completed and the forefoot lands on the ground, the increase in the horizontal width of the perception area stops and then shows a decreasing trend, which is determined as the point where the first step is completed.

[0115] Even if the pedestrian recognition area (Box width) increases after movement begins, the pedestrian speed prior to the completion of the first step is not detected; therefore, a prediction algorithm is used to predict the steady-state speed at this stage.

[0116] Pedestrian motion information acquired by roadside sensors is extracted into a pedestrian recognition area (Box), and stride length, lower body speed, and upper body tilt are transformed into a format suitable for sensor acquisition.

[0117] The present invention performs pedestrian path prediction by considering the time required for a standing pedestrian to reach a steady state speed, and verifies that an error of within 20 to 40 cm occurs when comparing the path prediction result with the actual travel distance (GT).

[0118] The time at which a pedestrian completes their first step is within 0.3 to 0.6 seconds, which corresponds to the 10th frame of the roadside sensor. According to the pedestrian prediction method of the present invention, it was verified through a sample of pedestrians at an actual intersection that it is possible to predict the path 2 to 3 seconds later at that time.

[0119] This section explains vehicle path prediction. For autonomous vehicles, accurate path prediction is possible by receiving information such as current position, heading angle, steering angle, and speed from the vehicle. Vehicle path prediction is relevant to non-autonomous vehicles; it treats the vehicle's driving lane as an individual model, processes the vehicle's speed and heading angle at the current time as input information, and uses a non-linear Extended Kalman Filter (EKF).

[0120] However, in the case of non-autonomous vehicles, their position, speed, and heading are recognized through roadside cameras. Yet, the heading angle is highly inaccurate, and information regarding vehicle size is also unavailable. Therefore, prior to designing the Kalman filter, the following is assumed.

[0121] Point mass is assumed for the model. Each model is predicted and updated according to the attributes of the road component. Speed ​​and yaw rate are configured as model inputs, and the yaw rate is determined by the curvature of the road.

[0122] As mentioned above, the heading angle extracted from roadside camera footage is highly inaccurate, resulting in the largest prediction error. Therefore, observations are composed solely of location, and the heading angle is estimated through a filter.

[0124] [Predicting Risk Situations]

[0125] Through the above process, objects such as autonomous vehicles, non-autonomous vehicles, and pedestrians are recognized and their locations and paths are predicted, and the risk judgment module (140) predicts a dangerous situation.

[0126] This document explains an algorithm for determining accident risk by route planning type, which assesses and predicts hazardous situations (prior risk, convergence risk) within intersections to provide information to vehicles and pedestrians.

[0127] If the paths of vehicles and pedestrians can be defined by a specific function or if regression is possible, the accident risk can be calculated based on the known function.

[0128] If f1 and f2 are known path functions of two objects used to determine collision probability, the solution to the following function is obtained using the Newton-Raphson method.

[0129]

[0130] If a valid solution exists, check whether there is an intersection in spacetime between the two paths at the point where the solution exists; if a solution exists at an approximate time, it can be determined that a collision is occurring.

[0131] However, the above method cannot be used when waypoints are known but the shape of the path is unknown or not standardized—that is, for unknown paths, such as paths estimated by the IMM-KF algorithm. Therefore, in such cases, the collision risk is calculated by assuming that the distance between a waypoint and a neighboring waypoint is a line segment.

[0132] Figure 9 shows a diagram for determining the collision risk through the straight segment L1 of the vehicle and the straight segment L2 of the pedestrian at a specific point in time.

[0133] In Fig. 9, the cross circle represents the vehicle's way point, and the X circle represents the pedestrian's way point.

[0134] First, compare each straight line segment on the path of the two objects to check if there is an intersection point, and if the intersection point is in the same space-time, determine that a collision is expected.

[0135] In order for a collision to occur accurately based on a point mass, the viewpoints on the path points of both objects at the intersection must be completely identical; however, considering the uncertainty on the expected paths of both objects and the dimensions of the vehicle / pedestrian, it is desirable to apply a predetermined tolerance as shown in Equation 3 below.

[0136]

[0137] In the above formula, T(obj1) represents the viewpoint at the intersection for object 1, and T(obj2) represents the viewpoint at the intersection for object 2.

[0138] The method for checking whether there is an intersection between straight line segments on the path points of two objects involves sequentially swapping the straight line segments on the path points of one object based on that object and comparing them with the straight line segments of the other object to check for intersection.

[0139] A straight line segment for each object can be defined using a first-order Bezier Curve as shown in Equation 4 below.

[0140]

[0141] Next, the straight line segment at the point where the extensions of each straight line segment intersect is calculated using the following mathematical formula 5.

[0142]

[0143] If the values ​​of the parameters calculated through the above matrix are all within the range

[0001] , there will be an intersection point between the two straight line segments, and the intersection point can be calculated as follows.

[0144]

[0145] The time to collision (TTC) can be calculated as follows.

[0146]

[0147] The above formula assumes that the intersection point between straight line segments exists between the j-th path point and the j+1-th path point relative to a given object, and time_offset_collision represents the time it takes for the object to travel from the j-th path point to the intersection point.

[0149] Meanwhile, when there are many autonomous vehicles and regular vehicles within an intersection, the amount of computation increases proportionally. With existing prediction methods, the amount of computation required for risk assessment can increase exponentially as the number of regular vehicles increases. Additionally, existing prediction methods have a problem in that the predicted values ​​become bound within the road model as points on the road model are mixed.

[0150] Therefore, in the present invention, the prediction method is changed so that the next state is calculated using mixed input values ​​for road models in the current mixed state. Since this eliminates the need for separate Kalman filter calculations during the prediction period, the time complexity of the improved method can be significantly reduced as follows.

[0152] [Support for safe driving and economical driving services linked to adjacent intersections]

[0153] The action plan module (150) establishes an action plan for autonomous vehicles, non-autonomous vehicles, and pedestrians to prevent or avoid predicted risks. Prevention or avoidance includes recommending a change of route for autonomous vehicles and providing warnings to non-autonomous vehicle drivers and / or pedestrians.

[0154] Meanwhile, the action planning module (150) supports safe driving and economical driving services by linking with the signal systems of the current intersection and adjacent intersections.

[0155] The list of data provided to service users to support safe driving and economical driving services includes intersection situation information service data, intersection unexpected situation service data, intersection predicted route service data, and intersection economical driving service data.

[0156] The action plan and guidance information established by the action plan module (150) is provided to roadside devices, autonomous vehicles, non-autonomous vehicles / pedestrian smartphones through a communication unit (not shown).

[0157] Through this configuration, the future-type intersection integrated information server (100) according to the present invention can support safe driving and economical driving services as follows, depending on the data provided to the service user.

[0158] ① Clock malfunction collision risk warning

[0159] As illustrated in FIG. 10, when it is difficult for a general-purpose sensor mounted on an autonomous vehicle to recognize information about a surrounding vehicle at risk of collision due to obstacles or bad weather, the integrated information server (100) transmits information about the vehicle to the autonomous vehicle either directly or through a roadside device. As shown in the figure below, information about surrounding vehicles is collected from a detection device and transmitted to a fusion server, and the fusion server generates data on the expected path of a hidden vehicle and a collision risk warning and transmits it to the autonomous vehicle.

[0160] ② Fuel Efficiency Optimal Speed ​​Recommendation Scenario

[0161] The optimal fuel efficiency recommended speed for a vehicle is calculated and transmitted to a non-autonomous vehicle approaching an intersection by applying traffic flow theory to match the signal light phase of the adjacent intersection. As shown in Fig. 11, signal phase information is transmitted to a convergence server, and the convergence server generates data for the optimal fuel efficiency speed and transmits it to the non-autonomous vehicle.

[0162] ③ Concealment Protection of Sudden Pedestrians

[0163] In cases where it is difficult for the general-purpose sensor mounted on the autonomous vehicle to recognize information about pedestrians at risk of collision due to adverse weather conditions, surrounding parked vehicles, structures, or buildings, such as in the event of a collision risk accident due to poor visibility, the roadside device (or integrated server) transmits the information about the sudden pedestrian to the autonomous vehicle. As shown in Fig. 12, information about the sudden pedestrian is collected from the roadside sensor installed at the intersection and transmitted to the fusion server, and the fusion server transmits the information to the receiving autonomous vehicle to prevent accidents.

[0164] ④ Pedestrian crossing support

[0165] This is a service for preventing accidents that can easily occur when a pedestrian has difficulty recognizing surrounding dangerous situations due to bad weather or other obstacles in the walking environment, or when attention is reduced due to the use of a smart device while walking. As shown in Fig. 13, collision prediction information, such as approaching vehicles, vehicles violating traffic signals, and vehicles turning right in the same direction of travel, collected from roadside sensors or other autonomous / non-autonomous vehicles, is warned through the pedestrian's smart device.

[0166] Based on the above explanation, the operational invention of the intersection integrated information provision system according to the present invention is summarized as follows.

[0167] Spatial information including the location and attributes of an object is collected from an infrastructure video information integration platform, an autonomous vehicle terminal, and a smart device, and preprocessing is performed to estimate the location and velocity of the object from the received spatial information (S110).

[0168] Next, the data obtained in the preprocessing step is analyzed to define entities and remove information with low relevance in the entity refinement step (S120), and the relationship between entities determined in the entity refinement step is defined in the context recognition step (S130).

[0169] Based on the information generated in the object refinement step and the situation awareness step, a risk factor within the intersection is determined using the information (S140), and a situation judgment and action plan are established based on the information from at least one of the object refinement step, the situation awareness step, and the risk judgment step (S150).

[0170] Warnings, recommendations, or guidance are given to pedestrians / non-autonomous vehicle drivers and autonomous vehicles according to the established action plan (S160).

[0171] The configuration of the present invention has been described in detail above with reference to several embodiments. However, this is merely an example, and it is understood that various modifications and changes are possible within the scope of the technical concept of the present invention. Accordingly, the scope of the rights of the present invention should be determined by the description in the following claims.

Claims

Claim 1 It includes a preprocessor that estimates the location and velocity of objects from spatial information received from an infrastructure video information integration platform, an autonomous vehicle terminal, and a smart device; an object refinement module that analyzes data obtained from the preprocessor to define objects and removes information with low relevance; a situation awareness module that defines relationships between objects determined by the object refinement module; a risk judgment module that determines risk factors within an intersection using information based on the object refinement module and the situation awareness module; and an action plan module that establishes a situation judgment and an action plan based on information from at least one of the object refinement module, the situation awareness module, and the risk judgment module. The situation awareness module generates a Recursive Least Squares (RLS)-based linear path prediction model based on collected data history to predict a pedestrian path, and estimates and reflects a time parameter required to reach a steady-state speed at which a pedestrian begins to move from a stationary state and enters stable walking. The time parameter is calculated by a model equation having a linear relationship with the time required for the pedestrian's first step in the steady-state speed range of 2.5 m / s or less, and in the range exceeding 2.5 m / s An intersection integrated information system characterized by being applied in 1.55 seconds. Claim 2 An intersection integrated information system according to claim 1, further comprising a guidance module that guides the action plan established by the action plan module. Claim 3 A crossroads integrated information system according to claim 1, further comprising a database including an object DB that stores objects interpreted by a preprocessor and a fusion DB that stores a list of each risk and countermeasure. Claim 4 In claim 1, the object refinement module is an intersection integrated information system that fuses collected information to generate a region map and performs high-precision position correction for multiple objects. Claim 5 The intersection integrated information system according to claim 1, wherein the object refinement module converts the location recognition information of an infrastructure camera, the location information of an autonomous vehicle, and the location information of a smartphone into a single coordinate system and unifies the units, and then fuses each data by applying the location variance ratio of the data from the infrastructure camera, the autonomous vehicle, and the smartphone as a weight. Claim 6 delete Claim 7 delete Claim 8 The intersection integrated information system according to claim 1, wherein the situation awareness module receives current position, heading angle, steering angle, and speed information from the vehicle in the case of an autonomous vehicle to predict the path, and in the case of a non-autonomous vehicle, recognizes the position of the non-autonomous vehicle through a roadside camera, estimates the heading angle through a Kalman filter, determines a rate based on the curvature of the road, and predicts the path therefrom. Claim 9 An intersection integrated information system according to claim 1, wherein the risk judgment module determines a collision situation based on whether a solution exists for the difference between each path function of the two objects using the Newton-Raphson method when the paths of the two objects can be defined by a specific function. Claim 10 An intersection integrated information system according to claim 1, wherein the risk judgment module assumes that for two objects whose path points are known but whose paths are unknown, the distance between each path point and neighboring path points is a straight line segment, compares each straight line segment on the path between the two objects to determine if there is an intersection, and determines that a collision is expected if the intersection is in the same space-time. Claim 11 The intersection integrated information system according to claim 1, wherein the action plan module establishes an action plan including route change recommendations and danger warnings for autonomous vehicles, non-autonomous vehicles, and pedestrians to prevent or avoid predicted risks, and supports safe driving and economical driving services by linking with the signal systems of the current intersection and adjacent intersections. Claim 12 The method comprises: a preprocessing step for estimating the location and speed of an object from spatial information received from an infrastructure video information integration platform, an autonomous vehicle terminal, and a smart device; an object refinement step for defining objects by analyzing data obtained in the preprocessing step and removing information with low relevance; a situation awareness step for defining relationships between objects determined in the object refinement step; a risk judgment step for determining risk factors within an intersection using information generated in the object refinement step and the situation awareness step; and an action planning step for establishing a situation judgment and an action plan based on information from at least one of the object refinement step, the situation awareness step, and the risk judgment step. The situation awareness step includes: a step of determining the completion of a pedestrian's first step based on a change in the increase amount of the pedestrian awareness area box width; and a step of identifying a parameter for the time required to reach the pedestrian's normal speed by applying a linear relationship with the time required for the first step in a normal speed section of 2.5 m / s or less and applying 1.55 seconds in a section exceeding 2.5 m / s, based on the regression analysis results of actual measurement data. A method for operating an intersection integrated information system, comprising the step of performing pedestrian path prediction through an RLS-based linear path prediction model by reflecting the above-mentioned time parameters. Claim 13 delete Claim 14 delete Claim 15 delete Claim 16 delete Claim 17 delete Claim 18 delete

Images