Method and System for Estimating a Movement Path of a Mobile Object in Sensor Shadow Regions in a Heterogeneous Sensor Environment

Abstract

The present invention relates to a multi-sensor-based method and system for estimating the actual movement path of a moving object within a sensor shadow zone by utilizing as input intermittent and discontinuous observation information occurring in a plurality of heterogeneous sensor environments with different observation characteristics, such as CCTV, WiFi access points (AP), BLE beacons, GPS, and drones. The method and system for estimating the movement path of a moving object within a sensor shadow zone in a heterogeneous sensor environment according to an embodiment of the present invention comprises: a step of collecting sensor events regarding a moving object from a plurality of heterogeneous sensors that generate information regarding at least one of the position, existence, identification, or movement state of the moving object; a step of defining an unobserved interval between two different sensor events as a sensor shadow zone and calculating an entry point and an exit point of said shadow zone; a step of generating a set of multiple path candidates selectable by the moving object between said entry point and exit point; a time-series continuity-based candidate filtering step for removing physically impossible or unnatural path candidates from said path candidates using the time-series continuity of the moving object's speed, acceleration, or direction of movement; and a path remaining after said candidate filtering step. It is characterized by including the step of calculating a probability score (P-score) for each path candidate for each candidate, and the step of recalculating the probability scores (P-scores) of the path candidates and updating the optimal path whenever a new sensor event is received.

Description

Technology Field

[0001] The present invention relates to a path generation and selection technology for estimating and correcting the spatiotemporal path of a moving body.

[0002] More specifically, the present invention relates to a multi-sensor-based method and system for estimating the actual movement path of a moving object within a sensor shadow zone by utilizing as input intermittent and discontinuous observation information generated in a multi-type sensor environment with different observation characteristics, such as CCTV, WiFi access points (AP), BLE beacons, GPS, and drones.

[0003] The present invention relates particularly to a technology for generating a plurality of candidate movement paths that a moving body can select in sections where sensors are not installed or observation is impossible, based on the topology of an urban road network, and for each path candidate, integrating probabilistic characteristics based on sensor events, probabilistic characteristics based on road structure and movement constraints, and probabilistic characteristics based on the time-series continuity of the moving body to calculate a probability-based score, thereby updating the priority of the path candidates online and re-estimating the optimal path according to subsequently received sensor events.

[0004] In addition, the present invention belongs to a technical field for solving incomplete and discontinuous observation problems that occur in typical urban IoT sensor environments, such as non-homogeneous sensor placement within the city, missed observations, irregular detection patterns of WiFi AP combinations, limited field of view of CCTVs, degradation of GPS signals, and sporadic drone observations.

[0005] Furthermore, the present invention belongs to the field of path estimation and correction technology applicable to various industries and information processing fields, such as map and location-based services, smart city urban control systems, accuracy correction of autonomous driving and V2X data, swarm movement pattern analysis, modeling of pedestrian and vehicle movement within cities, and construction of a movement behavior analysis platform. Background Technology

[0006] In urban spaces, various platform-based sensors such as CCTVs, WiFi access points (APs), BLE beacons, GPS, and drones are being operated in large numbers and types.

[0007] Since these sensors differ in observation angle, observation period, positional precision, noise characteristics, and visibility conditions, they have inherent limitations in reliably obtaining continuous time-series position information (time-series trajectory) for the same moving object.

[0008] In particular, in areas where the sensor's detection range does not reach—that is, sensor shadow zones—the location of a moving object cannot be directly measured by any sensor, making it difficult to accurately estimate its actual movement path.

[0009] To address these issues, various location estimation and path reconstruction technologies using GPS, CCTV, WiFi, BLE, etc., have been proposed.

[0010] However, conventional technology has not been able to adequately reflect the discontinuity of sensor observations, unbalanced coverage, and complex road phase structures that occur in actual urban environments.

[0011] For example, while GPS-based continuous positioning technology can achieve a certain level of accuracy in open environments, its reliability drops sharply in urban environments due to signal shielding by buildings and multipath phenomena.

[0012] In addition, it is difficult to apply in indoor or underground spaces where GPS signals are not received, or in CCTV-centric surveillance environments, and there is a problem where the number of path candidates increases rapidly if observations are missed.

[0013] Furthermore, GPS signals alone are insufficient to adequately reflect road phase information, such as branching and merging structures, the possibility of turning, and speed limits.

[0014] Meanwhile, CCTV-based moving object tracking technology can generate time-series location information through object detection and re-identification within the video, but because the camera's field of view is limited, a problem occurs where the trajectory of the moving object is discontinuous in non-overlap environments.

[0015] In particular, in sections where a moving object disappears from one camera and reappears from another, the actual path taken cannot be determined, and the reliability of image-based re-identification technology also fluctuates significantly depending on changes in lighting, shooting angle, and resolution.

[0016] Near-field location estimation technologies using WiFi APs or BLE beacons can confirm the presence or approximate location of a moving object in the form of events; however, precise location calculation is difficult due to uneven installation spacing and detection ranges, as well as the sensitivity of RSSI-based distance estimation to environmental noise.

[0017] As a result, the time interval between events is large and the detection pattern appears irregular, and it is difficult to resolve path ambiguity occurring within the sensor blind spot with a single sensor alone.

[0018] Techniques have been proposed to correct the path of a moving object by combining heterogeneous sensors, but most rely on one-shot interpolation methods that directly calculate a single path based on the entry and exit points of the sensors.

[0019] This method has limitations in that it cannot structurally generate and manage multiple path candidates that may actually exist in sensor-blinded areas, and it fails to adequately reflect the branching, merging, and turning constraints of the road network.

[0020] Some technologies utilize GPS-based map-matching techniques, such as HMM or Viterbi algorithms. However, as these techniques are models designed based on the continuity of GPS signals, they fail to adequately reflect the characteristics of urban IoT environments, such as the rarity of WiFi AP combinations, the discontinuity of CCTV re-identification events, and the sporadic observations of drones.

[0021] Furthermore, it is difficult to apply the model as is when sensor types are changed or added, and since most are implemented using a batch processing method, they cannot provide an online structure to re-evaluate or update the path whenever a new sensor event occurs.

[0022] In addition, conventional technology often fails to adequately consider the time-series continuity of a moving object.

[0023] Existing filtering methods based on distance or average speed fail to account for changes in the rotation angle and acceleration of moving objects, as well as the naturalness of the velocity profile. This results in physically unrealistic path candidates remaining undetected, increasing the likelihood that mispaths deviating from the actual route will be selected as the final path.

[0024] Furthermore, in urban sensor network environments, sensor events such as WiFi events, gaps, CCTV re-identification, gaps again, and GPS single observations occur irregularly in chronological order, and each event imposes new constraints on path estimation.

[0025] Nevertheless, most existing technologies adopt a batch processing method that determines the path only once after all data has been collected, making it difficult to apply to fields requiring real-time or near-real-time path estimation, such as smart city control, abnormal behavior detection, and autonomous driving assistance.

[0026] As such, in real urban environments, the frequent occurrence of sensor blind spots, the existence of multiple candidate paths, differences in observation characteristics and reliability among sensors, and the time-series movement characteristics of moving objects have a decisive influence on path selection.

[0027] However, conventional technology fails to provide a structure capable of generating and evaluating multiple path candidates by comprehensively considering these elements, and continuously updating the path whenever a new sensor event occurs.

[0028] Therefore, to effectively solve the sensor blind spot problem in an urban IoT-based mobile tracking environment, a new mobile path estimation technology is required that structurally defines sensor blind spots, generates multiple path candidates based on road topology information, evaluates path candidates by considering the characteristics and reliability of heterogeneous sensor events together with the time-series continuity of the mobile, and updates path probabilities online whenever a new sensor event occurs.

[0029] As a technique for estimating the path of moving objects in urban environments, numerous location estimation and path correction technologies utilizing various sensors such as CCTV, WiFi, and GPS have been proposed.

[0030] These conventional technologies aim to generate or correct trajectories using observation information of moving objects obtained from sensors, but they have limitations in that they do not sufficiently reflect the characteristics of actual urban environments where sensor blind spots frequently occur.

[0031] For example, Korean Registered Patent Publication No. 10-2587427 proposes a technology that corrects the movement path of a specific object by fusing the trajectories of a moving object collected from heterogeneous sensors such as CCTV and WiFi.

[0032] This document discloses a structure that searches for a set of similar trajectories by comparing already generated moving object trajectories for each sensor and corrects the final trajectory through spatiotemporal alignment.

[0033] However, this technology does not include the concept of structurally generating multiple path candidates that a moving object can select in sensor-blinded areas, nor does it have a path candidate generation process itself to estimate which path the moving object actually selected between two sensor observation points.

[0034] Furthermore, path generation using road topological information such as the link-node structure of the road network, turnability, and speed limits is not performed, and a model considering sensor-specific probabilistic characteristics such as the rarity of WiFi AP combinations, RSSI patterns, and CCTV re-identification events is not presented.

[0035] Furthermore, the technology is implemented in a batch processing method where the corrected trajectory is determined once, and thus fails to provide an online update structure that re-evaluates or updates the path based on subsequent sensor events.

[0036] Korean Patent Publication No. 10-2024-0107638 proposes a technology for interpolating the position of a moving object within a sensor blind spot using CCTV, GPS, and IoT sensors.

[0037] This document discloses a structure that generates a single interpolated path using movement speed and distance, based on information regarding the entry and exit points of a moving object from a shaded area.

[0038] However, this technology adopts a method that relies on one-shot interpolation, which assumes a single path, without considering that multiple possible movement paths may exist within the shaded area.

[0039] Accordingly, the generation of multiple path candidates based on road topology information, consideration of path branching and merging structures, and reflection of turning constraints are not taken into account.

[0040] Furthermore, the technology lacks a sensor event-based probability score combination structure, such as the rarity of WiFi AP combinations, BLE event timing, and CCTV re-identification information, and does not provide an online update function capable of renewing the path even when new sensor events are received.

[0041] Meanwhile, research has also been conducted on various map-matching techniques for aligning GPS points with road networks. These techniques disclose structures that generate candidate road segments, calculate transition and emission probabilities, and then select an optimal path using HMM, the Viterbi algorithm, Random Forest, or hybrid machine learning techniques.

[0042] However, as these technologies are models designed based on the continuity of GPS signals, they fail to adequately reflect the characteristics of intermittent and atypical sensor events in urban IoT environments, such as the rarity of WiFi AP combinations, limited CCTV field of view, and BLE events.

[0043] Furthermore, most map-matching technologies are implemented using a batch processing method, so they do not provide an online update structure that recalculates the path whenever a new sensor event occurs, and they do not actively utilize time-series continuity, such as changes in the rotation angle of the moving object, acceleration, and the naturalness of the velocity profile, during the candidate filtering stage.

[0044] Furthermore, while it can handle temporary interruptions in GPS signals, it does not include the concept of sensor shadow zones, which are structurally defined sections where no sensors exist at all.

[0045] As such, conventional technologies focus on simple fusion of trajectories between sensors, single-path interpolation, or GPS-based map matching, and thus fail to fundamentally solve the problem of path ambiguity occurring in sensor blind spots.

[0046] In particular, a technical structure that structurally generates multiple movement path candidates, evaluates path candidates by integrally considering the characteristics of sensor events, road phases and structural constraints, and the time-series continuity of the moving object, and re-estimates the path whenever a new sensor event occurs has not been presented in conventional technology.

[0047] Therefore, a new vehicle path estimation technology is required that structurally defines sensor blind spots, evaluates and selects multiple path candidates by combining heterogeneous sensor events, road phase information, and time-series movement characteristics, and continuously updates the path online. Prior art literature

[0048] Korean Registered Patent Publication No. 10-2587427 (Registration Date: October 05, 2023, Title: System for Generating Location and Movement Trajectory of a Moving Object) Korean Published Patent Publication No. 10-2024-0107638 (Publication Date: July 09, 2024, Title: Local Server and Method for Controlling Road Traffic) The problem to be solved

[0049] Accordingly, the problem to be solved by the present invention is to provide a method and system for estimating a vehicle's path, which, in order to estimate the actual movement path of a vehicle within a sensor shadow zone, generates multiple path candidates by integrally reflecting phase information of an urban road network, observation events collected from heterogeneous sensors, and the time-series movement characteristics of the vehicle, probabilistically evaluates the probability of each path candidate, and determines the final path by re-updating the scores of the path candidates according to subsequently received sensor events.

[0050] To this end, the present invention aims to provide a probability-based multi-path path estimation technique that implicitly performs identification of the same object between heterogeneous sensors by probabilistically combining the suitability of sensor events, the suitability of road structures, and the time-series continuity of moving objects, and calculating the probability that two sensor observations can be explained as a continuous trajectory of the same object.

[0051] Specifically, the present invention structurally generates a plurality of possible path candidates within a sensor-blinded section based on road phase information, evaluates the probability of each path candidate through a probabilistic model integrating sensor events, road structure, and time-series information, and performs time-series continuity-based filtering to remove physically impossible or unnatural paths in advance.

[0052] In addition, the present invention aims to implement a probabilistic and modular path estimation technology capable of naturally integrating heterogeneous sensors by providing an online update structure capable of updating the optimal path by recalculating the probability of path candidates whenever a new sensor event occurs. means of solving the problem

[0053] A method for estimating a movement path within a sensor shadow zone of a moving object in a heterogeneous sensor environment according to an embodiment of the present invention for achieving the aforementioned purpose comprises: a step of collecting sensor events regarding a moving object from a plurality of heterogeneous sensors that generate information regarding at least one of the position, existence, identification, or movement state of the moving object; a step of defining an unobserved section between two different sensor events as a sensor shadow zone and calculating an entry point and an exit point of said shadow zone; a step of generating a set of multiple path candidates selectable by the moving object between said entry point and exit point; a time-series continuity-based candidate filtering step for said path candidates, which removes physically impossible or unnatural path candidates by utilizing the time-series continuity of the moving object's speed, acceleration, or direction of movement; a step of calculating a probability score (P-score) for each path candidate remaining after said candidate filtering step; and a step of recalculating the probability scores (P-scores) of said path candidates and updating the optimal path whenever a new sensor event is received.

[0054] The above heterogeneous sensor is characterized as being one of a CCTV, WiFi AP, BLE beacon, GPS, drone, WiFi scanner, or Lidar.

[0055] The step of generating the above-mentioned set of multiple path candidates is characterized by being performed using a road graph containing topology information of an urban road network.

[0056] The step of generating the above-mentioned set of multiple path candidates is characterized by being performed by reflecting road phase constraints including at least one of whether it is rotatable, speed limit, and whether it is one-way.

[0057] The step of calculating a probability score (P-score) for each of the above path candidates is characterized by calculating the probability score (P-score) for each path candidate by reflecting at least two of the following elements: a sensor-based probability element (Psensor) representing the probability of occurrence of a sensor event, a road-based probability element (Proad) representing the suitability of road structure and movement constraints, and a time-series-based probability element (Pseries) representing the time-series continuity of the speed, acceleration, or direction of movement of the moving object.

[0058] The above Psensor is characterized by including at least one of the rarity of WiFi AP combinations, RSSI patterns, CCTV re-identification time, or BLE detection time.

[0059] The above road-based probability element (Proad) is characterized by being based on road network information including at least one of the road's node and link connection relationships, turnability, speed limit, number of lanes, one-way traffic status, and road congestion.

[0060] The above time series-based probability elements (Pseries) are characterized by being calculated based on at least one of the amount of change in direction of movement, the continuity of the velocity profile, or the amount of change in acceleration.

[0061] The step of updating the optimal path described above is characterized by including an online update mechanism that recalculates probability scores (P-scores) for the top K path candidates and selects the optimal path when a new sensor event occurs.

[0062] The above new sensor event is characterized by including at least one event among WiFi detection, CCTV re-identification, BLE detection, and GPS signal input.

[0063] A system for estimating the movement path of a moving object within a sensor shadow zone in a heterogeneous sensor environment, which is another embodiment of the present invention, comprises: a sensor data collection unit that collects sensor events regarding the moving object from a plurality of heterogeneous sensors that generate information regarding at least one of the position, existence, identification, or movement state of the moving object; a shadow zone detection unit that defines an unobserved section between two different sensor events as a sensor shadow zone and calculates an entry point and an exit point of the sensor shadow zone using the time and location information of the occurrence of the sensor events; a road topology information storage unit that stores a road graph including topology information of an urban road network; a path candidate generation unit that generates a set of multiple path candidates selectable by the moving object using the road graph between the entry point and the exit point; a time-series continuity-based candidate filtering unit that removes physically impossible or unnatural path candidates using the time-series continuity of the moving object's speed, acceleration, or direction of movement for the plurality of path candidates; and a probability score calculation unit that calculates a probability score (P-score) for each path candidate that is not removed by the candidate filtering unit. It is characterized by including an online update unit that recalculates the probability score (P-score) of the path candidates and updates the optimal path whenever a new sensor event is received.

[0064] The above path candidate generation unit is characterized by reflecting road phase constraints including at least one of whether it is rotatable, speed limit, and whether it is one-way.

[0065] The probability score calculation unit above calculates a sensor-based probability element (Psensor) indicating the probability of occurrence of a sensor event, a road-based probability element (Proad) indicating the suitability of road structure and movement constraints, and a time-series-based probability element (Pseries) indicating the time-series continuity of the speed, acceleration, or movement direction of a moving object, respectively, and calculates a probability score (P-score) for each path candidate by reflecting at least two of the calculated probability elements.

[0066] The above online update unit is characterized by storing the probability score of an existing path candidate in memory and updating the stored probability score upon receiving a new sensor event.

[0067] Specific details of other embodiments are included in the detailed description and drawings. Effects of the invention

[0068] According to the present invention, by integrating discontinuous and unstructured observation information collected from heterogeneous sensors in sensor-blinded sections to generate multiple movement path candidates based on topological information of an urban road network, and by evaluating and updating the path according to a probabilistic model that combines sensor information, road structure information, and time-series characteristics of the moving object, various problems of moving object path estimation that conventional technology could not solve can be effectively improved.

[0069] Specifically, unlike conventional interpolation methods that assume a single path in sensor-blinded areas, the present invention structurally generates all possible candidate travel paths between a starting point and an ending point based on road topology information. Accordingly, complex urban road structures such as branching, merging, intersections, and U-turns are naturally reflected, and a large number of candidate paths with a high probability of actual travel are secured. As a result, compared to conventional methods that do not consider the structural characteristics of blinded areas, the accuracy of path selection is significantly improved and the cumulative error is reduced.

[0070] Furthermore, the present invention can utilize sensor-specific unstructured information, such as the rarity of WiFi AP combinations, RSSI patterns, CCTV re-identification timing, and BLE proximity events, by modeling it as probabilistic elements for path evaluation. These sensor event-based probabilistic elements act as constraints that strongly exclude or prioritize specific path candidates, thereby significantly improving the ability to distinguish between path candidates. In particular, even for sensors such as WiFi APs that do not directly provide coordinate information, the detection area itself acts as a strong spatial constraint, providing the effect of effectively distinguishing multiple path candidates even in a single-sensor environment.

[0071] Furthermore, the present invention performs time-series continuity-based filtering by actively utilizing time-series continuities, such as changes in the movement direction of a moving object, speed profiles, acceleration patterns, and the feasibility of turning suitable for road structures, to pre-eliminate physically impossible or unnatural path candidates. As a result, abnormal or non-physical path candidates are removed at an early stage, reducing the amount of computation required for subsequent probability score calculation and ensuring the physical naturalness of the finally selected path.

[0072] Furthermore, the present invention provides an online update structure that re-evaluates path candidates and updates probability scores whenever a new sensor event occurs. Accordingly, while maintaining the top candidate paths, the priority of the path can be immediately readjusted when new observation information, such as WiFi detection, CCTV re-identification, BLE events, or GPS reception, is input. Through this structure, the present invention enables real-time or near-real-time path estimation, and constraints resulting from sensor events are immediately reflected, thereby continuously improving the accuracy of path estimation. Additionally, the present invention provides particularly advantageous effects in environments requiring large-scale real-time data processing, such as smart city control, swarm monitoring, and autonomous driving assistance systems.

[0073] Since the present invention selects a path based on a probabilistic model that integrates sensor information, road phase information, and time series information, the criteria for path selection are clear and reproducible. Unlike existing methods that rely excessively on specific sensor signals or are based on empirical rules, it is possible to implement a consistent system in which the same output is guaranteed for the same input by a mathematical and probabilistic model. Accordingly, the present invention ensures the reliability and objectivity of the system and provides scalability that can flexibly respond to the addition of sensors or changes in the environment.

[0074] Furthermore, since the present invention structures heterogeneous sensor data into modular probabilistic elements such as Psensor, Proad, and Pseries and processes them within a single framework, it can be easily expanded without the need to redesign the entire system even when new sensors or external information are added. For example, LiDAR detection events can be added to sensor-based probabilistic elements, road congestion information can be reflected in road-based probabilistic elements, and movement speed patterns can be reflected in time-series-based probabilistic elements.

[0075] Consequently, the present invention can maintain the stability and accuracy of moving object path estimation even in urban environments with high sensor density and complexity, and provides the effect of structurally improving the problems of path discontinuity and mispath selection inherent in existing technologies by mutually compensating for uncertainties between sensors and strengthening time-series consistency. Brief explanation of the drawing

[0076] Figure 1 is a schematic diagram showing the detection zone of a sensor placed within an urban space and an example of road information included within the detection zone. Figure 2 is an example of a case in which the movement trajectory of a moving object is detected using a CCTV sensor installed in an urban space. Figure 3 illustrates a method for estimating the movement path of a moving object within a sensor blind spot in a heterogeneous sensor environment. Figure 4 is an example diagram showing multiple path candidates that a moving body can select within a sensor shaded area in the form of a topological relationship graph of a road network. Figure 5 is a schematic diagram showing an example of a topological relationship graph modeled as a graph structure of an urban road network. Figure 6 shows a table comparing probability scores (P-scores). FIG. 7 is an example diagram showing the section directly observed by the sensor, the sensor shaded section that is not observed, and the possible movement path candidates within the shaded section during the process of a moving object moving in a heterogeneous sensor environment. Figure 8 shows a system for estimating the movement path of a moving object within a sensor blind spot in a heterogeneous sensor environment. Specific details for implementing the invention

[0077] The following detailed description of the present invention refers to the accompanying drawings, which illustrate specific embodiments in which the present invention can be practiced in order to clarify the objects, technical solutions, and advantages of the present invention. These embodiments are described in sufficient detail to enable a person skilled in the art to practice the present invention.

[0078] Furthermore, throughout the detailed description and claims of the invention, the word "comprising" and its variations are not intended to exclude other technical features, additions, components, or steps. Other objects, advantages, and characteristics of the invention will become apparent to a person skilled in the art, partly from this description and partly from the practice of the invention. The following examples and drawings are provided as examples and are not intended to limit the invention. Moreover, the invention encompasses all possible combinations of the embodiments shown herein. It should be understood that various embodiments of the invention differ from one another but need not be mutually exclusive. It should also be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. Accordingly, the following detailed description is not intended to be taken in a limiting sense, and the scope of the invention is limited only by the appended claims, including all equivalents to those claimed therein, provided they are properly described.

[0079] Meanwhile, unless otherwise indicated in this specification or clearly contradictory to the context, items referred to in the singular encompass the plural unless otherwise required by the context. Furthermore, in describing the present invention, if it is determined that a detailed description of related known configurations or functions could obscure the essence of the invention, such detailed description is omitted.

[0080] A system or method for estimating a path in a moving object's shaded area according to one embodiment of the present invention may be implemented in a server environment, a local computing environment, an edge device, or a combination thereof, and may be configured by combining a graph search algorithm, a probability calculation module, a sensor data parser, etc., and may also be implemented as a single software module or a distributed computing structure.

[0081] The following describes in detail how the present invention is actually implemented and operated, focusing on preferred embodiments of the invention. However, this is merely an example to aid in understanding the invention, and the scope of the invention is not limited thereto.

[0082] Figure 1 is a schematic diagram showing the detection zone of a sensor placed within an urban space and an example of road information included within the detection zone.

[0083] In the upper part of Fig. 1, a city area with multiple CCTV sensors installed is used as an example, and the detection range of each CCTV is visualized as a circular or fan-shaped area and displayed superimposed on a city map.

[0084] As such, each sensor has a unique installation location and field of view or detection radius, and the detection zones of each sensor may overlap or be spaced apart from one another.

[0085] In the upper example of Fig. 1, each sensor detection area refers to the spatial range in which a moving body can be observed by the corresponding sensor, and the area outside the detection area corresponds to an area where direct observation is not made by the corresponding sensor.

[0086] In the present invention, based on the spatial distribution of such sensor detection zones, the period from when a specific moving object leaves the detection zone of one sensor until it enters the detection zone of another sensor is defined as a sensor shaded area.

[0087] That is, as illustrated in FIG. 1, the non-overlapping area formed between the multiple sensor detection zones forms a shaded area where direct observation of the moving object is impossible.

[0088] In the lower part of Fig. 1, a road network within an urban area where the sensor detection zone overlaps, and an example of the movement trajectory of a moving object detected based on CCTV are shown together.

[0089] In the diagram below, roads are represented as a set of line segments in the form of links, and each link has a structure connected by nodes such as intersections, junctions, or junctions.

[0090] In addition, in some road sections, the actual movement trajectory of the moving object is displayed in the form of a line, exemplifying the time-series movement path of the moving object observed within the sensor detection area.

[0091] As can be seen from Figure 1, the location or movement path of a moving object can be observed relatively clearly within the sensor detection area, but in the shaded area between the detection areas, it is not possible to directly determine which road the moving object traveled along.

[0092] In particular, since urban road networks include multiple branching, merging, and intersection structures, there may be multiple routes that a moving object can choose from, even if the starting and ending points of a shaded section are the same.

[0093] Based on the spatial distribution of sensor detection zones as illustrated in FIG. 1 and road phase information included within the detection zones, the present invention sets the start and end points of sensor shadow sections and structurally generates a plurality of path candidates selectable by a moving body between them on a road network.

[0094] Through this, the present invention aims to resolve the problem of path ambiguity caused by the disconnection of sensor detection zones, and subsequently to more accurately estimate the actual movement path of a moving object by applying probability-based evaluation and time-series constraints.

[0095] FIG. 2 is an example of a case in which the movement trajectory of a moving object is detected using CCTV sensors installed in an urban space. As shown in FIG. 2, each CCTV has a fixed installation position and a limited field of view, and can continuously observe the position or movement status of the moving object only within that field of view.

[0096] Figure 2 illustrates an example in which, when a moving object enters a CCTV detection zone and is continuously observed for a certain period of time, the change in position of the moving object is recorded in the form of a trajectory in chronological order.

[0097] At this time, the detection trajectory of the moving object can be generated based on frame-by-frame object detection results or re-identification results, and each trajectory represents the time-series movement path while the moving object stays within the detection area of ​​the CCTV.

[0098] As can be seen from the example illustrated in Figure 2, since the detection area of ​​the CCTV is spatially limited, the trajectory of the moving object ends the moment the moving object moves out of the CCTV's field of view.

[0099] Subsequently, during the period until the moving object enters the detection zone of another CCTV and is re-observed, direct observation of the object's location or movement path is not made.

[0100] As such, unobserved sections existing between CCTV detection zones form sensor blind spots, and it is impossible to determine the actual path a moving object took in these sections based solely on CCTV information.

[0101] In addition, in environments where multiple CCTVs are installed, the detection zones of each CCTV often have a non-overlap structure that does not overlap with one another, so the trajectory of a moving object is observed in a disconnected form at the individual CCTV level.

[0102] As a result, as shown in FIG. 2, the detection trajectory for the same moving object can be separated into multiple partial trajectories, and it is difficult to determine the connection relationship between these partial trajectories based solely on CCTV information.

[0103] The present invention is intended to structurally define sensor blind spots existing between the trajectories of a moving object detected by different CCTVs, taking into account the discontinuity characteristics of such CCTV-based moving object detection trajectories, and to generate multiple path candidates that a moving object can select within the blind spots based on road topology information.

[0104] Figure 3 illustrates a method for estimating the movement path of a moving object within a sensor blind spot in a heterogeneous sensor environment.

[0105] Hereinafter, with reference to FIG. 3, a method for estimating the movement path of a moving object within a sensor shaded area in a heterogeneous sensor environment according to an embodiment of the present invention will be described.

[0106] First, a step (step S110) of collecting sensor events regarding a moving object from a plurality of heterogeneous sensors that generate information regarding at least one of the position, existence, identification, or movement state of the moving object is performed. In step S110, data (sensor events) are collected from several sensors of different types. The sensor data according to the present invention has the following structure.

[0107] attribute explanation sensor_id Unique sensor identifiers such as CCTV, WiFi, and BLE timestamp Time of sensor event occurrence location Sensor or object re-identification (Re-ID) based coordinates metadata RSSI, frame number, signal strength, Re-ID score, etc.

[0108] (Structure of sensor data)

[0109] The data collected in the above S110 step may include not only sensor event data, but also, if necessary, data regarding the sensor ID, time of sensor event occurrence, sensor coordinates, and additional metadata.

[0110] In the sensor data collection step, sensor events regarding a moving object are collected from a plurality of heterogeneous sensors that generate information regarding at least one of location, presence, identification, or movement status, such as CCTV, WiFi access point (AP), BLE beacon, GPS, and drone.

[0111] The above heterogeneous sensor may be composed of one of CCTV, WiFi AP, BLE beacon, GPS, drone, WiFi scanner, or Lidar, and it goes without saying that such heterogeneous sensor is merely an example and may be configured to include other types of sensors.

[0112] These sensors provide information regarding the location, presence, identification, or movement status of a moving object, and by using these sensors, the path of the moving object can be estimated more accurately and reliably within sensor blind spots.

[0113] Each sensor event may include a sensor identifier, the time of the event, sensor or observation location information, and additional metadata such as RSSI, frame number, and re-identification confidence. These sensor events are subsequently used as key input data throughout the entire path estimation process.

[0114] The above-mentioned moving objects refer to various forms of moving targets, such as objects, people, and vehicles, and are not necessarily limited to the moving objects exemplified.

[0115] Each of the following sensors collects data in a unique way, and by combining this data, it is possible to estimate the movement path more accurately.

[0116] - CCTV (Closed-Circuit Television):

[0117] It is a closed-circuit television system that monitors specific points within a space to provide real-time video, and is useful for verifying the location and presence of objects. Data collected from CCTV sensors includes video frame-based object detection and tracking results.

[0118] In one embodiment of the present invention, CCTV data generates time-series position coordinates of a moving object by applying an object tracking algorithm to consecutive video frames, and can assign a unique object identifier (ID) to each object.

[0119] In addition, a Re-ID technique may be applied to identify the same moving object among multiple CCTVs; in this process, a Re-ID probability value or confidence score indicating the reliability of the Re-ID result is calculated. This Re-ID confidence serves as an important input value in the subsequent process of calculating sensor-based probability components.

[0120] - WiFi AP (Access Point): / BLE Beacon (Bluetooth Low Energy Beacon):

[0121] A Wi-Fi AP is a wireless network access point that can determine whether a target is connected to a Wi-Fi network or has approached the vicinity, and can also estimate the approximate location based on connection strength.

[0122] A BLE Beacon is a beacon that emits low-power Bluetooth signals to detect the presence of an object within a certain range, and is useful for determining the accurate location within a relatively small range.

[0123] Data collected from WiFi or BLE sensors takes the form of discontinuous, event-based data rather than continuous location coordinates. For example, when a mobile object enters the detection range of a WiFi access point (AP) or BLE beacon, the identifier of the corresponding AP or beacon, the time of detection, the signal strength (RSSI), and combined information regarding simultaneously detected APs or beacons are collected in the form of an event.

[0124] In the present invention, such events can be interpreted as spatial area information rather than point coordinates, and, if necessary, a radius or area where a moving object is likely to exist can be estimated based on signal strength. This area-based information is combined with road phase information and utilized to evaluate path candidates.

[0125] - GPS (Global Positioning System) / Drone:

[0126] GPS is a system that determines the precise location of a target using satellite signals, providing location information with high accuracy over a wide range. Drones can cover large areas and are useful for identifying the interrelationships of moving targets.

[0127] Data collected from GPS sensors or drone-based sensors may include location information measured at regular intervals or on a one-time basis.

[0128] In the case of GPS data, since positional accuracy can vary depending on the satellite signal reception environment, reliability values ​​such as horizontal accuracy degradation indicators (e.g., HDOP) may be provided. For drone-based data, one-time position information may be provided as a result of observing a moving object from the air at a specific point in time, and the time information and spatial resolution of such observation may also be included as metadata.

[0129] In the present invention, path estimation errors caused by data with large positional errors are prevented by adjusting the influence of GPS or drone data by considering such reliability information.

[0130] - WiFi Scanner:

[0131] It is a device that tracks a target by scanning WiFi signals and can estimate its location by interacting with various surrounding WiFi devices.

[0132] - Lidar (Light Detection and Ranging):

[0133] It is a device that uses a laser to recognize surrounding three-dimensional structures, and can recognize objects in detail and measure distances.

[0134] Next, the unobserved interval between two different sensor events is defined as a sensor shadow zone, and a step (step S120) of calculating the entry and exit points of the shadow zone is performed.

[0135] Here, the 'sensor shadow zone' refers to the interval between different sensor events that is not observed by the sensor.

[0136] In the method for estimating a moving object's path according to the present invention, first, sensor shaded sections are structurally defined using observation information collected from a plurality of heterogeneous sensors.

[0137] Specifically, the observable areas of two different sensors are geometrically modeled. Specifically, the entry point of the shaded area is set based on the time and location where the moving object was last observed by Sensor A, and the exit point of the shaded area is set based on the time and location where the moving object was first re-observed by Sensor B. The time interval between the two sensor events is defined as the sensor shaded area.

[0138] The present invention enables adaptation to various sensor environments by flexibly modeling the Entry and Exit as points or regions according to sensor characteristics. For example, in the case of a sensor that provides the location of an object using relatively precise coordinates, such as a CCTV sensor, the starting point or the ending point can be defined as a single point in two-dimensional or three-dimensional space.

[0139] On the other hand, for area-based sensors with a detection radius, such as WiFi access points (APs), it is difficult to calculate the exact coordinates of a moving object at a specific point in time; therefore, the starting or ending point is defined as a possible region that includes the detection range of the sensor.

[0140] As such, the present invention can naturally absorb differences in precision of location information collected from different sensors by flexibly modeling the starting point and the ending point as points or regions according to the spatial resolution and observation characteristics of the sensor.

[0141] Subsequently, path candidates generated are evaluated based on whether they pass through a point if the start and end points are points, and selected based on whether they can pass through a possible region if they are regions.

[0142] At this time, metadata such as the observation time per sensor, sensor location coordinates, and reliability may be considered together. The sensor shaded area defined in the present invention may be represented in a two-dimensional or three-dimensional space as a structure including start point information, end point information, a time interval between the two times, and sensor-specific detection range information.

[0143] Through this, shaded intervals function not merely as temporal gaps, but as key input elements for path estimation where both spatial and temporal constraints are considered.

[0144] Next, a step (step S130) is performed to generate a set of multiple path candidates that a moving body can select between the starting point and the ending point. In this step, several possible paths are generated as candidates using topological information of the city road network to determine which paths a moving body can select when moving on the road network.

[0145] A road network is represented in the form of a graph that includes the connection structure of nodes and links, whether left and right turns are possible, conditions prohibiting U-turns, speed limits, the number of lanes, whether it is one-way, and entry and exit restrictions based on the urban structure.

[0146] In the method for estimating a moving object's path according to the present invention, by utilizing such road topology information and applying an A* algorithm, BFS, Dijkstra, or DAG-based search technique, a plurality of path candidate sets that a moving object can actually select within a shaded section are generated.

[0147] Each path candidate is represented as a sequential set of road nodes or links. Through this, the present invention can secure a structural set of candidate paths, unlike existing techniques that rely on a single interpolation value.

[0148] The step of generating the above multiple path candidate sets can be performed using a road graph containing topology information of the city road network.

[0149] In other words, when generating multiple candidate paths for a moving object to travel within a shaded area, instead of simply listing all possible paths, topological information of the city's road network is utilized. For instance, if roads branch off in multiple directions from a specific intersection, each of these roads can serve as a potential path.

[0150] 'Topological information' refers to structural information about roads, including how they are connected, such as points where roads intersect or diverge, one-way roads, and speed limits.

[0151] A road graph represents the road structure of a city in the form of a graph (Node / Link). By expressing the relationships between roads—such as intersections, junctions, one-way streets, and turning points—in a graph format, it facilitates calculations and route search.

[0152] Figure 4 is an example diagram showing multiple path candidates that a moving body can select within a sensor shaded area in the form of a topological relationship graph of a road network.

[0153] The circular nodes (N) shown in Fig. 4 represent road nodes such as intersections, junctions, or junctions, and the line segments connecting each node represent road links where actual vehicles can travel. These nodes and links are represented in the form of a graph, reflecting the connection structure of the urban road network.

[0154] On the left side of Fig. 4, the point where the moving body is last observed by the sensor is indicated as the entry point of the shaded section, and on the right side, the point where the moving body is subsequently observed again by the sensor is indicated as the exit point of the shaded section.

[0155] It shows that multiple road nodes and links exist between the aforementioned starting point and ending point, so the path through which the moving object can actually travel is not limited to a single path but branches out into multiple paths.

[0156] In FIG. 4, different path candidates connecting the same starting and ending points are illustrated as candidate path C1, candidate path C2, and candidate path C3, respectively. Each candidate path consists of a path sequence in which multiple road nodes and road links are sequentially connected, and may have a structure in which they share some sections or branch and merge at specific sections.

[0157] For example, some road links are included in common across multiple candidate routes, while others are included only in specific candidate routes, forming structural differences between each candidate route.

[0158] Additionally, in Fig. 4, links indicated by solid lines represent road sections included in the corresponding candidate path, while links indicated by dotted lines represent road sections included in other candidate paths or alternatively selectable. This visually demonstrates that there are multiple combinations of paths that a moving object can select within the same road network structure.

[0159] Figure 4 illustrates the feature of the present invention, which structurally generates multiple path candidates based on road phase information, without assuming the actual movement path of a moving object within a sensor shaded area as a single path.

[0160] The present invention comprehensively evaluates sensor event suitability, road structure suitability, and time series continuity information for each candidate path generated as shown in FIG. 4, and selects the path most likely to have actually moved as the final path.

[0161] Figure 5 is a schematic diagram illustrating an example of a topological relationship graph modeling an urban road network as a graph structure. As shown in Figure 5, a road network within urban space can be represented in the form of a graph in which multiple intersections, junctions, or merging points are defined as nodes, and road sections connecting these nodes are defined as links. Each node represents a road intersection or connection point, and each link represents a road section where an actual moving object can travel.

[0162] In the present invention, a graph of the topological relationship of such a road network is used to generate multiple path candidates that a moving object can select within a sensor-blinded section. In the graph example illustrated in FIG. 5, a complex road network is visually represented, including a structure connecting a single starting node to multiple branching nodes, a structure where nodes rejoin, and multiple path combinations. This demonstrates that there may be multiple paths that a moving object can select even between the same starting and ending points.

[0163] Each path candidate C according to the present invention i is defined as a sequential set of multiple road elements that are continuously connected on the above road network topological relationship graph. Specifically, one path candidate C i Is

[0164] C i = (P1, P2, …, P m )

[0165] It has the form of, where each element P j corresponds to a node or link that constitutes a road network graph. That is, a path candidate is represented as a sequence in which intersection nodes and road links are connected in a temporal and spatial order.

[0166] The path candidates defined in this way faithfully reflect road topological information, such as the actual connection relationships of the road, whether it is turnable, whether it is one-way, and branching and merging structures. Therefore, the path candidates generated based on the graph structure as shown in Fig. 5 are limited to paths where a moving object can physically move, and unrealistic paths unrelated to the road network structure are excluded.

[0167] Based on this path candidate structure, the present invention determines whether each candidate can connect the start and end points of a sensor-blinded section, and then evaluates the validity of each path candidate by applying sensor-based probability elements, road-based probability elements, and time-series continuity-based probability elements.

[0168] Through this, the candidate that best matches the actual movement path of the moving object can be effectively selected from among multiple path candidates generated on the topological relationship graph of the road network.

[0169] The step of generating the above set of multiple path candidates can be performed by reflecting road topology constraints including at least one of whether it is turnable, speed limit, and whether it is one-way. That is, to reflect realistic road conditions when generating multiple path candidates, the following road conditions may be considered.

[0170] - Rotation capability: To determine if the moving object can rotate in a specific direction, check for one-way roads or no-turn zones.

[0171] - Speed ​​Limit: Consider the speed limit of each road section and ensure that the excess speed is not entered into the calculation.

[0172] - One-way status: Check if the road is one-way to exclude travel routes in opposite directions.

[0173] Through this, realistic and feasible path candidates that a moving object can actually explore are generated, and each path candidate consists of a route that does not violate mobility or road usage regulations.

[0174] Next, for the plurality of path candidates, a time-series continuity-based candidate filtering step (step S140) is performed to remove physically impossible or unnatural path candidates by utilizing the time-series continuity of the speed, acceleration, or direction of movement of the moving body.

[0175] In step S140, unrealistic path candidates are eliminated using heading change amounts and speed and acceleration profiles. That is, under the premise that the moving object moves physically naturally, physical impossible or unnatural path candidates are eliminated in advance by analyzing the continuity of the change in direction of movement, the gentleness of the speed profile, the naturalness of changes in acceleration and deceleration, the presence of excessive turns, and whether the direction of movement is inconsistent with the road structure.

[0176] This allows candidates with a significantly low probability of actual movement to be removed from the set of path candidates in the initial stage, thereby improving the efficiency and accuracy of subsequent probability calculations. Here, time series continuity means that the movement of the object must be continuous over time.

[0177] In a path estimation method according to one embodiment of the present invention, a candidate filtering step based on time-series continuity is performed to remove candidates that do not correspond to the actual physical movement characteristics of a moving object among a plurality of path candidates generated based on road phase information. This step functions as a preprocessing step to reduce a set of path candidates generated within a sensor blind spot to physically feasible candidates.

[0178] Specifically, the present invention mathematically evaluates the change in direction of movement, change in velocity, and acceleration characteristics for each path candidate to eliminate candidates that violate predefined physical constraints.

[0179] First, to evaluate the continuity of the movement direction, the change in movement direction between adjacent path elements included in each path candidate Ci is calculated. For example, path candidate C i = (P1, P2, …, P m For ), the direction of movement in two consecutive segments, respectively and If defined as such, the change in direction of movement is calculated as follows.

[0180]

[0181] In the present invention, the amount of change in the direction of movement is a preset maximum allowable direction change threshold. If it exceeds

[0182]

[0183] The corresponding path candidate is identified as physically unnatural and removed. This reflects the physical characteristic that it is difficult for the moving object to perform abrupt changes in direction within a short period of time.

[0184] Next, velocity continuity-based filtering is performed. For each path candidate, the time interval of the sensor blind spot. and the total length of the path candidates Average movement speed using Calculate as follows.

[0185]

[0186] In addition, the speed of the moving object estimated in the sensor detection section before or after the shaded area or It can be defined as follows. In the present invention, the difference between the average moving speed and the adjacent section speed is a preset allowable speed change threshold. If it exceeds,

[0187]

[0188] The corresponding path candidate is deemed unrealistic and removed. This excludes path candidates based on excessively fast or slow movement.

[0189] Next, acceleration-based filtering is performed. In the present invention, for each path candidate, acceleration is used with the amount of change in movement speed and the time interval Calculate as follows.

[0190]

[0191] At this time, the absolute value of the calculated acceleration is the pre-set maximum allowable acceleration threshold If it exceeds,

[0192]

[0193] The candidate path is eliminated as it is determined to be based on movement that exceeds the physical limits of the moving object. This reflects the realistic movement characteristics that make it difficult for the moving object to repeatedly undergo rapid acceleration or deceleration.

[0194] In addition, the present invention performs filtering on temporal reachability. That is, for each route candidate, the road speed limit Minimum travel time considering Calculate the minimum travel time, and the corresponding minimum travel time is the actual time interval between sensor events. If it exceeds,

[0195]

[0196] The candidate path is determined to be temporally unreachable and removed.

[0197] After undergoing such a time-series continuity-based filtering process, the present invention forms a residual candidate set consisting only of path candidates that satisfy all conditions regarding movement direction, speed, acceleration, and time constraints. This residual candidate set is subsequently used as input for a probability score calculation step that combines sensor-based probability components, road-based probability components, and time-series-based probability components.

[0198] In this way, the time-series continuity-based candidate filtering step of the present invention mathematically models the physical movement characteristics of a moving object to remove unrealistic path candidates in advance, thereby significantly improving the accuracy and reliability of movement path estimation within sensor blind spots.

[0199] Next, a step (step S150) is performed to calculate a probability score (P-score) for each path candidate remaining after the aforementioned candidate filtering step. In this step, for each path remaining as a candidate, a probability score (P-score) is calculated to evaluate the likelihood that the path is real.

[0200] The step of calculating a probability score (P-score) for each of the above path candidates involves calculating a probability score (P-score) for each path candidate by reflecting at least two of the following elements: a sensor-based probability element (Psensor) representing the probability of occurrence of a sensor event, a road-based probability element (Proad) representing the suitability of road structure and movement constraints, and a time-series-based probability element (Pseries) representing the time-series continuity of the speed, acceleration, or direction of movement of the moving object.

[0201] In the step of calculating probability scores for each path candidate, the likelihood of each path candidate is evaluated by combining various probability factors.

[0202] The sensor-based probability factor (Psensor) is an indicator representing the probability that a candidate path actually generated a sensor event, and can be calculated by including the entropy of the WiFi AP set, the directionality of the WiFi RSSI pattern, the consistency of the distance between the BLE detection time and the candidate path, the reachability of the CCTV re-identification time, and the quality and latency error of the GPS signal.

[0203] The sensor-based probabilistic factor (Psensor) evaluates the probability that each path candidate Ci generated a sensor event, as shown in the example below.

[0204] (a) Rarity of WiFi AP combinations

[0205]

[0206] (b) CCTV Re-ID reachability

[0207]

[0208] (c) BLE proximity event

[0209]

[0210] As an example, Psensor can be configured as follows.

[0211]

[0212] Below, the rarity of WiFi AP combinations, RSSI patterns, CCTV re-identification time, or BLE detection time is explained in more detail.

[0213] - Rarity of WiFi AP combinations:

[0214] A WiFi AP is a wireless access point, and rarity indicates how uncommon a specific WiFi AP signal combination is. For example, if a WiFi AP signal is detected only in a specific area, that combination can be considered rare. If a mobile object detects this rare WiFi AP combination, it is highly likely to be located in that specific location.

[0215] - RSSI pattern:

[0216] RSSI (Received Signal Strength Indicator) refers to a signal strength indicator. RSSI patterns track changes in the strength of various WiFi signals; for example, by analyzing the pattern of how WiFi signal strength changes while walking or moving, the location of a moving object can be inferred more accurately.

[0217] - CCTV re-identification point:

[0218] CCTV re-identification refers to the point in time when a CCTV camera recognizes a specific moving object. For example, if the same CCTV recognizes a moving object multiple times, it indicates that the object has remained within the CCTV's range. Furthermore, the path of a moving object can be estimated through the consecutive identification points of different CCTVs.

[0219] - BLE detection point:

[0220] BLE (Bluetooth Low Energy) is a low-power Bluetooth technology, and the BLE detection point refers to the time when a mobile object detects a specific BLE beacon signal. For example, a movement path can be estimated using time information in which a mobile object detects signals from multiple BLE beacons.

[0221] In addition, the present invention calculates a road-based probability component (Proad) for each path candidate.

[0222] The Road-based Probabilistic Component (Proad) is an indicator representing how naturally a candidate path matches road structure and movement constraints. It is calculated by considering the fit of the time interval between the estimated travel time based on speed limits and actual sensor events, the naturalness of turn frequency and turn angle, road congestion or urban structural characteristics, and the consistency with actual observation points relative to the candidate path length.

[0223] Through this, route candidates that are unrealistic in light of the road structure receive a relatively low evaluation.

[0224] As an example, the Proad calculation is path C i It evaluates the road structure and how natural it is.

[0225]

[0226] As an example, the above road-based probability factor (Proad) can be calculated based on road network information including at least one of the following: the node and link connection relationships of the road, whether it is turnable, speed limit, number of lanes, whether it is one-way, and road congestion.

[0227] - Road Node and Link Connection Relationships:

[0228] When modeling a road network as a graph, each intersection (node) constituting the road and the roads (links) between the intersections are represented as connections. This allows for the construction of a path to move from a specific point to another.

[0229] - Rotatability:

[0230] When moving at a specific intersection, it considers whether left turns, right turns, or going straight are possible. This information indicates the direction in which each link (road section) connects and is an important factor in determining the physical path a moving object can take.

[0231] - Speed ​​limit:

[0232] Speed ​​limit information stipulated for each road is used. Since moving objects cannot travel at speeds exceeding the permitted speed when moving on the road, this becomes an important factor when calculating the travel time for each candidate route.

[0233] - Number of lanes:

[0234] Consider the number of road lanes. The number of lanes plays a crucial role in determining road capacity and traffic flow, and is used to assess how quickly and efficiently vehicles can travel along the road.

[0235] - One-way status:

[0236] Consider whether the road is one-way or two-way. One-way roads allow movement only in a specific direction, which becomes a significant constraint when selecting a route.

[0237] - Road congestion:

[0238] The congestion level of each road section is evaluated. Road congestion can vary depending on traffic volume at specific times, which directly affects the expected travel time when a vehicle selects a specific route.

[0239] In addition, the present invention calculates time-series-based probability components (Pseries) that reflect the time-series movement characteristics of a moving body.

[0240] Time-series-based probabilistic factors (Pseries) quantitatively evaluate how consistent each path candidate is with the actual time-series pattern of the moving object by comprehensively considering the velocity and acceleration patterns in the preceding and subsequent segments of the moving object, the amount of change in the direction of movement vector, the naturalness of position changes over time, and the correlation with surrounding sensor event occurrence patterns.

[0241] As an example, the Pseries calculation is path C i Evaluates how closely it matches the time-series movement of the moving object.

[0242]

[0243] For example, assuming that it is more natural for a moving body to maintain a constant pattern without suddenly changing direction or speed, the validity of the path can be evaluated by considering this continuity.

[0244] The above time series-based probability elements (Pseries) can be calculated based on at least one of the following changes in direction of movement, continuity of the velocity profile, or changes in acceleration.

[0245] - Change in direction of movement:

[0246] This analyzes the changes that occur when a moving object changes direction while in motion. The magnitude of the change in direction can serve as an important clue for identifying the object's movement patterns. For example, a sudden change in direction can be interpreted as the object deviating from the usual road or engaging in unexpected behavior.

[0247] - Continuity of velocity profile:

[0248] By continuously analyzing the speed profile of a moving object—that is, how its speed changes over time—it is possible to distinguish between cases where the object moves at a constant speed and cases where it accelerates or decelerates. This information is highly useful for predicting movement paths. For example, if a moving object suddenly increases or decreases its speed, it may indicate specific conditions along that path.

[0249] - Change in acceleration:

[0250] Not only changes in velocity but also acceleration—that is, the rate of change of velocity—is considered an important factor. By analyzing the change in acceleration, it is possible to determine how rapidly a moving object accelerates or decelerates. This analysis helps to understand the object's movement patterns more precisely.

[0251] By comprehensively analyzing these factors to calculate time-series-based probability elements (Pseries), it is possible to determine the probability score of the path among the candidate paths that best reflects the physical movement and temporal continuity of the moving object. This calculation process plays a crucial role in selecting the path that is most natural—that is, the path most likely to occur in reality.

[0252] The present invention calculates a final probability score (P-score) for each path candidate by weightedly combining the Psensor, Proad, and Pseries calculated in this way.

[0253] In this case, weight values ​​are set to reflect the relative importance of the sensor environment, road structure, and time-series information, thereby transforming the path selection problem into a comprehensive probabilistic model that includes sensor information, road structure, and temporal characteristics.

[0254] The present invention can calculate a probability score for each path candidate by reflecting at least two probability elements among a sensor-based probability element (Psensor) indicating the probability of occurrence of a sensor event, a road-based probability element (Proad) indicating the suitability of road structure and movement constraints, and a time-series-based probability element (Pseries) indicating the time-series continuity of the speed, acceleration, or movement direction of a moving object.

[0255] This enables more reliable path prediction by comprehensively evaluating various factors rather than considering only a single factor.

[0256] Next, whenever a new sensor event is received, a step (step S160) is performed to update the optimal path by recalculating the probability score (P-score) of the path candidates.

[0257] Finally, the present invention determines the path candidate with the highest probability score (P-score) as the final path most likely taken by the moving object within the sensor blind spot. Through this process, highly accurate estimation of the moving object's path becomes possible by comprehensively considering incomplete and discontinuous observation information occurring in a heterogeneous sensor environment.

[0258] Specifically, the present invention compares a probability score (P-score) calculated by combining a sensor-based probability component (Psensor), a road-based probability component (Proad), and a time-series-based probability component (Pseries) for each path candidate, with respect to a set of remaining path candidates that has undergone a time-series continuity-based candidate filtering step. The probability score for [it] quantitatively indicates the likelihood that the candidate can explain the actual movement path of the moving object within the sensor blind spot.

[0259] The present invention selects the path candidate having the maximum value among the probability scores as the final path. That is, the final selected path It can be determined according to the following relationship.

[0260]

[0261] Here, is a path candidate It refers to the probability score calculated for. For example, if a specific candidate among multiple path candidates has a relatively higher probability score compared to other candidates, that candidate is determined to be the path most likely to have been traveled by the moving body.

[0262] The final path selected in this way is the path most likely to have actually moved within the sensor blind spot, and is subsequently used to construct the entire time-series movement trajectory of the moving body.

[0263] In addition, in the present invention, when a new sensor event is additionally received, the probability scores are recalculated, and the final selected path can be updated as needed.

[0264] Figure 6 shows a table comparing probability scores (P-scores).

[0265] The table shown in Fig. 6 illustrates an example of comparing probability scores (P-scores) indicating the likelihood that each path candidate is the actual path of a moving object for a plurality of path candidates generated within the sensor shaded area.

[0266] In Figure 6, path candidates C1, C2, and C3 represent different movement path candidates connecting the start and end points of the same shaded section, and the P-score assigned to each path candidate is a value calculated by comprehensively reflecting sensor event suitability, road structure suitability, and time series continuity information.

[0267] As shown in Figure 6, path candidate C3 has a relatively high probability score compared to other candidates, so it is determined to be the path most likely to have been selected when the moving body actually passes through the shaded section.

[0268] Accordingly, the present invention compares the probability scores of each path candidate, selects the path candidate with the maximum value as the final path, and outputs it as the actual movement path of the moving object.

[0269] Accordingly, the final path determination step of the present invention performs the function of objectively and reproducibly deriving the actual movement path of a moving object within a sensor blind spot by comprehensively reflecting discontinuous observation information collected from heterogeneous sensors, road phase information, and the time-series movement characteristics of the moving object.

[0270] FIG. 7 is an example diagram showing the section directly observed by the sensor, the sensor shaded section that is not observed, and the possible movement path candidates within the shaded section during the process of a moving object moving in a heterogeneous sensor environment.

[0271] Figure 7 visually illustrates the observed trajectory and path estimation results of a moving object against the backdrop of an actual road network in an urban area.

[0272] The path indicated by the light blue dashed line in Fig. 7 represents the actual movement trajectory of the moving body in the section directly observed by the sensor.

[0273] In this section, the position of a moving object is continuously detected within the detection range of sensors such as CCTVs, thereby securing a relatively accurate time-series trajectory. This measured trajectory is generated based on the time and location information of sensor events.

[0274] The area marked with a light green square box in Fig. 7 represents a sensor shaded area where the sensor's detection range does not reach. This shaded area is formed between adjacent sensor detection zones, and while a moving object passes through this area, direct position observation is not made from any sensor.

[0275] Therefore, it is not possible to determine which road the moving object actually traveled along within the shaded area based solely on observation data.

[0276] The path indicated by the red solid line in Fig. 7 represents an example of the path the moving body actually traveled within the shaded area. This path was verified retrospectively or presented as a reference path and serves as a comparison point to explain the actual movement path within the sensor shaded area.

[0277] Meanwhile, the multiple paths indicated by the yellow dotted lines in Fig. 7 represent the top candidates among the multiple path candidates that a moving object can select within the sensor shaded area. These path candidates are structurally generated using the topological information of the urban road network based on the start and end points of the shaded area.

[0278] Each candidate path reflects road structures such as branches, mergers, and intersections, and consists of paths through which a moving object can physically move. In Figure 7, the top candidates with high probability scores among these candidate paths are visually depicted.

[0279] As such, FIG. 7 illustrates the measured trajectory obtained in the sensor detection section, the spatial range of the sensor shaded section, an example of an actual movement path within the shaded section, and a plurality of path candidates generated and evaluated by the present invention, thereby intuitively explaining the path ambiguity problem occurring in the sensor shaded section and the path candidate generation and selection process of the present invention to solve it.

[0280] In particular, the present invention confirms that the movement path within the shaded area can be estimated more accurately by not assuming a single path within the sensor shaded area, but by generating multiple path candidates based on road phase information and then selecting the path that best matches the actual movement path through a probabilistic model.

[0281] Meanwhile, the step of updating the optimal path may include an online update mechanism that recalculates probability scores (P-scores) for the top K path candidates and selects the optimal path when a new sensor event occurs.

[0282] New sensor data (events) is received when the moving object exits the shaded area or within it. When new sensor data arrives during this process, the existing set of path candidates is updated, and this update mechanism is performed in real time.

[0283] Meanwhile, the present invention provides an online update structure that maintains the top K path candidates with high probability scores and recalculates the probability score of each path candidate whenever a new sensor event is received.

[0284] For example, when a WiFi detection event, CCTV re-identification event, BLE event, or GPS reception occurs, the event is immediately reflected in the Psensor, Proad, and Pseries calculation process, and the P-score of each candidate is updated.

[0285] Accordingly, the priority among path candidates is dynamically re-evaluated, and the optimal path is immediately replaced if necessary.

[0286] Thus, the present invention selects the most likely path candidate as the optimal path based on the recalculated probability score. The top K candidates are those selected as the top K paths based on the previously calculated probability score.

[0287] The above new sensor event may include at least one of WiFi detection, CCTV re-identification, BLE detection, and GPS signal input. Whenever a mobile object detects a WiFi signal, is captured by a CCTV camera, detects a BLE beacon signal, or receives a GPS signal, the corresponding event occurs, causing the system to update the path of the mobile object, and in this way, the system can estimate and update the most probable path of the mobile object.

[0288] A new sensor event refers to a specific sensor-based event that occurs when a moving object passes a specific location, and the specific types of sensor events are as follows:

[0289] - WiFi Detection: This is an event that occurs when a mobile object enters the WiFi signal range. For example, this event occurs when a mobile object enters the signal range of a specific WiFi access point (AP).

[0290] - CCTV Re-identification: This is an event that occurs when a moving object is identified again within the CCTV camera's field of view. For example, this occurs when the CCTV re-identifies a person upon entering a specific range.

[0291] - BLE Detection: This is an event that occurs when a BLE (Low-Power Bluetooth) beacon signal is detected. It is a method of detecting a signal when a mobile object enters the range of a specific BLE beacon.

[0292] - GPS Signal Input: This is an event that occurs when the location of a moving object is determined using GPS. This event occurs when the moving object receives a GPS signal and updates its location.

[0293] Figure 8 shows a system for estimating the movement path of a moving object within a sensor blind spot in a heterogeneous sensor environment.

[0294] Hereinafter, with reference to FIG. 8, a system (1000) for estimating the movement path of a moving body within a sensor shaded area in a heterogeneous sensor environment according to another embodiment of the present invention will be described.

[0295] The system (1000) for estimating the movement path of a moving object within a sensor blind spot in a heterogeneous sensor environment according to the present invention utilizes data obtained from various types of sensors to estimate the movement path of the moving object within a section (sensor blind spot) where the sensors cannot provide direct data, thereby more accurately determining the position or movement of the moving object. It comprises a sensor data collection unit (100), a blind spot detection unit (200), a road phase information storage unit (300), a path candidate generation unit (400), a time series continuity-based candidate filtering unit (500), a probability score calculation unit (600), and an online update unit (700).

[0296] The sensor data collection unit (100) collects sensor events regarding a moving object from a plurality of heterogeneous sensors that generate information regarding at least one of the location, existence, identification, or movement state of the moving object. The heterogeneous sensors include sensors such as CCTV, WiFi AP, BLE beacon, GPS, and drone.

[0297] The shadow zone detection unit (200) defines the unobserved section between two different sensor events as a sensor shadow zone, and calculates the entry point and exit point of the sensor shadow zone using the time and location information of the sensor events. The shadow zone detection unit plays the role of setting the boundary of the section where a moving body is not observed using the time and location information of the sensor events.

[0298] The road topology information storage unit (300) stores a road graph containing topology information of the city road network. The road topology information storage unit plays the role of storing and managing a road graph containing the structure, connection relationships, constraints, etc. of the road that a moving object can move on.

[0299] The path candidate generation unit (400) generates a set of multiple path candidates that a moving body can select using the road graph between the starting point (Entry) and the ending point (Exit). The path candidate generation unit generates multiple path candidates that a moving body can actually select between the starting point and the ending point of the shaded section.

[0300] The above path candidate generation unit (400) can reflect road phase constraints including at least one of whether it is rotatable, speed limit, and whether it is one-way.

[0301] The path candidate generation unit (400) uses the given road network information (topology) to generate a path.

[0302] Road network information is generated by including all constraints, such as how roads are connected, whether turns are possible, whether there are speed limits, and whether movement is restricted to a specific direction (one-way roads, etc.).

[0303] That is, the path candidate generation unit (400) can generate various paths through which the moving body can move, evaluate the feasibility of each path, and more accurately estimate the current position of the moving body based on the results.

[0304] For example, while there are multiple paths for a moving object to travel from a specific point to the next, paths where rotation is impossible or where the speed is set too low can be filtered out and excluded. As a result, the generated path candidates consist of the most realistic and feasible routes for the actual movement of the object.

[0305] The time series continuity-based candidate filtering unit (500) removes physically impossible or unnatural path candidates from the plurality of path candidates by utilizing the time series continuity of the speed, acceleration, or direction of movement of the moving body.

[0306] Through the time series continuity-based candidate filtering unit (500), only physically feasible paths can be kept and unrealistic paths can be filtered out.

[0307] The probability score calculation unit (600) calculates a probability score (P-score) for each path candidate that was not removed by the candidate filtering unit (500).

[0308] The probability score calculation unit (600) calculates a probability score for each path by reflecting the probability of sensor events occurring, the suitability of road structures and movement constraints, the speed of the moving body, and the continuity of the movement direction.

[0309] The probability score calculation unit (600) calculates a sensor-based probability element (Psensor) indicating the probability of a sensor event occurring, a road-based probability element (Proad) indicating the suitability of the road structure and movement constraints, and a time-series-based probability element (Pseries) indicating the time-series continuity of the speed, acceleration, or movement direction of the moving body, respectively, and calculates a probability score (P-score) for each path candidate by reflecting at least two of the calculated probability elements. The probability score (P-score) is a value indicating the probability that each path candidate is the actual path of the moving body.

[0310] A sensor-based probability factor (Psensor) is a probability factor that represents the probability of various sensor events occurring.

[0311] For example, a sensor-based probabilistic factor (Psensor) analyzes the frequency or patterns of sensor events, such as WiFi AP (Access Point) detection, CCTV re-identification, and BLE (Bluetooth Low Energy) detection, to calculate the probability that these events will occur in a specific path.

[0312] The Road-based Probabilistic Factor (Proad) evaluates whether each path candidate actually exists and is traversable based on physical information of the road network, such as the connectivity between nodes (intersections) and links (road sections), turnability, speed limits, number of lanes, one-way traffic status, and road congestion.

[0313] Time-series-based probabilistic components (Pseries) reflect the time-series continuity of a moving object's direction of movement, velocity, acceleration, etc.

[0314] For example, time-series-based probabilistic factors (Pseries) evaluate the probability that a moving object will follow a specific path by considering factors such as changes in direction, the continuity of the velocity profile, and changes in acceleration when moving along that path. This is an important factor in increasing the probability that the object's movement is natural and physically possible.

[0315] The online update unit (700) recalculates the probability score (P-score) of the path candidates whenever a new sensor event is received and updates the optimal path.

[0316] The online update unit (700) can continuously update the path estimation results by reflecting new sensor data in real time and select the optimal path. This update aims to continuously improve the accuracy of the path estimation by reflecting additional information provided by new sensor events.

[0317] The above online update unit (700) stores the probability score of an existing path candidate in memory and updates the stored probability score when a new sensor event is received.

[0318] Although the present invention has been described above with reference to the embodiments illustrated in the drawings, this is merely illustrative, and those skilled in the art will understand that various modifications and equivalent alternative embodiments are possible therefrom.

[0319] In addition, the present invention may include various modified embodiments. For example, LiDAR or radar sensors may be added to the sensor-based probabilistic component, or Pseries may be enhanced using a deep learning-based time series model, and it may be implemented as a real-time stream processing structure linked with a smart city control server or as an onboard system of an autonomous vehicle. The components of the present invention can be sufficiently implemented in a general computing environment by combining a graph search algorithm, a probability calculation module, and a sensor data processing module.

[0320] The technical configuration of the present invention can be applied identically to similar systems and methods without significant modification, provided that it does not deviate from the technical concept. Therefore, the true technical scope of protection of the present invention should be determined solely by the appended claims. Explanation of the symbols

[0321] 1000: System for estimating the movement path of a moving object within sensor blind spots in a heterogeneous sensor environment 100: Sensor data collection unit 200: Shadow section detection unit 300: Road topology information storage unit 400: Path candidate generation unit 500: Time series continuity-based candidate filtering unit 600: Probability score calculation unit 700 : Online Update Department

Claims

Claim 1 A step of collecting sensor events regarding a moving object from a plurality of heterogeneous sensors that generate information regarding at least one of the location, existence, identification, or movement state of the moving object; a step of defining an unobserved interval between two different sensor events as a sensor shadow zone, and calculating an entry point and an exit point of the shadow zone using the time and location information of the sensor events; a step of generating a set of multiple path candidates that the moving object can select between the entry point and the exit point; a time-series continuity-based candidate filtering step for the plurality of path candidates, which removes physically impossible or unnatural path candidates using the time-series continuity of the moving object's speed, acceleration, or direction of movement; and a step of calculating a probability score (P-score) for each path candidate remaining after the candidate filtering step. A method for estimating a movement path of a moving object within a sensor blind spot in a heterogeneous sensor environment, comprising the step of recalculating the probability scores of the path candidates and updating the optimal path whenever a new sensor event is received, wherein the step of generating the set of the plurality of path candidates is characterized by generating the plurality of path candidates through which a moving object can move between the starting point and the ending point based on the topology of the urban road network, and generating the path candidates by reflecting the link and node structure, rotatable status, and movement constraints of the road network, wherein the sensor event includes an event that occurs when the presence or location of the moving object is detected by a WiFi access point (AP), BLE beacon, CCTV, or drone, and the step of calculating the probability score (P-score) is characterized by probabilistically evaluating the probability of each path candidate by reflecting the suitability of the sensor event, the suitability of the road structure, and the time-series continuity of the moving object. Claim 2 A method for estimating the movement path of a moving object within a sensor blind spot in a heterogeneous sensor environment, characterized in that, in claim 1, the heterogeneous sensor is one of CCTV, WiFi AP, BLE beacon, GPS, drone, WiFi scanner, and Lidar. Claim 3 delete Claim 4 A method for estimating a movement path within a sensor shaded area of ​​a moving object in a heterogeneous sensor environment, wherein, in claim 1, the step of generating a plurality of path candidate sets is performed by reflecting road phase constraints including at least one of whether rotation is possible, whether the speed limit is high, and whether the road is one-way. Claim 5 A method for estimating a movement path of a moving object within a sensor blind spot in a heterogeneous sensor environment, wherein, in claim 1, the step of calculating a probability score (P-score) for each path candidate is characterized by reflecting at least two of the following elements: a sensor-based probability element (Psensor) indicating the probability of occurrence of a sensor event, a road-based probability element (Proad) indicating the suitability of road structure and movement constraints, and a time-series-based probability element (Pseries) indicating the time-series continuity of the speed, acceleration, or direction of movement of the moving object. Claim 6 A method for estimating the movement path of a moving object within a sensor blind spot in a heterogeneous sensor environment, wherein, in claim 5, the Psensor includes at least one of the rarity of a WiFi AP combination, an RSSI pattern, a CCTV re-identification time, or a BLE detection time. Claim 7 A method for estimating the movement path of a moving object within a sensor blind spot in a heterogeneous sensor environment, wherein, in claim 5, the road-based probability element (Proad) is based on road network information including at least one of the road's node and link connection relationships, whether it is turnable, speed limit, number of lanes, whether it is one-way, and road congestion. Claim 8 A method for estimating the movement path of a moving object within a sensor shaded area in a heterogeneous sensor environment, characterized in that, in claim 5, the time-series-based probability element (Pseries) is calculated based on at least one of the amount of change in movement direction, the continuity of the velocity profile, or the amount of change in acceleration. Claim 9 A method for estimating the movement path of a moving object within a sensor blind spot in a heterogeneous sensor environment, wherein, in claim 1, the step of updating the optimal path includes an online update mechanism that recalculates a probability score (P-score) for the top K path candidates and selects the optimal path when a new sensor event occurs. Claim 10 A method for estimating the movement path of a moving object within a sensor blind spot in a heterogeneous sensor environment, characterized in that, in claim 1, the new sensor event includes at least one event among WiFi detection, CCTV re-identification, BLE detection, and GPS signal input. Claim 11 A sensor data collection unit that collects sensor events regarding a moving object from a plurality of heterogeneous sensors that generate information regarding at least one of the location, existence, identification, or movement state of the moving object; a shadow zone detection unit that defines an unobserved interval between two different sensor events as a sensor shadow zone and calculates an entry point and an exit point of the sensor shadow zone using the time and location information of the occurrence of the sensor events; a road topology information storage unit that stores a road graph including topology information of an urban road network; a path candidate generation unit that generates a set of multiple path candidates selectable by the moving object using the road graph between the entry point and the exit point; a time-series continuity-based candidate filtering unit that removes physically impossible or unnatural path candidates from the plurality of path candidates using the time-series continuity of the moving object's speed, acceleration, or direction of movement; and a probability score calculation unit that calculates a probability score (P-score) for each path candidate that is not removed by the candidate filtering unit. and an online update unit that recalculates the probability scores (P-scores) of the path candidates and updates the optimal path whenever a new sensor event is received;A system for estimating the movement path of a mobile body within a sensor blind spot in a heterogeneous sensor environment, comprising: a plurality of path candidate sets, wherein the plurality of path candidate sets include a plurality of path candidates in which a mobile body can move between a starting point and an ending point based on the topology of an urban road network, and reflect the link and node structure, rotatable status, and movement constraints of the road network; wherein the sensor events include events that occur when the presence or location of a mobile body is detected by a WiFi access point (AP), BLE beacon, CCTV, or drone; and wherein the probability score (P-score) probabilistically evaluates the probability of each path candidate by reflecting the suitability of the sensor event, the suitability of the road structure, and the time-series continuity of the mobile body. Claim 12 A system for estimating the movement path of a moving object within a sensor blind spot in a heterogeneous sensor environment, wherein, in claim 11, the path candidate generation unit reflects road phase constraints including at least one of whether rotation is possible, speed limit, and whether it is one-way. Claim 13 A system for estimating the movement path of a moving body within a sensor blind spot in a heterogeneous sensor environment, wherein, in claim 11, the probability score calculation unit calculates a sensor-based probability element (Psensor) indicating the probability of occurrence of a sensor event, a road-based probability element (Proad) indicating the suitability of road structure and movement constraints, and a time-series-based probability element (Pseries) indicating the time-series continuity of the speed, acceleration, or movement direction of the moving body, respectively, and calculates a probability score (P-score) for each path candidate by reflecting at least two of the calculated probability elements. Claim 14 A system for estimating the movement path of a moving object within a sensor blind spot in a heterogeneous sensor environment, wherein, in claim 11, the online update unit stores the probability score of an existing path candidate in memory and updates the stored probability score upon receiving a new sensor event.

Images