System and Method for Generating Real-world Situation Vector Model Based on Natural Language Synthesis of Heterogeneous Data

Abstract

The present invention relates to a system and method for generating a real-world situation vector model based on natural language synthesis of multiple offline data, comprising: a multi-data collection and refinement unit for collecting heterogeneous offline data occurring in a specific spatiotemporal region; a natural language synthesis and tag extraction unit for fusing the collected heterogeneous offline data and converting it into an intermediate representation in the form of natural language that describes the situation in the spatiotemporal region; a situation vector embedding unit for inputting the converted intermediate representation in the form of natural language into a text embedding model to generate a continuous situation vector encoded with the real-world situation context of the spatiotemporal region; and a vector database and maintenance unit for storing and maintaining the generated situation vector in a database.

Description

Technology Field

[0001] The present invention relates to the field of artificial intelligence-based data fusion and vector embedding technology. More specifically, it relates to a system and method for collecting heterogeneous data generated in a specific spatiotemporal region of the offline real world, synthesizing it into an intermediate representation in the form of natural language, and then converting it into a continuous situation vector encoded with real-world situational context through a text embedding model for storage and utilization. More specifically, the invention relates to a technology for natural language conversion of image data using a vision language model, heterogeneous data fusion applying confidence-based weights, dynamic vector management through a time decay function, and providing real-time situational context to external autonomous systems such as autonomous vehicles, industrial robots (patrol / safety monitoring), disaster response systems (smart cities), and robots. Background Technology

[0002] In the modern urban environment, real-time situational awareness and prediction technologies are essential in various fields, including smart city development, autonomous driving system operation, disaster management, and traffic optimization. In particular, accurately identifying real-world offline phenomena—such as changes in population flow, traffic patterns, safety accidents, and large-scale events occurring in physical spaces—and representing them in a digital form is a key element for effective urban management and the operation of autonomous systems.

[0004] Traditionally, various heterogeneous data sources, such as CCTV footage, sensor data, mobile carrier population mobility information, and social media posts, have been utilized to understand these real-world situations. However, because these data differ in format, structure, and reliability levels, it is difficult to integrate them to generate a consistent representation of the situation.

[0006] Conventional data integration technologies have primarily focused on processing data in the online digital domain. For instance, technologies have been developed to embed and vectorize text-based digital data, such as user behavior logs, network traffic data, and academic literature; however, these technologies have limitations in reflecting the dynamic characteristics of offline physical spaces. In particular, there is a problem in that text-based embedding models find it difficult to directly process unstructured visual data, such as CCTV footage.

[0008] Furthermore, conventional technology suffers from inefficiency as data collection is performed at static intervals or continuously, consuming unnecessary computing resources even when no meaningful events occur. Since the real world presents stark differences between normal conditions and the occurrence of events, static collection methods that fail to reflect these dynamic characteristics have limitations in terms of real-time performance and efficiency.

[0010] Conventional multimodal embedding technology had limitations in that structured indexing was impossible, either by directly mapping each modality data into a shared vector space or by utilizing natural language only as the final output.

[0012] Furthermore, methods that simply combine heterogeneous data or integrate them with equal weights risk generating inaccurate representations of the situation because they fail to account for differences in the reliability of each data source. For example, if official traffic information from public institutions and speculative posts from personal blogs are treated equally, there is a problem where the accuracy of the final result is reduced.

[0014] Offline real-world phenomena are inherently transient or possess characteristics where their relevance changes over time. For example, the relevance of a concert event in a specific region to the current situation decreases rapidly as time elapses after it has concluded. However, conventional technologies have limitations in accurately reflecting this temporal dynamism because they store generated vectors statically or apply only simple time filtering.

[0016] Furthermore, autonomous systems, such as self-driving vehicles and robots, must accurately perceive the real-time conditions of their surrounding environment to perform safe and efficient path determination and action decisions. However, conventional technologies face difficulties in integrating with the autonomous system ecosystem because they utilize data only within specific application systems or do not provide standardized interfaces with external systems.

[0018] Due to these limitations, a new technical approach is required to effectively integrate diverse heterogeneous data from the offline real world, convert multimodal information including unstructured image data into consistent vector representations, respond adaptively to the occurrence of real-world events, generate accurate situational representations by reflecting the reliability of each data source, perform vector management considering temporal dynamism, and provide real-time situational context to external autonomous systems.

[0020] Korean Patent Publication No. 10-2021-0058421 (May 6, 2021)

[0021] Korean Patent Publication No. 10-2025-0089313 (July 3, 2025)

[0023] The present invention aims to solve the technical problem of effectively integrating and modeling various heterogeneous data occurring in the offline real-world spatiotemporal domain.

[0025] Conventional data processing systems have primarily focused on structured data generated in online digital environments, such as user behavior logs, network traffic, and text documents, and have had limitations in integrally processing heterogeneous offline data occurring in physical spaces, such as pedestrian traffic, CCTV footage, sensor measurements, and social media mentions. In particular, there was a lack of a systematic methodology for converting unstructured multimodal data, such as video data, into a format that text-based embedding models could process, and there was a problem where the accuracy of the final analysis results was degraded due to a simple combination method that did not consider the differences in reliability of each data source.

[0027] Furthermore, conventional technologies collected data based on static periods, consuming unnecessary computing resources even when no meaningful events occurred. They also had limitations in that they failed to reflect the ephemeral nature of offline real-world phenomena, treating outdated information equally to reduce the accuracy of real-time situational awareness. Moreover, the lack of standardized interfaces that allow external autonomous systems, such as self-driving vehicles or robots, to utilize generated situational information in real time has made it difficult to implement autonomous systems that adaptively respond to changes in the real-world environment. Prior art literature

[0028] Korean Patent Application No. 10-2022-0000505 Korean Patent Application No. 10-2025-0057027 Korean Registered Patent 10-2911684 The problem to be solved

[0029] Accordingly, the present invention aims to provide an integrated system and method that enables accurate digital representation of real-world situations and expansion into various application fields by effectively integrating heterogeneous data in the offline real-world spatiotemporal domain, constructing a two-stage multimodal processing pipeline that converts image data into natural language using a vision language model and then vectorizes it into a text embedding model, improving accuracy through natural language synthesis with reliability-based weights applied per data source, efficiently utilizing computing resources with a dynamic event triggering mechanism, reflecting the transience of offline phenomena by applying a time decay function, and implementing an API interface that provides real-time context to an autonomous system. means of solving the problem

[0030] A system for generating a real-world situation vector model based on natural language synthesis of multiple offline data according to one aspect of the present invention for solving the above problem may include: a multi-data collection and refinement unit that collects heterogeneous offline data occurring in a specific spatiotemporal region; a natural language synthesis and tag extraction unit that fuses the collected heterogeneous offline data and converts it into an intermediate expression in the form of natural language that describes the situation in the spatiotemporal region; a situation vector embedding unit that inputs the converted intermediate expression in the form of natural language into a text embedding model to generate a continuous situation vector encoded with the real-world situation context of the spatiotemporal region; and a vector database and maintenance unit that stores and maintains the generated situation vector in a database.

[0032] The above-mentioned multi-data collection and refinement unit may include a dynamic event triggering module that detects when the amount of change in the flow of people in the spatiotemporal area or the amount of mention of a specific text exceeds a preset threshold and triggers data collection and subsequent layer operations, and a vision language model-based image parsing module that analyzes image data collected from the outside using a vision language model and converts it into scene description data in text form for collection.

[0034] The above natural language synthesis and tag extraction unit synthesizes numeric, ratio, and coordinate data included in the collected heterogeneous offline data into a single complete natural language descriptive text using a language model or a rule-based template, and synthesizes by applying different weights based on the confidence level pre-assigned to each data source, and when synthesizing intermediate expressions in the form of natural language, it can extract structured metadata tags including the category, scale, and spatial type of the spatiotemporal region in parallel.

[0036] The above vector database and maintenance unit may include a time decay application unit that applies a time decay function to decrease the relevance weight of the said situation vector in proportion to the time elapsed since the occurrence of the stored situation vector in order to reflect the dynamic characteristics of offline real-world situations, and a dynamic association graph generation unit that calculates the similarity between accumulated situation vectors and periodically generates and updates an event association graph.

[0038] The above system may further include a vector-based sub-application unit that calls the generated and maintained situation vector to perform at least one application process among situation search, automatic classification, time series forecasting, anomaly detection, and providing context to an external autonomous system.

[0040] The above vector-based sub-application unit can perform at least one of the following functions: automatically classifying categories by comparing the situation vector with a predefined anchor vector space; predicting future situations by tracking the time-series change patterns of situation vectors generated in the same spatial unit; identifying an abnormal situation when the situation vector deviates abruptly from a previously accumulated vector pattern; and providing real-time situation vectors of a specific spatiotemporal region to an external autonomous vehicle or robot for use in path determination.

[0042] A method for generating a real-world situation vector model based on natural language synthesis of multiple offline data according to another aspect of the present invention for solving the above problem may include: a step of collecting heterogeneous offline data occurring in a specific spatiotemporal region; a step of fusing the collected heterogeneous offline data to convert it into an intermediate expression in the form of natural language that describes the situation in the spatiotemporal region; a step of inputting the converted intermediate expression in the form of natural language into a pre-trained text embedding model to generate a continuous situation vector encoded with the real-world situation context of the spatiotemporal region; and a step of storing and maintaining the generated situation vector in a database.

[0044] The step of converting into the intermediate expression in the form of natural language involves synthesizing the intermediate expression in the form of natural language by assigning weights based on the reliability pre-assigned to each source of the heterogeneous offline data, and during the synthesis process, structured tag data regarding the real-world situation of the corresponding spatiotemporal domain can be simultaneously extracted and structured.

[0046] The above-mentioned collecting step collects scene description data in the form of natural language text obtained by analyzing external image data with a vision language model and includes it in the above-mentioned heterogeneous offline data, and the above-mentioned storing and maintaining step can update real-time relevance by periodically applying a time decay algorithm to the above-mentioned situation vector to reflect the transient characteristics of offline phenomena.

[0048] The above method may further include a step of performing at least one of the following: performing a hybrid situation search by combining a similarity operation between a user's query vector and the situation vector and spatial filtering based on the generated and maintained real-world situation vector; and providing an inference result based on the real-world situation vector to an external autonomous system as a real-time context information API. Effects of the invention

[0049] A method and system for generating a real-world situation vector model based on natural language synthesis of multiple offline data according to an embodiment of the present invention provides the following effects.

[0050] First, perfect semantic integration of heterogeneous offline data is possible. Conventionally, it was impossible to directly combine data of completely different forms, such as population flow figures (e.g., 12,000 people), sales growth rates (e.g., +250%), and geographical coordinates, to compare their meanings. By converting and synthesizing these numerical and sensor data into a common medium called 'Natural Language,' the present invention perfectly integrates complex situations of the offline real world, which could not be processed by existing text embedding technology, into a single high-dimensional vector that a computer can compute and understand.

[0051] Second, it provides a universal 'Real-World Foundation Model' that is not dependent on specific applications.Unlike conventional technologies that were limited to simple event 'detection' or specific 'search' services, this invention constructs a model that represents the offline world situation itself as a continuous vector. Through this, once a situation vector is generated, it can be expanded into an infinite number of downstream applications, such as spatial / semantic-based hybrid search, automatic category classification, time-series-based future demand forecasting, and accident / disaster anomaly detection through vector pattern deviation detection.

[0052] Third, it has infinite sensor capacity and system scalability. It is possible to integrate not only existing GPS or text data but also 'Scene Description' text derived by analyzing CCTV or drone footage using a Vision Language Model (VLM) within the same pipeline. Since all sensors are ultimately converted into natural language and encoded into the same context vector regardless of their form, the system can flexibly accommodate any new external data (such as noise sensors or carrier data) added in future smart city and IoT environments without structural changes.

[0053] Fourth, it can be utilized as a core situational awareness infrastructure for autonomous systems (Physical AI). The generated real-time real-world situation vectors can be provided in real-time via API to external systems operating in the physical world, such as autonomous vehicles, delivery robots, and drones. Through this vector data, autonomous systems can immediately recognize the dynamic context of the surrounding environment—e.g., "the area is currently densely populated due to a large festival"—and perform advanced responses, such as determining alternative routes.

[0054] Fifth, it reflects dynamic changes in the physical world in real time.Unlike existing search technologies that target static online web documents with infrequent changes, this invention accurately reflects the characteristics of offline events that are constantly generated and disappearing. By applying a time decay algorithm to the context vector, the weight of older events is gradually reduced, thereby maintaining real-time synchronization and relevance between the current state of the real world and the vector model with high accuracy.

[0055] According to the present invention, through a two-stage multimodal processing pipeline that utilizes a vision language model to convert image data into natural language and then vectorizes it into a text embedding model, significant technical effects can be provided, such as effectively structuring unstructured image data and standardizing it into a form that can be processed by a text-based embedding model, compared to conventional single-modal or simple combination methods. In particular, by converting various visual data, such as CCTV, drone, and satellite images, into natural language scene descriptions, complex visual information such as objects, actions, and scene composition within the image can be accurately encoded into linguistic expressions, and a single natural language intermediate expression can be generated by seamlessly integrating this with other heterogeneous data, such as pedestrian traffic, sensor data, and social media text.

[0057] In addition, the natural language synthesis technique of the present invention, which applies reliability-based weights for each data source, controls the quality of the final situation vector by differentially reflecting the accuracy and reliability of each data source. This allows for effective adjustment of conflicting information compared to the conventional method of applying equal weights and prioritizes the reflection of highly reliable data. This has the advantageous effect of significantly improving the accuracy of the final situation vector by more strongly reflecting information from highly reliable sources, such as traffic information from public institutions and mobile carrier population data, and appropriately limiting the influence of sources with relatively low reliability, such as social media posts.

[0059] The dynamic event triggering mechanism of the present invention enables the intensive allocation of computing resources at the time of meaningful event occurrence compared to conventional static period-based collection methods by activating the data collection and processing pipeline only when the amount of change in pedestrian traffic or text mentions exceeds a threshold. This allows for the avoidance of data processing at unnecessary times, thereby significantly improving the computing resource efficiency of the entire system, while also achieving an unexpected synergistic effect of ensuring real-time capability by reacting immediately to important real-world events.

[0061] The dynamic vector management technique applying a time decay function of the present invention can mathematically and accurately model the transient characteristics of offline real-world phenomena compared to conventional static vector storage methods by decreasing relevance weights in proportion to the time elapsed since the vector was generated. This allows the influence of transient events such as concerts, protests, and accidents to naturally diminish over time, significantly improves the accuracy of real-time situational awareness by giving higher priority to the latest situational information, and solves technical challenges by effectively eliminating the negative impact of outdated information on current decision-making.

[0063] The API interface for providing autonomous system context of the present invention provides real-time situation vectors to external autonomous systems, such as autonomous vehicles, robots, and drones, via a standardized API. This enables various autonomous systems to perceive changes in the real-world environment in real time and utilize them for path determination and action decisions, compared to conventional methods of utilizing them within a closed system. Through this, autonomous vehicles can identify traffic congestion, accidents, and weather changes along the destination route in advance to select an optimal path, robots can adaptively respond to dynamic environmental changes, and drones can avoid risk factors along their flight paths, thereby providing significant industrial benefits that simultaneously enhance the safety and efficiency of autonomous systems.

[0065] The structured metadata tag parallel extraction technique of the present invention enables hybrid search compared to conventional methods that perform only one of natural language generation or metadata extraction, by extracting metadata such as category, scale, and spatial type simultaneously with the synthesis of natural language intermediate expressions. This has the advantageous effect of simultaneously improving search accuracy and efficiency by combining vector similarity-based semantic search with metadata filtering, and significantly enhancing the performance of various application processes, such as automatic classification through anchor vector spatial comparison and anomaly detection based on vector pattern deviation.

[0067] The dynamic association graph generation technique of the present invention can infer causal relationships between spatially and temporally adjacent events and predict chain events by periodically generating and updating an association graph between events through the calculation of similarity between accumulated situation vectors. Through this, a technical advantage can be secured in systematically modeling the interactions of complex real-world phenomena, such as discovering patterns in which traffic congestion in a specific area leads to an increase in pedestrian traffic in nearby areas, or analyzing the impact of large-scale events on the revitalization of surrounding commercial districts.

[0069] The present invention can be immediately utilized in various application fields such as smart city integrated control, autonomous vehicle route optimization, large-scale event safety management, disaster early warning, commercial area analysis, and site selection, and can provide high industrial value as an innovative platform that fills the technical gap in the field of offline real-world data processing and realizes the seamless integration of physical and digital spaces.

[0070] Meanwhile, the effects obtainable from the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present invention belongs from the description below. Brief explanation of the drawing

[0071] The following drawings attached to this specification illustrate preferred embodiments of the present invention and serve to further enhance understanding of the technical concept of the present invention together with the detailed description of the invention; therefore, the present invention should not be interpreted as being limited only to the matters described in such drawings. FIG. 1 is a conceptual diagram showing a system architecture according to one embodiment of the present invention as a four-layer pipeline structure. FIGS. 2a and 2b are block diagrams showing the detailed configuration of a multi-data collection and refinement unit (100) according to an embodiment of the present invention, illustrating a heterogeneous data continuous collection unit (110), a vision language model-based image parsing unit (120), and a dynamic event triggering unit (130). FIG. 3a shows a table illustrating various methods of natural language synthesis according to one embodiment of the present invention. FIG. 3b shows an example of simultaneous extraction of structured tags according to one embodiment of the present invention. FIG. 4 is a block diagram showing the detailed configuration of a situation vector embedding unit (300) according to an embodiment of the present invention, illustrating a high-dimensional continuous vector mapping unit (310) and a multidimensional context encoding unit (320). FIG. 5 is a block diagram showing the detailed configuration of a vector database and maintenance unit (400) according to an embodiment of the present invention, illustrating a time decay application unit (410) and a dynamic association graph generation unit (420). FIG. 6 is a block diagram showing the detailed configuration of a vector-based sub-application unit (500) according to one embodiment of the present invention. FIG. 7 is a flowchart illustrating the overall operation flow of a method for generating a real-world situation vector model according to an embodiment of the present invention, showing a heterogeneous offline data collection and event trigger step (S100), a natural language intermediate expression synthesis and tag extraction step reflecting confidence weights (S200), a situation vector generation step based on a pre-trained language model (S300), a time decay function application and vector model maintenance step (S400), and a sub-application provision step based on the real-world vector model (S500). FIG. 8 is a table showing examples of data collected from various data sources according to one embodiment of the present invention. FIG. 9 shows an example of a descriptive text generated as a result of natural language synthesis according to one embodiment of the present invention. FIG. 10 shows an example of a high-dimensional continuous vector generated through a text embedding model according to one embodiment of the present invention. FIG. 11 is a table showing the results for various search queries according to one embodiment of the present invention. Specific details for implementing the invention

[0072] The above objects, other objects, features, and advantages of the present invention will be easily understood through the following preferred embodiments associated with the accompanying drawings. However, the present invention is not limited to the embodiments described herein and may be embodied in other forms. Rather, the embodiments introduced herein are provided to ensure that the disclosed content is thorough and complete and to ensure that the spirit of the invention is sufficiently conveyed to a person skilled in the art.

[0073] In this specification, when a component is described as being on another component, it means that it may be formed directly on the other component or that a third component may be interposed between them. Also, in the drawings, the thicknesses of the components are exaggerated for the effective description of the technical content.

[0074] The embodiments described herein will be explained with reference to cross-sectional and / or plan views, which are exemplary illustrations of the invention. In the drawings, the thicknesses of films and regions are exaggerated for effective explanation of the technical content. Accordingly, the shapes of the exemplary drawings may be modified by manufacturing techniques and / or tolerances, etc. Accordingly, the embodiments of the invention are not limited to the specific shapes depicted but include variations in shape produced according to the manufacturing process. For example, a region depicted as a right angle may be rounded or have a certain curvature. Accordingly, the regions illustrated in the drawings have properties, and the shapes of the regions illustrated in the drawings are intended to illustrate specific shapes of the regions of the device and are not intended to limit the scope of the invention. Although terms such as first, second, etc., have been used to describe various components in the various embodiments of this specification, these components should not be limited by such terms. These terms are used merely to distinguish one component from another. The embodiments described and illustrated herein also include their complementary embodiments.

[0075] The terms used herein are for describing the embodiments and are not intended to limit the invention. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. As used herein, 'comprises' and / or 'comprising' do not exclude the presence or addition of one or more other components to the mentioned components.

[0076] In describing the specific embodiments below, various specific details have been included to explain the invention more specifically and to aid understanding. However, a reader with sufficient knowledge in the art to understand the invention will recognize that it can be used without these various specific details. In some cases, it is noted in advance that commonly known aspects that are not significantly related to the invention have been omitted to prevent unnecessary confusion in describing the invention.

[0078] The present invention relates to a system and method for integrating various heterogeneous offline data occurring in a specific spatiotemporal region of the physical real world and modeling the dynamic situation of that region in a mathematically expressible vector form. While existing data processing technologies have primarily targeted structured data in the online digital domain, fragmented data with completely different formats and structures—such as pedestrian traffic, CCTV footage, sensor measurements, social media mentions, and weather information—occurs simultaneously in real-world offline spaces. The core technical challenge that the present invention aims to solve is to effectively integrate such heterogeneous data, reflect the dynamic characteristics of real-world situations that change over time, and provide them in a form that can be utilized by physical autonomous systems, such as autonomous vehicles or robots.

[0080] Figure 1 shows the 4-layer pipeline architecture of the present system.

[0082] The system of the present invention is logically composed of five layers and can operate organically. The lowest layer, the multi-data collection and refinement unit (100), collects heterogeneous data from the physical real world, the natural language synthesis and tag extraction unit (200) converts the collected data into an integrated natural language intermediate representation, the context vector embedding unit (300) encodes the natural language representation into a high-dimensional vector, the vector database and maintenance unit (400) stores and manages the generated vector, and the highest layer, the vector-based sub-application unit (500), can provide various application services by utilizing the constructed vector model.

[0084] Each layer functions independently yet is interconnected, allowing complex real-world situational information to be progressively refined and abstracted, ultimately transforming into a mathematical vector model capable of computation. This hierarchical structure enhances system scalability and maintainability, while providing the flexibility to selectively apply various algorithms or models at each layer.

[0086] FIG. 2 illustrates a block diagram of a multi-data collection and refinement unit (100) according to one embodiment of the present invention.

[0088] The multi-data collection and refinement unit (100) can perform the role of collecting and refining various heterogeneous offline data occurring in a specific spatiotemporal region of the physical real world. The unit may include a heterogeneous data continuous collection unit (110), a vision language model-based image parsing unit (120), and a dynamic event triggering unit (130).

[0090] The continuous collection unit (110) of heterogeneous data can collect fragmented heterogeneous offline data occurring in a specific spatial unit in real time or periodically. Here, the spatial unit can be defined in various forms such as a grid, administrative district, geohash, etc., and can be appropriately selected according to the application field or required resolution. The data to be collected may include, but is not limited to, user-generated content, pedestrian traffic information through public APIs, weather information, POS sales data, traffic sensor data, noise sensor data, etc.

[0092] The collection unit (110) manages the interface with each data source and can have a flexible structure capable of accommodating data even if the data format differs, such as numerical, text, coordinate, or ratio format. The collected data can be processed into a form suitable for subsequent processing through refinement processes such as deduplication, noise filtering, coordinate normalization, and time synchronization. For example, location information expressed in different coordinate systems can be converted into a unified coordinate system, and time information recorded in various time zones and formats can be unified based on Coordinated Universal Time.

[0094] The vision language model-based image parsing unit (120) can go beyond simply analyzing visual data collected from CCTVs, drones, satellites, etc., on a pixel-by-pixel basis and can convert it into scene description data in the form of natural language containing context by utilizing a vision language model. The vision language model is a pre-trained large-scale model and can have the ability to recognize objects, actions, and scene compositions within the image and express them as natural language sentences.

[0096] The parsing unit (120) can selectively sample frames in which meaningful changes occur from a video stream and input them into a vision language model. The model can analyze the input video to generate specific scene descriptions such as "20 tent booths installed on the road, a queue of about 200 meters" or "a large number of people moving in a specific direction in a pedestrian-dense area of ​​a commercial district." Metadata such as the time of video capture, location coordinates, and camera identifier can be automatically attached to the generated natural language description.

[0098] In this way, unstructured image data can be standardized into a natural language form that can be processed by a text embedding model, and this can be seamlessly integrated with other text data in a subsequent natural language synthesis step. According to some embodiments, various models such as GPT-4V, LLaVA, and BLIP-2 may be selectively applied as vision language models, and fine-tuned models specialized for a specific domain may also be used.

[0100] The dynamic event triggering unit (130) can activate the operation of subsequent layers only when specific conditions are met, instead of passing all data through the entire pipeline at all times to efficiently manage system resources. The triggering unit (130) can monitor the amount of change in pedestrian traffic in a specific spatiotemporal area or the amount of mentions of related keywords in real time.

[0102] Changes in the volume of floating population can be tracked through carrier base station data, Wi-Fi connection logs, transportation card usage records, etc., and cases where it increases by more than a certain percentage compared to normal times can be detected. Text mention volume can be measured by tracking the frequency of appearance of keywords related to a specific region in social media, news, communities, etc. If these indicators exceed a preset threshold, the corresponding spatiotemporal area can be identified as an event candidate, and the operation of the subsequent layers, the natural language synthesis and tag extraction unit (200) and the situation vector embedding unit (300), can be triggered.

[0104] Trigger conditions can be set not only as a single indicator but also as complex conditions combining multiple indicators. For example, an AND logical operation can be applied to trigger the event only when mentions of a specific keyword surge simultaneously with an increase in foot traffic, while an OR logical operation can be applied to trigger the event if either condition is met. Thresholds can be dynamically adjusted based on regional characteristics, time of day, day of the week, etc., and can be optimized through the analysis of historical data.

[0106] This dynamic triggering mechanism enhances the efficiency of computing resources by processing data intensively only at the time of meaningful events, while simultaneously ensuring real-time capability to rapidly capture significant situational changes.

[0108] Figure 3a shows various methods of natural language synthesis, and Figure 3b shows an example of simultaneous extraction of structured tags.

[0110] The natural language synthesis and tag extraction unit (200) can fuse collected heterogeneous offline data and convert it into an intermediate expression in the form of natural language that describes the situation in the spatiotemporal domain. The unit may include a natural language intermediate expression converter (210), a reliability-based data weighting application unit (220), and a structured tag parallel extraction unit (230).

[0112] The natural language intermediate expression converter (210) can synthesize these into a single complete natural language description to solve the problem that collected heterogeneous data cannot be directly input into an existing text embedding model because they have completely different formats, such as numbers, ratios, and coordinates. The converter (210) can perform natural language synthesis using a large language model or a rule-based template.

[0114] When utilizing a large language model, all collected data can be input into the model in a structured form, and prompts can be designed to enable the model to convert this into natural sentences. For example, along with instructions such as "Integrate the following data to synthesize a single natural sentence," data such as pedestrian traffic figures, sales growth rates, location information, and social media mentions can be provided. The model can analyze this data to generate a descriptive text with rich context, such as "Due to the hosting of a large pop-up store in Seongsu-dong, Seongdong-gu, Seoul, approximately 12,000 visitors were generated, which is 3.4 times the usual level, causing waiting times of over 30 minutes at nearby cafes and restaurants."

[0116] When utilizing rule-based templates, predefined sentence structures can be applied to data of specific patterns. For example, sentences can be generated by mapping collected data to a template such as "at + location name + due to + event type + phenomenon + occurred". This approach may have the advantage of fast processing speed and high consistency of results.

[0118] According to some embodiments, large language models and rule-based templates may be utilized in a hybrid manner. By rapidly processing structured data with templates and handling complex or unstructured data with language models, both efficiency and quality can be ensured simultaneously.

[0120] The reliability-based data weight application unit (220) can synthesize conflicting or noisy data collected from multiple sources by applying weights based on reliability assigned to each data source. A reliability score indicating the reliability of the source may be assigned to each data source, which can be manually set by an administrator or automatically calculated through an analysis of the accuracy of past data.

[0122] For example, official traffic information provided by public institutions can have high reliability, as can mobile carriers' population mobility data. Conversely, personal blogs or unverified social media posts may have relatively low reliability. During the natural language synthesis process, weights proportional to these reliability scores are applied, allowing the content of highly reliable data to be more strongly reflected in the final explanatory text.

[0124] Specifically, in the event of conflicting information, information from a source with higher reliability may be adopted preferentially. For example, if official weather agency data indicates "clear" while a personal blog mentions "rain," the weather agency data with higher reliability may be reflected in the final description. Additionally, if multiple sources provide similar information, the final figure or expression may be determined through a weighted average based on the reliability of each source.

[0126] According to some embodiments, reliability weights may be dynamically adjusted over time rather than being fixed values. A learning-based adjustment mechanism may be applied to track the degree to which past predictions or reports from a specific data source match the actual situation, thereby increasing the reliability of sources with high accuracy and decreasing the reliability of sources with low accuracy.

[0128] The structured tag parallel extraction unit (230) can simultaneously synthesize natural language documents and simultaneously extract and structure standardized metadata tags, such as scale, category, and spatial attributes of the corresponding event, in parallel. The extracted tags may include category, scale, spatial type, time zone, sentiment, etc.

[0130] Category tags can indicate the type of situation, such as traffic, safety, events, weather, or commercial activity, and can be automatically assigned through a classification model. Scale tags can be classified into small, medium, or large scales, and can be determined based on pedestrian traffic figures or the scope of influence. Spatial type tags can represent commercial districts, residential areas, parks, transportation facilities, public facilities, etc., and can be determined by matching location information with a spatial database.

[0132] Time zone tags can indicate temporal characteristics such as commuting times, lunch breaks, nighttime, and weekends, while sentiment tags can be classified into positive, negative, or neutral through sentiment analysis of social media data. These tags can be extracted using various methods, including entity extraction based on Named Entity Recognition, automatic category assignment via classification models, and rule-based pattern matching.

[0134] The extracted structured tags are stored along with natural language intermediate representations and can be utilized for hard filtering in subsequent search and application stages. For example, if a user wishes to search only for "indoor events," search efficiency and accuracy can be improved by first filtering only for vectors with a spatial type tag of "indoor" and then performing a vector similarity search.

[0136] FIG. 4 illustrates a block diagram of a situation vector embedding unit (300) according to one embodiment of the present invention.

[0138] The situation vector embedding unit (300) can input an intermediate representation in the form of a converted natural language into a text embedding model to generate a continuous situation vector encoded with a real-world situation context in a spatiotemporal domain. The unit may include a high-dimensional continuous vector mapping unit (310) and a multidimensional context encoding unit (320).

[0140] The high-dimensional continuous vector mapping unit (310) can generate a context vector, which is a high-dimensional single numeric array, by passing the synthesized natural language document through a text embedding model based on a pre-trained language model. The text embedding model is a model that receives natural language text as input and outputs a vector that mathematically expresses its meaning, and various models such as BERT, RoBERTa, Sentence-BERT, and OpenAI Embeddings can be utilized.

[0142] The mapping unit (310) can input a natural language intermediate representation into a selected embedding model and receive a high-dimensional vector output by the model. The number of dimensions of the vector may vary depending on the model used, and generally, dense vectors of hundreds to thousands of dimensions can be generated. For example, the Sentence-BERT model can generate a 768-dimensional vector, and OpenAI's text-embedding-ada-002 model can generate a 1536-dimensional vector.

[0144] According to some embodiments, instead of using a general-purpose pre-trained model as is, domain fit can be improved by performing additional fine-tuning with a spatiotemporal context description dataset. For example, by fine-tuning the model using previously collected real-world context descriptions and corresponding category or similarity information as training data, the subtle nuances of offline real-world situations can be reflected more accurately in the vector space.

[0146] The multidimensional context encoding unit (320) can mathematically encode complex real-world situational contexts, such as the physical location of the corresponding space-time, population size, nature of the event, atmosphere of the site, accessibility, and weather constraints, into a single generated situational vector. The text embedding model can learn the relationships between words, semantic nuances, and contextual information within a natural language sentence and express them as locations in a vector space.

[0148] For example, expressions such as "large crowds" and "small gatherings" can be mapped to different locations in the vector space, and "indoor events" and "outdoor events" can also be distinguished. In addition, emotional nuances such as "positive mood" and "negative mood" can also be reflected in the vector. Through this multidimensional context encoding, a single vector can comprehensively represent the complex characteristics of a real-world situation, which can be utilized in applications such as subsequent search, classification, and prediction.

[0150] According to some embodiments, metadata tags extracted from the structured tag parallel extraction unit (230) along with natural language vectors can also be vectorized and integrated. Metadata tags can be converted into vectors through one-hot encoding or an embedding layer, and can be combined with natural language vectors to form a final situation vector. Various methods can be applied for the combination, such as simple concatenation, weighted summation, and dynamic weighting through an attention mechanism.

[0152] The generated context vectors can be adjusted to unit length through L2 normalization, which optimizes subsequent cosine similarity calculations. Additionally, extremely biased vectors or outliers are filtered out, ensuring the quality of the vector database.

[0154] FIG. 5 illustrates a block diagram of a vector database and a maintenance unit (400) according to one embodiment of the present invention.

[0156] The vector database and maintenance unit (400) can store and maintain the generated situation vector in the database. The unit may include a time decay application unit (410) and a dynamic association graph generation unit (420).

[0158] The time decay application unit (410) may apply a time decay function to a situation vector stored in a vector database to reflect the fact that offline real-world situations have dynamic characteristics that decay over time, unlike static online web documents. The time decay function may gradually decrease the relevance or search weight of the corresponding vector in proportion to the time elapsed since the time the vector was created.

[0160] The form of the time decay function can be selected in various ways. A linear decay function is a form in which weights decrease uniformly in proportion to time, and it can have simple and predictable characteristics. An exponential decay function decreases slowly initially and then decreases rapidly over time, allowing higher weight to be assigned to recent information. A logarithmic decay function decreases rapidly initially and then gradually flattens out, allowing older information to maintain a certain level of weight.

[0162] According to some embodiments, differential damping functions may be applied depending on the type of event. For example, for temporary events such as concerts or pop-up stores, a fast damping function may be applied so that the weight decreases rapidly after the event ends, while for continuous phenomena such as construction or changes in commercial areas, a slow damping function may be applied so that the weight is maintained for a long period.

[0164] The application unit (410) can recalculate the time decay weights of all stored vectors at set intervals. The recalculation interval can be set in units of hours, days, etc., and can be determined by considering the system's real-time requirements and computing resources. The updated weights are stored in vector metadata and can be referenced during searching or application processes.

[0166] Through this time decay mechanism, higher priority is given to the latest situation and the influence of outdated information is automatically reduced, allowing the system to always reflect the latest state.

[0168] The dynamic association graph generation unit (420) can automatically generate and update an event association graph by periodically calculating the cosine similarity between accumulated vectors. Cosine similarity is an indicator that evaluates similarity by measuring the angle between two vectors, and vectors located close to each other in a vector space can have high similarity.

[0170] The generation unit (420) can calculate the similarity with existing vectors whenever a new vector is added. Pairs of vectors with a similarity greater than a preset threshold can be connected by edges to form a graph structure. For example, if the similarity between the "Seongsu-dong Pop-up Store" vector and the "Hannam-dong Pop-up Store" vector is high, the two vectors can be connected on the graph, which can indicate that the two events have similar characteristics.

[0172] When generating the graph structure, additional spatiotemporal constraints may be applied. Physically adjacent regions or temporally close events can be preferentially connected, which can be utilized to identify local or temporal chains of events. The generated graph can be stored in a graph database, and closely related event clusters can be identified through community detection algorithms.

[0174] These dynamic association graphs can be utilized for predicting chain events, analyzing causal relationships, and discovering patterns. For example, a pattern where sales at nearby cafes increase when a pop-up store is held in a specific area can be identified through the graph, and this can be used to predict similar events in the future.

[0176] According to some embodiments, the vector database and maintenance unit (400) may additionally include a vector lifecycle management function. Different retention periods may be set for each event type, and after the set period has elapsed, the vector may be moved to cold storage or deleted. Duplicate vectors with extremely high similarity may be merged, and vectors determined to have low reliability or be noise may be removed.

[0178] FIG. 6 illustrates a block diagram of a vector-based sub-application unit (500) according to one embodiment of the present invention.

[0180] The vector-based sub-application unit (500) can independently perform various application services suitable for a specific domain by calling the constructed general-purpose real-world vector model. The unit may include a search and re-ranking module, an automatic classification module, a time series forecasting module, an anomaly detection module, and an autonomous system context provision module.

[0182] The search and re-ranking module can perform a search by vectorizing a user's natural language query and calculating the similarity with generated context vectors. When a user inputs an abstract query such as "a hip place to go even if it rains," this module can vectorize the query into the same text embedding model. The cosine similarity between the generated query vector and all stored context vectors can be calculated, and the results can be sorted in order of highest similarity.

[0184] Spatial filtering and tag filtering can be combined during the search process. For example, if a user specifies a particular geographical range, the search can be limited to vectors within that range. Additionally, if a condition such as "indoor" is present, vectors with the spatial type tag "indoor" can be filtered first, followed by a vector similarity search. This hybrid search method can simultaneously improve search efficiency and accuracy.

[0186] According to some embodiments, search results can be re-ranked through a large language model. After the top K candidates are selected based on vector similarity, the language model can analyze each candidate's natural language description and user query together to evaluate more accurate relevance and adjust the final ranking.

[0188] The automatic classification module can automatically classify and tag newly generated situation vectors to the closest category by comparing them with the anchor vector space of predefined major categories. Anchor vectors are vectors representing each category and can be generated by embedding typical situational descriptions of that category. For example, an anchor vector for the "Festival" category can be generated from descriptions such as "large-scale outdoor festival, various performances and booths, large crowds."

[0190] This module calculates the distance between a new situation vector and all anchor vectors and can assign the category of the nearest anchor to the corresponding vector. If there is high similarity with multiple anchors, multiple labels may be assigned. A confidence score may be provided along with the classification result, which indicates the reliability of the classification.

[0192] The time series prediction module can predict future situations or surges such as delivery demand by tracking the time-series change patterns of situation vectors generated within the same spatial unit. Past situation vectors of a specific region can be arranged in chronological order to construct a time-series sequence, which can then be input into recurrent neural networks, LSTMs, GRUs, or Transformer-based prediction models.

[0194] The model can predict the situation vector at the next point in time by learning past vector sequences. For example, if similar pattern vectors occur in a specific commercial district every Saturday afternoon, the model can predict the situation vector for the following Saturday afternoon. The predicted vector can be decoded into natural language and provided as interpretable predictive information.

[0196] These prediction capabilities can be utilized in various fields, such as forecasting demand for delivery services, predicting traffic congestion, and predicting the number of event participants.

[0198] The anomaly detection module can immediately identify accidents and anomalies and generate alerts when the current situation vector of a specific spatiotemporal location deviates sharply from the normally accumulated vector pattern or rapidly approaches a predefined disaster anchor vector. This module can learn the normal vector distribution of a specific region, which can be accomplished through anomaly detection algorithms such as Autoencoder, Isolation Forest, and One-Class SVM.

[0200] When a new vector is generated, this module can evaluate how far it deviates from the normal distribution. When using an autoencoder, the vector's reconstruction error can be calculated, and if the error exceeds a threshold, it may be identified as an anomaly. Additionally, an anomaly may be identified if the similarity to a predefined disaster anchor vector increases rapidly.

[0202] When an abnormal situation is detected, an immediate notification can be sent to the administrator, which may include information such as the location, type, and severity of the anomaly. This enables the early detection and rapid response to disasters, accidents, and abnormal crowd behavior.

[0204] The autonomous system context provision module transmits generated real-time situation vectors or inference results based thereon to autonomous vehicles, delivery robots, etc., via external APIs, enabling them to be utilized as core context for situation recognition—such as identifying that a specific area is currently crowded with people due to a festival—and for determining alternative routes. This module can provide RESTful API or gRPC interfaces, through which autonomous systems can query real-time situation information.

[0206] When an autonomous vehicle transmits a destination route to the system prior to departure, this module can retrieve the latest situation vectors for all spatiotemporal regions along the route. The retrieved vectors can be converted into natural language or structured JSON format and returned. For example, if the situation in a specific section is described as "road surface wet due to rainfall, reduce speed," the vehicle can recognize this and drive at a reduced speed or select an alternative route.

[0208] According to some embodiments, the module may support real-time streaming via WebSocket or Server-Sent Events. When the conditions of the path the autonomous system is traveling change, the changed information can be pushed immediately, and the system can replan the path in real time based on this.

[0210] FIG. 7 illustrates a flowchart of a method for generating a real-world situation vector model according to an embodiment of the present invention.

[0212] FIG. 8 is a table showing examples of data collected from various data sources according to an embodiment of the present invention. FIG. 9 is an example of a descriptive text generated as a result of natural language synthesis according to an embodiment of the present invention. FIG. 10 is an example of a high-dimensional continuous vector generated through a text embedding model according to an embodiment of the present invention. FIG. 11 is a table showing results for various search queries according to an embodiment of the present invention.

[0214] The present method may include a heterogeneous offline data collection and event trigger step (S100), a natural language intermediate expression synthesis and tag extraction step reflecting confidence weights (S200), a situation vector generation step based on a pre-trained language model (S300), a time decay function application and vector model maintenance step (S400), and a sub-application provision step based on a real-world vector model (S500).

[0216] In the heterogeneous offline data collection and event triggering step (S100), the system can monitor pedestrian traffic and metadata by spatial unit. Data is accumulated normally, and when the amount of change in pedestrian traffic in a specific area or related social mentions exceeds a predetermined threshold, it can be triggered as an event. For the triggered spatiotemporal area, all available heterogeneous fragmented data, such as geographic information, weather information, nearby POS sales information, user-generated content within the app, and vision-based scene description text obtained through CCTV, can be intensively collected and refined.

[0218] The collected data sources may include, but are not limited to, mobile carrier mobility data, social media posts, public APIs, POS sales data, IoT sensor data, and CCTV footage. In particular, CCTV or drone footage can be analyzed by a vision language model and converted into scene descriptions in the form of natural language text, which can be fed into subsequent stages just like other text data.

[0220] In the step of synthesizing natural language intermediate expressions and extracting tags reflecting reliability weights (S200), collected data in the form of numerical values, ratios, locations, and text can be converted into a single natural language descriptive text with rich context through a language model or a predefined template algorithm, without directly merging them. At this time, the accuracy of the natural language document can be guaranteed by assigning weights based on the source reliability of each data source, and structured tag data such as the scale, category, and spatial attributes of the corresponding event can be extracted in parallel and structured simultaneously with the natural language conversion.

[0222] For example, when data such as pedestrian traffic, social media mentions, and CCTV video analysis results are collected, the language model can synthesize them to generate a description such as, "Large crowds gathered at a large pop-up store in Seongsu-dong, Seongdong-gu, Seoul for the sale of limited edition goods; current visitors number approximately 12,000, which is 3.4 times the usual level, with waiting times of over 30 minutes at nearby cafes and restaurants." At the same time, tags such as "event" for category, "large scale" for scale, and "commercial district" for space type can be extracted.

[0223] For example, in autonomous driving, intersection CCTV footage → VLM generates a scene description of 'approx. 3,000 pedestrians crowded, occupying the road' + traffic sensor 'average vehicle speed 5 km / h' + surge in social media mentions of '○○ festival start' → synthesizes the three data into a single natural language: "3,000 pedestrians occupying the road due to the ○○ intersection festival, vehicle traffic virtually impossible, festival scheduled to end at 6 PM" → creates a situation vector → an agent of the autonomous vehicle receives the situation vector and performs context-based decisions: selecting an alternative route if it is a long-lasting event (festival), or selecting to slow down and wait if it is an event expected to resolve quickly (temporary pedestrian crowding). Even for the same 'pedestrian crowding,' the vehicle can autonomously decide on different behavioral strategies depending on the nature, expected duration, and scale of the event encoded in the situation vector. Through this, autonomous driving systems such as robotaxis can make appropriate decisions based on real-time situational context even in long-tail situations (irregular festivals, impromptu rallies, temporary control, etc.) that are not present in prior training data.

[0225] For example, disaster response (119 Comprehensive Situation Solution) converts 119 call voice STT to "Smoke from the 2nd floor of a commercial building in ○○-dong, people are running out" + video from 4 CCTV cameras within the building's radius → VLM analysis: "Black smoke leaking from a 3rd-floor window, approximately 30 people evacuating from the 1st-floor entrance, insufficient space for fire trucks on the road" + real-time collection of SNS / UGC from a 500m radius: "○○ building is on fire, photo attached," "Strong smell of smoke, evacuating" (3 entries) + weather API: "Wind speed 15m / s southerly wind" + pedestrian traffic data for the area: "2.1 times higher than usual (affected by a nearby event)" → synthesizes all data into a single natural language: "Fire presumed in a 5-story commercial building in ○○-dong. Smoke leak confirmed from the 3rd floor, evacuation of approximately 30 people from the 1st floor in progress. Concerns about smoke spreading to northern residential areas due to a 15m / s southerly wind. Current pedestrian traffic in the area is 2.1 times higher than usual, high risk of secondary damage." "Detour needed due to traffic congestion on fire truck access route" → Situation vector generated → Data such as firefighters arriving at the scene over time, additional reports ("Flames on the 4th floor as well"), and new CCTV frames ("Flame visualization") are joined to the existing event document for updating and re-embedding (Living Event Timeline) → Vector-based severity changes are tracked on the commander's dashboard to determine additional dispatches, road closures, and expansion of evacuation areas.

[0227] For example, Industrial Smart Factory + Patrol Robot: Camera on a quadruped patrol robot inside a semiconductor factory cleanroom → VLM Analysis: "Front panel warning light flashing on etching equipment on Line 3, Building B; traces of liquid on the floor; 2 workers within a 3m radius" + Raw data from the IoT sensor of the corresponding etching equipment: "Chamber internal pressure -2.3kPa (deviation from normal range -1.8~-2.0kPa), coolant temperature 28.7°C (deviation from normal 22±2°C), gas flow rate deviation +12%" + Equipment MES Log: "72 hours elapsed since last PM (Preventive Maintenance); history of similar pressure deviations on the same equipment 3 times / month" + Cleanroom Environment Sensor: "Particle count exceeding Class 100 threshold detected" → Natural Language Synthesis: "Signs of anomaly in etching equipment on Line 3, Building B. Chamber pressure, coolant temperature, and gas flow rate simultaneously deviated from normal ranges. Coolant leak suspected due to liquid traces on the floor. Equipment with 72 hours elapsed since PM; 3 recent similar anomalies" History. Cleanroom particle standards exceeded. Two workers in close proximity" → Situation vector generated → Detected a sudden deviation from the existing normal operation pattern vector (anomaly detection module) → Immediate notification: Recommendation to evacuate workers + automatic line stop + dispatch engineer → Subsequently, the robot patrols at 5-minute intervals, accumulating and updating new video and sensor data in the document, and the vector reflects the "deterioration / stabilization" trend in real time.

[0229] In the pre-trained language model-based situation vector generation step (S300), a single natural language description derived can be input into a pre-trained text embedding model. The model can generate a general-purpose real-world model by analyzing the semantic context embedded in the text and outputting a high-dimensional continuous vector representation capable of mathematical operations. The generated vector can mathematically encode complex situational contexts such as the physical location, population size, nature of the event, atmosphere of the site, accessibility, and weather constraints of the corresponding spatiotemporal region.

[0231] In the step of applying a time decay function and maintaining a vector model (S400), the generated real-world situation vector can be stored in a vector database along with meta tags. To reflect the inherent characteristic that offline phenomena are transient, a time decay algorithm can be periodically applied to reduce the influence of existing vectors and maintain real-time relevance. Additionally, the event association graph can be continuously updated by forming association clusters based on the similarity between newly entered vectors and existing vectors.

[0233] Structure A (baseline + real-time correction) generates a baseline vector based on historical time series and updates the correction with real-time data.

[0234] Structure B (Similarity Point Vector Transition) transfers a similarity point pattern within the vector space to a new point without data.

[0235] Structure D (situation trajectory) generates a trend vector by resynthesizing the chronological vector change pattern into natural language.

[0236] Specifically,

[0237] - Structure A (Baseline Profile + Real-time Correction) -

[0238] The derivation location is the time axis extension of Layer 2 (natural language synthesis).

[0239] It is a structure designed to overcome the limitation that real-time sensor data alone cannot cover all locations.

[0240] 1) Baseline Generation: Statistical analysis of historical time-series data → Synthesis of natural language profiles using LLM → Vector embedding

[0241] 2) Real-time Correction: Convert real-time data into natural language → Update situation vector by correcting the current situation relative to the baseline

[0242] Examples by domain:

[0243] Autonomous Driving (FSD): Intersection baseline (commute congestion patterns) + Real-time (under construction) → Corrected for severe congestion

[0244] Autonomous Robot: Delivery Zone Baseline (Pedestrian Density during Lunchtime) + Real-time (Rain Forecast) → Corrected for Reduced Pedestrians

[0245] Smart City: Road section baseline (weekday evening congestion) + Real-time (festival held) → Adjusted to extreme congestion

[0246] Hash# App: Commercial area baseline (Saturday afternoon peak) + Real-time (rain forecast) → Adjusted for reduced foot traffic

[0247] - Structure B (Similar Point Vector Transfer) -

[0248] Derived location: Layer 34 vector space) utilization

[0249] It is a structure that generates a situation vector for a new point without data from existing points with similar characteristics within a vector space. This is possible because the vector space of the Real World Model exists, and the methodology solves the cold start problem on its own.

[0250] Process: Synthesize basic attributes of the new point (location, type, environment, etc.) in natural language to create embeddings → Find the baseline pattern of the most similar existing point in the vector space and apply it as the initial vector → Gradually replace as actual data accumulates

[0251] Examples by domain:

[0252] Autonomous Driving (FSD): When there is no driving history at a newly opened intersection, transfer existing intersection patterns with similar characteristics, such as "four lanes in each direction, in front of a school, with traffic lights."

[0253] Autonomous robot: When deployed to a new zone, transfers existing zone patterns with similar road environments (pedestrian density, road width, obstacle types)

[0254] Smart City: Transferring traffic patterns from similar road sections to newly installed sensor sections

[0255] Disaster Response: Estimating initial risk in new areas lacking damage data by transferring situational patterns from past affected areas with similar topography, population density, and building types. Hash# App: Transferring patterns from similar commercial areas to new commercial districts lacking time-series data.

[0256] - Structure C. Multi-resolution Vector-

[0257] Derived Location: Spatial Axis Extension of Layer 2 Natural Language Synthesis)

[0258] It is a structure that simultaneously generates different situation vectors from the same raw data according to spatial resolution (micro / meso / macro). Since the natural language synthesis itself changes depending on the resolution, the meaning contained in the vectors is qualitatively different.

[0259] Examples by domain:

[0260] Autonomous Driving (FSD): "This lane: Vehicle ahead brakes suddenly" (Micro, immediate avoidance) vs "This intersection: Overall congestion" (Medium) vs "Gangnam Station to Yeoksam Station section: Commute congestion" (Macro, route recalculation)

[0261] Autonomous Robot: "This sidewalk: Construction obstacle (Micro) vs "This block: Pedestrian density (Medium) vs "This entire area: Lunchtime congestion (Macro)

[0262] Military / Defense: "This building: Heat source detected" (Micro) vs "This block: Civilian density" (Medium) vs "Entire area: Engagement" (Macro, operational level assessment)

[0263] Hash# App: "In front of this cafe: 20-minute wait" (Mild) vs "Hongdae commercial district: Crowded" (Medium) vs "Mapo-gu as a whole: Weekend peak" (Macro)

[0264] - Structure D. Situation Trajectory -

[0265] Recursive application of Derivation Location: Layer 2 Natural Language Synthesis

[0266] It is a structure designed to determine not only "what the current situation is" but also "where it is heading." It generates a trend vector by synthesizing the change patterns of the chronological situation vector itself into natural language.

[0267] Examples by domain:

[0268] Autonomous Driving (FSD): Even with the same current condition, the decision to detour changes completely depending on whether "current congestion" or "worsening over the past 30 minutes" applies. Autonomous Driving Robots: "currently high pedestrian traffic" or "decreasing pedestrian traffic" influences the decision between waiting and starting immediately.

[0269] Smart City: "Current Congestion" and "Rapidly Worsening" Are Key Factors in Determining Preemptive Signal System Response

[0270] Disaster Response: "Current flooding of 50cm" and "Rapidly rising over the last hour" determine whether an immediate evacuation order is issued. Hash# App: "Currently crowded" and "Leaving empty" or "Getting more crowded" are directly used to decide when to visit.

[0272] Existing systems have a one-off structure where they send a notification once upon detecting an event and then stop. However, real-world events evolve over time. For example, after the 'Seongsu-dong Pop-up Store' event is first created, additional UGC is uploaded, new footage comes in from CCTV, and pedestrian traffic data is updated—this new data joins the natural language document of the existing event to update the document, and the updated document is then re-embedded as a vector.

[0274] In other words, for a single event, the natural language document is accumulated and updated over time as a living document, and the vector is regenerated each time. Through this, the vector always reflects the latest state of 'what the current event is'.

[0276] In the sub-application provision step (S500) based on the real-world vector model, at least one application process can be executed based on the generated and maintained real-world situation vectors, including hybrid situation search and language model reranking combining vector similarity and spatial filtering, automatic category classification based on anchor vector comparison, future situation and demand forecasting using time-series vector accumulated data, automatic identification of abnormal situations and accidents through detection of deviation from normal vector patterns, and provision of real-time context information APIs for external autonomous systems driving in the physical world.

[0278] According to some embodiments of the present invention, various models such as BERT, RoBERTa, Sentence-BERT, and OpenAI Embeddings may be selectively applied as text embedding models. Each model may differ in terms of vector dimensionality, processing speed, and semantic representation capabilities, and an appropriate model may be selected according to the requirements of the application field.

[0280] According to some other embodiments, various models such as Gemini 3,1 Flash GPT-4V, LLaVA, BLIP-2, and Flamingo may be used as vision language models. A fine-tuned model specialized for a specific domain may also be used, which can improve the accuracy of image analysis in that domain.

[0282] According to some embodiments, the time decay function may be selected as one of linear, exponential, or logarithmic functions, and different decay functions may be applied depending on the event type. For example, an exponential decay function may be applied to transient events so that weights decrease rapidly, while a logarithmic decay function may be applied to continuous phenomena so that weights decrease slowly.

[0284] According to some embodiments, confidence weights may not be fixed values ​​but may be automatically adjusted based on learning. Reinforcement learning or Bayesian methods may be applied by tracking the historical accuracy of each data source to increase the confidence of sources with high accuracy and decrease the confidence of sources with low accuracy.

[0286] According to some embodiments, a multilayer vector representation may be utilized. Global vectors can represent the macroscopic situation of the entire spatiotemporal domain, while local vectors can represent the microscopic situation of specific regions. Through this hierarchical vector structure, situation analysis at various resolutions may be possible.

[0287] Structure C has a structure where the natural language synthesis itself varies qualitatively depending on micro, meso, and macro resolutions within the same raw data. For example, 'This lane: car ahead braked suddenly' (micro) vs. 'This intersection: general congestion' (meso) vs. 'Gangnam Station to Yeoksam Station section: evening rush hour congestion' (macro).

[0289] According to some embodiments, multilingual embedding models may be utilized for multilingual support. Using models such as mBERT and XLM-R, multilingual social media data can be processed in an integrated manner in areas with a high concentration of foreign tourists, which can enhance usability in global cities.

[0291] According to some embodiments, edge computing-based distributed processing may be applied. By deploying edge devices in each spatiotemporal region to perform primary data collection and refinement locally and transmitting them to a central server to generate an integrated vector, network bandwidth can be saved and real-time performance can be improved.

[0293] According to some embodiments, privacy protection techniques may be applied. Personal identification information may be automatically masked, and differential privacy may be applied to comply with personal information protection regulations. Distributed model training through federated learning may also be performed.

[0295] According to the present invention, complex and heterogeneous offline data of the physical real world can be effectively integrated and represented as a single vector model. Through technical features such as natural language conversion of image data using a vision language model, data quality control through the application of confidence-based weights, and the reflection of dynamic characteristics through a time decay function, accurate digital representation of real-world situations can be achieved.

[0297] By activating the data processing pipeline only when meaningful events occur through a dynamic event triggering mechanism, the efficiency of computing resources can be increased, while simultaneously ensuring real-time capability to rapidly capture critical situational changes.

[0299] The constructed vector model can be utilized in various application fields such as search, classification, prediction, anomaly detection, and providing context for autonomous systems, and can be immediately applied in a wide range of industries including smart cities, autonomous driving, disaster management, and commercial area analysis.

[0301] The embodiments of the present invention described above are not implemented only through devices and methods, but may also be implemented through a program that realizes a function corresponding to the configuration of the embodiments of the present invention or a recording medium on which such program is recorded.

[0303] Although embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements by those skilled in the art using the basic concept of the present invention as defined in the following claims also fall within the scope of the present invention. Explanation of the symbols

[0304] 100: Multi-data collection and refinement unit 110: Continuous collection unit of heterogeneous data 120: Vision Language Model-Based Image Parsing Unit 130: Dynamic Event Triggering Section 200: Natural Language Synthesis and Tag Extraction Section 210: Natural Language Intermediate Representation Converter 220: Confidence-based data weighting application section 230: Structured Tag Parallel Extraction Unit 300: Context vector embedding section 310: High-dimensional continuous vector mapping section 320: Multidimensional Context Encoding Unit 400: Vector Database and Maintenance Department 410: Time decay application section 420: Dynamic Association Graph Generation Unit 500: Vector-based sub-applications

Claims

Claim 1 A system for generating a real-world situation vector model based on natural language synthesis, comprising: a multi-data collection and refinement unit for collecting heterogeneous offline data occurring in a specific spatiotemporal region; a data fusion unit for fusing the collected data and a natural language synthesis unit for converting the data into an intermediate representation in the form of natural language that describes the situation in the spatiotemporal region; a situation vector embedding unit for inputting the converted intermediate representation in the form of natural language into a text embedding model to generate a continuous situation vector encoded with the real-world situation context of the spatiotemporal region; and a vector database and maintenance unit for storing and maintaining the generated situation vector in a database. Claim 2 A natural language synthesis-based real-world situation vector model generation system according to claim 1, wherein the multi-data collection and refinement unit includes: a dynamic event triggering module that detects when the amount of change in the floating population of the spatiotemporal area or the amount of mention of a specific text exceeds a preset threshold and triggers data collection and subsequent layer operations; and a vision language model-based image parsing module that analyzes image data collected from the outside using a vision language model (VLM) and converts it into text-based scene description data for collection. Claim 3 A natural language synthesis-based real-world situation vector model generation system according to claim 1, wherein the natural language synthesis unit synthesizes numeric, ratio, and coordinate data included in the collected heterogeneous offline data into a single complete natural language descriptive text using a language model or a rule-based template, and synthesizes by applying different weights based on a reliability pre-assigned to each data source, and when synthesizing intermediate expressions in the form of natural language, it extracts structured metadata tags including categories, scales, and spatial types of the spatiotemporal domain in parallel. Claim 4 A natural language synthesis-based real-world situation vector model generation system according to claim 1, wherein the vector database and maintenance unit comprises: a time decay application unit that applies a time decay function to decrease the relevance weight of the situation vector in proportion to the time elapsed since the occurrence of the stored situation vector in order to reflect the dynamic characteristics of offline real-world situations; and a dynamic association graph generation unit that calculates the similarity between accumulated situation vectors and periodically generates and updates an event-to-event association graph. Claim 5 A natural language synthesis-based real-world situation vector model generation system, further comprising: a vector-based sub-application unit that calls the generated and maintained situation vector to perform at least one application process among situation search, automatic classification, time series forecasting, anomaly detection, and context provision to an external autonomous system, in the first aspect. Claim 6 In claim 5, the vector-based sub-application unit is characterized by performing at least one of the following: a function of automatically classifying categories by comparing the situation vector with a predefined anchor vector space; a function of predicting future situations by tracking the time-series change patterns of situation vectors generated in the same spatial unit; a function of identifying an abnormal situation when the situation vector deviates abruptly from a previously accumulated vector pattern; and a function of providing real-time situation vectors of a specific spatiotemporal region to an external autonomous vehicle or robot for use in path determination. Claim 7 A method for generating a real-world situation vector model based on natural language synthesis of multiple offline data, comprising: a step of collecting heterogeneous offline data occurring in a specific spatiotemporal region; a step of fusing the collected heterogeneous offline data to convert it into an intermediate representation in the form of natural language that describes the situation in the spatiotemporal region; a step of inputting the converted intermediate representation in the form of natural language into a learned text embedding model to generate a continuous situation vector encoded with the real-world situation context of the spatiotemporal region; and a step of storing and maintaining the generated situation vector in a database. Claim 8 A method for generating a real-world situation vector model according to claim 7, wherein the step of converting into an intermediate expression in the form of natural language comprises synthesizing the intermediate expression in the form of natural language by assigning weights according to the reliability pre-assigned to each source of the heterogeneous offline data, and simultaneously extracting and structuring standardized tag data for the real-world situation of the corresponding spatiotemporal domain during the synthesis process. Claim 9 A method for generating a real-world situation vector model according to claim 7, wherein the collecting step involves collecting scene description data in the form of natural language text obtained by analyzing external image data with a vision language model (VLM) and including it in the heterogeneous offline data, and the storing and maintaining step involves periodically applying a time decay algorithm to the stored situation vector to update real-time relevance in order to reflect the temporary characteristics of offline phenomena. Claim 10 A method for generating a real-world situation vector model according to claim 7, further comprising the step of performing at least one of: a step of performing a hybrid situation search combining a similarity operation between a user's query vector and the situation vector and spatial filtering based on the generated and maintained real-world situation vector; and a step of providing an inference result based on the real-world situation vector to an external autonomous system as a real-time context information API.

Images