Spatial information integrated platform system, information processing method, and program

Abstract

This invention provides a spatial information integration platform system, information processing method, and program that enable the digital integration of physical information and its integration with business processes. [Solution] The spatial information integration platform system according to this disclosure is characterized by comprising: a spatial information integration unit that integrates data between a plurality of sensing devices, an analysis engine, and a business system using a spatial identifier; a generative artificial intelligence engine that analyzes log data and business history to propose the configuration and workflow of the sensing devices, the analysis engine, and the business system; and an execution control unit that controls the sensing devices, the analysis engine, and the business system based on the workflow.

Description

Technical Field

[0001] This disclosure relates to...

Background Art

[0002] In recent years, iPaaS (Integration Platform as a Service) technology for linking structured data between cloud applications has been known (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, it has not been able to handle the linkage with unstructured data (such as images, sounds, 3D point clouds, etc.) from sensor devices and robots in the physical space, and the dynamic system configuration optimization based on them. In industries such as construction sites, plants, and agriculture, there has been a problem that although it is required to digitally integrate physical information obtained from the site and link it with business processes, the means are limited.

[0005] Therefore, the present disclosure has been made in view of the above problems, and an object thereof is to provide a spatial information integration type platform system, an information processing method, and a program capable of digitally integrating physical information and linking it with a business process.

Means for Solving the Problems

[0006] According to the present disclosure, a spatial information integration unit that integrally links data among a plurality of sensing devices, an analysis engine, and a business system using a spatial identifier, A generative artificial intelligence engine that analyzes log data and business history to propose the configuration and workflow of the sensing device, the analysis engine, and the business system, An execution control unit that controls the sensing device, the analysis engine, and the business system based on the workflow described above, A spatial information integration platform system is provided, characterized by having the following features.

[0007] Furthermore, according to this disclosure, a process is used to integrate and link data between multiple sensing devices, analysis engines, and business systems using spatial identifiers. The process involves analyzing log data and business history to propose the configuration and workflow of the sensing device, the analysis engine, and the business system using generative artificial intelligence, A process that controls the sensing device, the analysis engine, and the business system based on the workflow described above, An information processing method for a spatial information integration platform system is provided, characterized by including the following:

[0008] Furthermore, according to this disclosure, a computer, A process that integrates and links data between multiple sensing devices, analysis engines, and business systems using spatial identifiers, The process involves analyzing log data and business history to propose the configuration and workflow of the sensing device, the analysis engine, and the business system using generative artificial intelligence, A process that controls the sensing device, the analysis engine, and the business system based on the workflow described above, A program is provided to execute it. [Effects of the Invention]

[0009] According to the present disclosure, it is possible to provide a spatial information integration type platform system, an information processing method, and a program that can digitally integrate physical information and cooperate with business processes.

Brief Description of the Drawings

[0010] [Figure 1] It is a diagram showing a configuration example of the system 1 according to an embodiment of the present disclosure. [Figure 2] It is a diagram showing a configuration example of the server device according to the same embodiment. [Figure 3] It is a diagram showing a software configuration example of the system according to the same embodiment. [Figure 4] It is an example of a flowchart of control according to the same embodiment. [Figure 5] It is another example of a flowchart of control according to the same embodiment. [Figure 6] It is another example of a flowchart of control according to the same embodiment. [Figure 7] It is another example of a flowchart of control according to the same embodiment. [Figure 8] It is another example of a flowchart of control according to the same embodiment. [Figure 9] It is another example of a flowchart of control according to the same embodiment. [Figure 10] It is another example of a flowchart of control according to the same embodiment. [Figure 11] It is another example of a flowchart of control according to the same embodiment. [Figure 12] It is another example of a flowchart of control according to the same embodiment. [Figure 13] It is another example of a flowchart of control according to the same embodiment. [Figure 14] It is another example of a flowchart of control according to the same embodiment.

Modes for Carrying Out the Invention

[0011] The preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant descriptions are omitted.

[0012] This embodiment relates to a spatial information integration type iPaaS (Integration Platform as a Service) that utilizes spatial information and generative AI to optimize the configuration of sensor devices, analysis engines, and business systems, and aims to automate workflows and continuously improve business operations. In particular, it relates to a platform that combines and optimizes sensing in the physical space and business processes in the cyber space by generative AI.

[0013] Conventional iPaaS (Integration Platform as a Service) targets the linkage of structured data between cloud applications such as Salesforce and Google Sheets, and does not support the linkage with unstructured data (videos, voices, 3D point clouds, etc.) from sensor devices and robots in the physical space, nor the dynamic system configuration optimization based on it. In industries such as construction sites, plants, and agriculture, there has been a need to digitally integrate physical information obtained from the site and link it with business processes, but there has been a problem that the means are limited. In particular, in an environment where business needs and field situations change, there has been a problem that a pre-designed static workflow cannot handle all situations.

[0014] This embodiment provides an iPaaS system and its information processing method that integratively link data generated among a plurality of sensor devices, image analysis engines, and business management tools using a spatial identifier (spatial ID) as a key, automate business processes based on them, and realize business support in terms of space. Also, by using generative AI to propose and update an optimal group of sensor devices, analysis engines, and business application configurations, and further automatically create, execute, and improve workflows, it aims to realize a labor-saving and advanced business process.

[0015] <System Configuration> Figure 1 is a block diagram showing the network configuration of a spatial information integrated platform system 100 (hereinafter also simply referred to as "the system") according to the first embodiment of the present invention. As shown in Figure 1, the system 100 is configured such that a server device 10 located in a cloud environment, a plurality of sensing devices 20 located on-site, a user terminal 40, and an external business system 30 are connected via a network NW. The system 100 may also include a controlled device 50.

[0016] The server device 10 can communicate with each device and system, for example, via an internet connection or VPN connection. It is a computer used by users and system administrators to input and output various types of information.

[0017] As shown in Figure 2, the server device 10 may include a control unit 11, a storage unit 12, an input unit 13, an output unit 14, and a communication unit 15. The control unit 11 handles the transfer of data between each unit and controls the entire device, and is realized, for example, by having a CPU (Central Processing Unit), MPU (Micro Processing Unit), or GPU (Graphics Processing Unit) execute a program stored in a predetermined memory. The storage unit 12 stores various information, including various databases described later. The storage unit 12 stores various data and programs, and may be non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), as well as magnetic disks, flexible disks, optical disks, compact disks, minidiscs, DVDs (Digital Versatile Discs), etc. The input unit 13 is for users and system administrators to input various data, and is implemented by, for example, a touch panel, buttons, keyboard, mouse, microphone, etc. The output unit 14 outputs various information generated by the control unit. The output unit is, for example, a liquid crystal display (LCD), touch panel, speaker, etc. The communication unit 15 is for communicating with other information processing devices, and has the function of a receiving unit that receives various data and signals transmitted from other information processing devices, etc., and a transmitting unit that transmits various data and signals to other information processing devices, etc., according to commands from the control unit.The communication unit is implemented by, for example, a NIC (Network Interface Card), an adapter for connecting to Ethernet (registered trademark), a modem for connecting to a public telephone network, a wireless communication device for wireless communication, a USB (Universal Serial Bus) connector or RS232C connector for serial communication, etc. The communication unit 15 can utilize wireless technologies including Bluetooth (registered trademark), Wi-Fi (registered trademark), and ZigBee, as well as microwaves, coaxial cables, fiber optic cables, twisted pair, digital subscriber lines (DSL), or infrared. When the server device 10 receives various commands (requests) via the input unit or communication unit, the control unit executes processing by program, and the processing results of the program (e.g., images, audio, etc.) are sent to the output unit or other information processing device, etc. Alternatively, the server device 10 receives various commands (requests) from other information processing devices, etc. via the communication unit, and transmits the processing results of the program executed in the control unit to the other information processing device, etc. Note that a part of the above program may be transmitted to the other information processing device and executed on the other information processing device. In this case, the other information processing device can be, for example, a smartphone, a mobile phone terminal, a tablet terminal, a personal computer, etc., and is connected to the user terminal 40 wirelessly or by wire via a network such as the Internet.

[0018] The sensing device 20 includes a camera device for acquiring image information (still images, video), a sensor device for acquiring various detection information, and a location information device, which are connected by communication means such as wired LAN, Wi-Fi, LTE, 5G, and LoRaWAN. The sensing device 20 may be fixedly installed at various sites, or it may be installed on mobile or moving objects such as robots, drones, and AGVs. The sensing device 20 may also be installed on the controlled device 50.

[0019] External business systems 30 are cloud-based business applications (such as kintone, Box, and Slack) and can communicate bidirectionally with system 100 via REST APIs, etc.

[0020] The user terminal 40 can access the system 100 via a web browser or a dedicated application to build workflows and monitor their execution status. The user terminal 40 is a computer handled by the user for inputting and outputting various types of information, and can consist of, for example, a smartphone, tablet computer, or personal computer.

[0021] The controlled devices 50 include robots, drones, AGVs, etc., which can receive control commands and operate autonomously.

[0022] The system 100 may also include an edge server. The edge server is a small server installed on-site that can perform analysis and control that requires real-time processing. The edge server can also be configured to receive data from the sensing device 20, perform analysis processing immediately, and then send the results to the server device 10 in the cloud.

[0023] The network NW consists of one or a combination of the Internet, LAN, WAN, and mobile communication network. Communication protocols such as HTTP, HTTPS, MQTT, WebSocket, and CoAP are used. MQTT, a lightweight protocol, is preferably used for data transmission from the sensing device 20, while WebSocket or UDP-based protocols are used for transmitting control commands to the controlled device 50, prioritizing real-time performance. For security, all communications are protected by encrypted communication of TLS 1.3 or higher. The server device 10 is built in a public cloud environment such as AWS, Azure, or GCP, but can also be built in an on-premises or private cloud environment. The server device 10 is equipped with a multi-core processor (Intel Xeon, AMD EPYC, etc.), GPU (NVIDIA A100, H100, etc.), RAM, NVMe SSD, and network interfaces.

[0024] Figure 3 is a functional block diagram showing the main architectural blocks of the system 100 of the present invention. As shown in Figure 2, the system 100 comprises a spatial information integration unit 110, an analysis engine group 120, a business system linkage unit 130, a generative artificial intelligence engine 140, an interface unit 150, an execution control unit 160, a logging unit 170, and a security management unit 180. Each of these units is implemented as a software module that operates on the server device 10. Data linkage between each module is performed via internal API calls, message queues (RabbitMQ, Apache Kafka, etc.), or a shared database. By adopting a microservices architecture, each module can be scaled and updated independently.

[0025] The spatial information integration unit 110 is responsible for assigning spatial identifiers (hereinafter referred to as "spatial IDs") to data collected from multiple sensing devices 20 and performing integrated processing with spatiotemporal consistency. The spatial information integration unit 110 may include, for example, a data collection module, a spatial ID assignment module, a spatiotemporal normalization module, and a data integration module. The data collection module provides an API endpoint for receiving data from the sensing devices 20 and functions as an MQTT broker, HTTP POST receiving server, WebSocket server, etc. The received data is serialized in formats such as JSON, Protocol Buffers, MessagePack, etc., and the data collection module parses these and converts them into an internal data structure. The spatial ID assignment module analyzes the location information (GPS coordinates, beacon ID, relative coordinates within the facility, etc.) contained in the received data and calculates and assigns the corresponding spatial ID. As a method for generating spatial IDs, a spatial mesh code based on latitude and longitude (compliant with ISO 19112), a hierarchical ID based on the hierarchical structure within a building, or a node ID in a graph database may be used. The spatiotemporal normalization module converts timestamps of data acquired from different sensing devices to a unified time system (UTC) and also converts the coordinate system to a unified coordinate system. The data integration module associates data with the same spatial ID or neighboring spatial ID and stores it in an integrated database. The integrated database is implemented using a time-series database (such as InfluxDB or TimescaleDB) or a NoSQL database (such as MongoDB or Cassandra), enabling high-speed searches using the spatial ID as an index key.

[0026] The analysis engine group 120 is responsible for processing sensing data linked to spatial IDs and generating advanced analysis results. The analysis engine group 120 may include, for example, an image processing engine, a three-dimensional information processing engine, an anomaly detection processing engine, and other analysis engines. The image processing engine performs processing such as object detection, segmentation, feature extraction, and OCR on image data acquired from camera devices. Deep learning models (YOLO, Mask R-CNN, EfficientNet, Vision Transformer, etc.) are executed on the GPU, and image analysis is performed in real time or in batch processing. Detected objects (people, vehicles, materials, equipment, etc.) are assigned bounding box coordinates, confidence scores, class labels, etc., and output as analysis results. The three-dimensional information processing engine generates three-dimensional point cloud data and 3D mesh models from multiple camera images, LiDAR data, depth sensor data, etc. Technologies such as Structure from Motion (SfM), Multi-View Stereo (MVS), SLAM (Simultaneous Localization and Mapping), and NeRF (Neural Radiance Fields) are used. A spatial ID is associated with the generated 3D model, enabling object placement in space, distance measurement, volume calculation, and change detection. The anomaly detection processing engine detects abnormal conditions by analyzing the time series of sensor data. Machine learning models (Autoencoder, Isolation Forest, LSTM, Transformer, etc.) are applied to data from vibration sensors, temperature sensors, pressure sensors, etc., to detect deviations from normal patterns. Detected anomaly information is assigned a spatial ID indicating the location of occurrence, an anomaly score, anomaly type (temperature anomaly, vibration anomaly, etc.), and time of occurrence. Other analysis engines include an acoustic analysis engine (anomaly sound detection, sound source localization, etc.), a text analysis engine (natural language processing, sentiment analysis, etc.), and a predictive analysis engine (time series forecasting, demand forecasting, etc.). Each analysis engine has an architecture that allows for additions via plug-ins, enabling external developers to incorporate their own analysis algorithms.The analysis results are stored in the analysis results database and can be searched using the spatial ID as the key.

[0027] The business system integration unit 130 is responsible for integrating with external business systems 30 via API and executing business processes using spatial IDs as keys. The business system integration unit 130 may include, for example, an API client module, a data mapping module, and an integration control module. The API client module sends HTTP requests to the REST API or GraphQL API of each business system 30 and receives responses. It supports authentication methods such as OAuth 2.0, API keys, and JWT to ensure secure communication. The data mapping module performs conversion between the internal data format of system 100 and the data format of business systems 30. For example, it converts anomaly detection results linked to spatial IDs into a record format for a business application (such as kintone) and maps them to predetermined fields. The integration control module controls business integration flows such as data storage processing, approval processing, and notification processing. In data storage processing, analysis results and integrated data are saved to cloud storage (such as Box, Google Drive, or Amazon S3) with the spatial ID included as the file name or metadata. In the approval process, when events such as anomaly detection or work completion occur, approval request records are automatically created in business applications (kintone, ServiceNow, etc.). The request record includes the space ID, time of occurrence, details of the anomaly, attached images, etc. In the notification process, when an anomaly is detected in a specific space ID, notifications are sent to relevant parties via email, Slack, Microsoft Teams, LINE, etc. The notification message includes location information corresponding to the space ID (building name, floor, area, etc.), allowing recipients to immediately understand the situation. The Webhook function also enables event notifications from business system 30 to system 100, realizing bidirectional communication.

[0028] The generative artificial intelligence engine 140 (generative AI engine) is a core component of the present invention and is responsible for the function of optimizing the entire system. The generative artificial intelligence engine 140 may include, for example, a data analysis module, a configuration proposal module, a workflow generation module, and a learning and improvement module. The data analysis module analyzes operation logs, sensing data logs, and business history accumulated in the log recording unit 170. Specifically, it collects and analyzes the operating status of sensing devices (data acquisition frequency, error rate, battery status, etc.), processing time, accuracy, resource usage of the analysis engine, data transmission frequency to the business system, processing completion rate, user workflow editing history, execution results, event occurrence frequency by spatial ID, response time, etc. SQL, aggregate functions, and statistical processing libraries (pandas, NumPy, etc.) are used for analysis to extract significant trends and patterns from a large amount of log data. The configuration proposal module proposes the optimal configuration of sensing devices, analysis engines, and business systems based on the data analysis results. APIs of large-scale language models (GPT-4, Claude, Gemini, etc.) are utilized for proposal generation. The data analysis module takes the extracted information as prompt context and instructs the LLM to "propose the optimal device configuration and analysis engine combination for these environmental conditions and business operations." The LLM refers to past success stories and generates a natural language proposal that includes the content of recommended configuration changes, expected effects, implementation difficulty, estimated cost, etc. The proposal includes supporting past success rates and reference links to similar cases, allowing the user to judge the validity of the proposal. The workflow generation module automatically generates a specific workflow based on the configuration proposal. The workflow consists of trigger conditions (spatial ID, time, sensor value threshold, etc.), processing nodes (data acquisition, analysis execution, conditional branching, data transformation, etc.), action nodes (notification sending, data storage, device control, approval request, etc.), and connection relationships (data flow and control flow between each node). The workflow is output as structured data in JSON or YAML format and visualized in the interface unit 150.The learning and improvement module receives workflow execution results as feedback and performs continuous improvement. It automatically executes the PDCA cycle (Plan: current situation analysis and proposal of improvement plans, Do: execution of the new workflow, Check: evaluation of execution results, Act: readjustment of configuration based on evaluation results). Evaluation metrics include the reduction rate of work completion time, recall and precision of anomaly detection, user satisfaction (feedback), and cost reduction amount. It is also possible to apply reinforcement learning methods and define a reward function to optimize the workflow configuration.

[0029] The interface unit 150 provides functions for users to visually build and edit workflows. The interface unit 150 may include, for example, a map display module, a workflow editing module, a preview function module, and a dashboard display module. The map display module provides a map interface that operates on a web browser. It uses a map library (Mapbox, Leaflet, Google Maps API, OpenLayers, etc.) to visualize spatial IDs on a map of the target area. Each spatial ID is displayed in the form of a grid, polygon, marker, heatmap, etc. Users can click on a spatial ID on the map to view detailed information about the area (sensor placement, past event history, current status, etc.). The workflow editing module provides the function to build workflows by placing processing nodes using drag-and-drop operations and connecting them with connection lines. The editing screen is implemented using libraries such as React Flow, jsPlumb, and Blockly to achieve an intuitive UI / UX. Each node allows for the selection of node type (data acquisition, analysis, notification, etc.), parameter settings (threshold, time interval, target device, etc.), and conditional branching settings (IF-THEN-ELSE structure). The workflow proposed by the generative artificial intelligence engine 140 is automatically displayed on the editing screen, and the user can modify it as needed. The preview function module provides the ability to simulate the constructed workflow and preview the results. The replay function using past data allows for verification of the workflow's operation. During simulation execution, the processing time of each node, data flow, and the results of conditional branching are displayed as animations, allowing the user to visually understand the workflow's behavior. The dashboard display module provides the ability to visualize the overall system's operating status in real time. Processing execution status for each spatial ID, error occurrence status, sensor operating status, and business process completion rate are displayed using graphs, tables, maps, etc. Integration with dashboard tools such as Grafana, Kibana, and Tableau is also possible.

[0030] The execution control unit 160 is responsible for controlling the entire system based on the constructed workflow. The execution control unit 160 may include, for example, a workflow execution engine, a device control module, and a cooperative control module. The workflow execution engine reads the workflow definition (JSON or YAML), monitors trigger conditions, executes processing nodes sequentially, and controls conditional branching. The execution engine is implemented based on a workflow management system such as Apache Airflow, Prefect, Temporal, or Argo Workflows. Each processing node is executed as a Python script, Docker container, Kubernetes job, etc., enabling distributed and parallel processing. An event-driven architecture is used for monitoring trigger conditions, and the workflow is started in response to changes in sensor data or the arrival of a specific time. The device control module is responsible for sending control commands to movable control target devices 50 (robots, drones, AGVs, etc.). The control commands include the target spatial ID (destination location), operation content (photography, measurement, transport, etc.), priority, and execution deadline. Control commands are transmitted using protocols such as REST API, MQTT, ROS (Robot Operating System), and DDS (Data Distribution Service). The device control module monitors the current location, battery level, and task execution status of the controlled device in real time, and takes action such as reassigning tasks to alternative devices or performing emergency shutdowns in the event of an anomaly. The cooperative control module is responsible for cooperative control functions such as collision avoidance, task sharing, and sequence control when multiple devices are working in the same spatial ID or nearby spatial ID. For example, when multiple drones inspect the same area, they optimize their flight paths to avoid overlaps and share their mutual position information to avoid collisions. Algorithms such as multi-agent reinforcement learning, distributed constraint optimization, and auction-based task assignment are used for cooperative control.

[0031] The logging unit 170 is responsible for recording the operation history of the entire system. The logging unit 170 may include, for example, a processing log recording module, a sensing data log recording module, a user operation log recording module, and an execution status log recording module. The processing log recording module records the start time, end time, processing time, success or failure status, error messages, etc., of each processing node. The sensing data log recording module records the reception time, data size, spatial ID, and data quality (missing data rate, outlier rate, etc.) of data acquired from each sensing device. The user operation log recording module records the history of user workflow editing operations, parameter changes, manual executions, system setting changes, etc. The execution status log recording module records the processing execution status (waiting, running, completed, error, etc.) for each spatial ID and visualizes it in real time. Log data is stored in time-series databases (such as InfluxDB and Prometheus), distributed log management systems (such as Elasticsearch + Logstash + Kibana and Splunk), or data lakes (such as Amazon S3 + Athena and Google BigQuery). Log data is tagged with spatial IDs, timestamps, event types, and detailed information, enabling fast searching, aggregation, and analysis. The retention period for log data is set according to legal regulations and business requirements, and older log data is automatically archived or deleted.

[0032] The Security Management Unit 180 is responsible for ensuring the security of the entire system. The Security Management Unit 180 may include, for example, an authentication / authorization module, a communication encryption module, a data encryption module, and an audit log module. The authentication / authorization module performs user authentication (ID / password, multi-factor authentication, SSO, SAML, OpenID Connect, etc.) and authorization (role-based access control RBAC, attribute-based access control ABAC, etc.). It also includes a function to match user location information with spatial IDs and dynamically control access rights to specific areas. For example, access to spatial IDs corresponding to restricted areas within a factory is restricted to authorized users only. Location information is obtained via GPS, BLE beacons, Wi-Fi RTT, QR code check-in, etc. The communication encryption module uses encryption protocols such as TLS 1.3 in API communication with external business systems and communication with sensing devices to prevent data eavesdropping and tampering. Let's Encrypt or private certificate authorities are used for certificate management, and the automatic certificate renewal function reduces the operational burden. The data encryption module encrypts (using AES-256, etc.) sensing data, analysis results, and log data stored in the database to protect them from unauthorized access. Key management services such as HSM (Hardware Security Module), AWS KMS, and Azure Key Vault are used for managing encryption keys. The audit log module records all access history and data manipulation history as audit logs, enabling tracking in the event of a security incident. The audit logs record user ID, operation details, target data, timestamp, source IP address, etc., and are protected by digital signatures or blockchain technology to prevent tampering.

[0033] <Processing Flow> Next, a typical processing flow in the system 100 of the present invention will be described with reference to Figure 4. Figure 4 is a flowchart showing a series of processing flows from anomaly detection to completion of response. First, the sensing device 20 acquires data (step S1). Specifically, the camera device takes images of the site, the sensor device measures physical quantities such as temperature, vibration, and pressure, and the location information device acquires GPS coordinates or beacon information. Subsequently, the spatial information integration unit 110 assigns a spatial ID to the received data (step S2). It analyzes the location information contained in the data, calculates and assigns the corresponding spatial ID, unifies the timestamp to UTC, and converts the coordinate system to a common coordinate system. Next, the spatial information integration unit 110 integrates data with the same spatial ID or nearby spatial IDs (step S3). Multiple sensor data acquired at the same time are associated and stored in the integrated database. Subsequently, the analysis engine group 120 analyzes the integrated data (step S4). The image processing engine detects objects in the image, the anomaly detection processing engine calculates the degree of anomaly in the sensor data, and the three-dimensional information processing engine generates a 3D model, among other processes.

[0034] Next, the execution control unit 160 performs a condition determination based on the analysis results (step S5). It determines whether the conditions defined in the workflow (e.g., abnormality level exceeds a threshold of 80%, a specific class of object is detected, etc.) are met. If the conditions are not met (No in step S5), the process ends and waits for the next data acquisition. If the conditions are met (Yes in step S5), the business system integration unit 130 sends a notification (step S6). A notification message including the spatial ID where the abnormality occurred, the abnormality details, the time of occurrence, attached images, etc., is sent to the relevant parties via Slack, email, LINE, etc. Subsequently, the execution control unit 160 sends a control command to the controlled device 50 (step S7). For example, it sends a command to fly a drone to the spatial ID where the abnormality was detected and to take detailed images. The drone moves to the specified spatial ID, performs the shooting, and sends the image data to the system 100. Next, the business system integration unit 130 creates a record in the business application (step S8). As an abnormality response task, a new record is created in a business application such as kintone, and the person in charge, deadline, priority, etc. are set. Next, the generative artificial intelligence engine 140 collects and analyzes log data (step S9). It collects a series of log data from the anomaly detection to the completion of the response and analyzes the processing time, success rate, whether or not there was user intervention, etc. Finally, the generative artificial intelligence engine 140 updates the configuration proposal (step S10). The results from this time are accumulated as training data, and configuration changes (addition of sensors, threshold adjustment, change of analysis algorithm, etc.) to improve the accuracy of anomaly detection in the future are reflected in the proposal. Through this processing flow, the entire cycle from anomaly detection to response and learning is automated. Note that in this system, only a part of the above processing flow may be executed.

[0035] <Details of spatiotemporal normalization processing> The spatiotemporal normalization process in the spatial information integration unit 110 will be explained in more detail. The spatiotemporal normalization process consists of four steps: time synchronization, coordinate system unification, spatial ID assignment, and event labeling. In the time synchronization process, the timestamps of data acquired by each sensing device are unified to a common time standard. Specifically, the clock deviations of each device are corrected based on the standard time obtained from a GPS time or NTP server. For example, if the timestamp of the camera device is 12:00:00 milliseconds and the timestamp of the sensor device is 12:00:00:089 milliseconds, the process may be performed to unify both to 12:00:00:100 milliseconds based on the time of the NTP server. This corrects the variation in data obtained from multiple devices and allows them to be accurately treated as events of the same time. The accuracy of the time correction is achieved on the order of a few milliseconds in the case of NTP and tens of nanoseconds in the case of GPS time. In the coordinate system unification process, data acquired by each sensing device in different coordinate systems is converted to a common coordinate system. For example, camera devices provide location information in pixel coordinates, LiDAR devices in polar coordinates, and GPS devices in latitude and longitude, but these are converted to a unified ENU coordinate system (East-North-Up) or an XYZ Cartesian coordinate system. A transformation matrix using the installation position and orientation information (calibration parameters) of each device is applied to the coordinate transformation. For example, in the process of converting pixel coordinates (u, v) on a camera image to 3D spatial coordinates (X, Y, Z), a projection transformation is performed using the camera's intrinsic parameters (focal length, principal point position, lens distortion coefficient) and extrinsic parameters (position, orientation).

[0036] In the spatial ID assignment process, a spatial ID based on location information is assigned to normalized data. The method for generating spatial IDs is selected according to the application, and methods such as spatial mesh codes based on latitude and longitude, hierarchical IDs based on the building's hierarchical structure, or node IDs in a graph database are used. In the case of spatial mesh codes based on latitude and longitude, Japanese regional mesh codes (1st to 4th level meshes) or Google S2 geometry cell IDs are used. For example, if a 10-meter mesh code is generated for a location at latitude 35.6812 degrees and longitude 139.7671 degrees (near Tokyo Station), a numerical ID such as 53393599 will be assigned. In the case of hierarchical IDs based on the building's hierarchical structure, an ID such as "BuildingA-3F-Room301" is generated based on the hierarchical structure of building name-floor name-area name. In the event labeling process, data with the same spatial ID and close time intervals (e.g., within ±1 second) are grouped together as the same event. This allows different types of sensing data, such as camera images, vibration data, and temperature data, to be associated as events that occurred in the same location at the same time. For example, in spatial ID "G03-G05," a common event ID "EVT-50109-120000-G03G05" is assigned to camera images, vibration data, and noise data acquired around 12:00:00. This event ID enables a complex situational assessment by combining data from multiple sensors in subsequent analysis processing.

[0037] <Details of the generative artificial intelligence engine> This section describes the details of the configuration proposals in the generative artificial intelligence engine 140. The configuration proposal module analyzes past operational performance based on multiple evaluation metrics and proposes the optimal configuration. There are five types of evaluation metrics: accuracy metrics, processing time metrics, cost efficiency metrics, field suitability metrics, and user evaluation metrics. For accuracy metrics, detection rate (Recall, the percentage of actual anomalies that the system was able to detect), precision rate (Precision, the percentage of things that the system judged to be anomalies that were actually anomalies), and F-score (harmonic mean of detection rate and precision rate) are used. For example, if the precision rate of anomaly detection by camera image analysis is low at 60% in a certain spatial ID, and it has been shown that adding a LiDAR sensor improves it to 90%, then adding a LiDAR sensor will be proposed in similar environments. For processing time metrics, average processing time (seconds from data acquisition to analysis completion), delay rate (percentage of times the target processing time was exceeded), and latency distribution (variation in processing time) are used. For applications requiring real-time performance (e.g., safety monitoring), a low-latency configuration is prioritized. Cost efficiency indicators include the number of results per sensor (number of anomaly detections, number of completed tasks, etc.), the effectiveness-to-cost ratio (degree of improvement in operational efficiency relative to the implementation cost), and ROI (return on investment). For example, if the same effect can be obtained with an expensive sensor A and an inexpensive sensor B, sensor B will be recommended first. Field suitability indicators include uptime (percentage of downtime due to communication failures, battery depletion, malfunctions, etc.), weather resistance (operational stability in environments such as rain and strong winds), and maintenance frequency. In outdoor environments, proposals will be made according to environmental conditions, such as prioritizing devices with high waterproof performance. User evaluation indicators include the number of manual corrections (number of times the user corrected the workflow proposed by the generating AI), intervention frequency (number of times the automated process failed and a human intervened manually), and reconfiguration frequency (number of times a different configuration was selected instead of the proposed configuration). Configurations with fewer interventions are evaluated as being easier for users to use.

[0038] The configuration proposal processing flow consists of four steps: history collection, feature extraction, classification and learning, and generation of the proposed configuration. In the history collection step, the log recording unit 170 collects processing configurations (devices used, analysis engine, workflow), execution results (success or failure, processing time, accuracy metrics), environmental conditions (indoor or outdoor, weather, time of day), and business categories (inspection, monitoring, surveying, etc.) for each spatial ID. For example, data such as "Spatial ID: SITE-A-G05, Application: Anomaly detection, Environment: Outdoor, sunny, Configuration A: Camera + image analysis, Success rate: 65%, Configuration B: Camera + LiDAR + 3D analysis, Success rate: 92%" is collected. In the feature extraction step, environmental conditions and processing conditions are converted into a format that can be used as input to a machine learning model. Indoor or outdoor is a binary variable (0 or 1), weather is a categorical variable (sunny, cloudy, rainy), time of day is a categorical variable (morning, noon, evening, night), and application is a categorical variable (inspection, monitoring, surveying). In the classification and learning step, a machine learning model (such as Random Forest, Gradient Boosting, or Neural Network) or generative AI is used to learn the conditions for a successful configuration. When using generative AI, past operational data can be input as prompt context, and the LLM may be queried in the format of, for example, "Past operational data: Spatial ID: SITE-A-G05, Application: Anomaly detection, Environment: Outdoor, sunny, Configuration A: Camera + image analysis, Success rate: 65%, Configuration B: Camera + LiDAR + 3D analysis, Success rate: 92%. New case: Spatial ID: SITE-B-G03, Application: Anomaly detection, Environment: Outdoor, sunny. Please propose the optimal configuration." The LLM will refer to successful cases under similar conditions and propose a specific configuration along with natural language reasons for the recommendation, such as, "In this environment, 3D analysis using LiDAR is effective. This is because in outdoor environments, the accuracy of image analysis tends to decrease due to the effects of shadows, and combining it with distance information from LiDAR improves accuracy." In the proposed configuration generation step, the proposed result generates structured data or natural language text that includes recommended device configuration (sensor type, placement), recommended analysis engine (image processing, 3D processing, etc.), recommended business system integration (notification destination, storage destination), reasons for the proposal (past success rate, reference to similar cases), and expected effects (improved accuracy, reduced processing time, etc.).This information is visually displayed on the GUI of the interface unit 150, allowing users to review the content and decide whether to adopt or modify it.

[0039] This section describes the retraining process in the learning and improvement module. The retraining process improves subsequent configuration proposals based on log information including the success or failure of each component's execution, processing time, user editing history, sensing data quality indicators, and business performance indicators. The workflow execution log collects information such as the success or failure of the process (success, failure, timeout), execution time (processing time for each node), network status (communication delay, packet loss rate), and environmental conditions (temperature, humidity, weather, etc.). The sensing data quality log collects information such as the missing data rate (percentage of data that could not be acquired), the anomaly rate (frequency of clearly unnatural values), and the analysis error history (image analysis failure, 3D reconstruction failure, etc.). The user operation history collects information such as manual modifications to the proposed configuration, the number of workflow edits, and the decision history of whether the proposal was accepted or rejected. Business performance indicators collect information such as KPI achievement status (target number of processed items, number of anomaly detections, etc.), processing efficiency (rate of reduction in work time, cost reduction rate), and trouble rate (frequency of accidents, failures, complaints, etc.). Based on this information, the generative artificial intelligence engine 140 implements improvements. For example, in the case of accuracy improvement, it learns that "in spatial IDs G03-G05, the anomaly detection rate was 60% with the camera alone, but improved to 85% when a vibration sensor was used in combination," and will suggest using a vibration sensor in similar environments in the future. In the case of cost optimization, it learns that "in spatial IDs G07-G08, an expensive LiDAR sensor was used, but the same accuracy could be obtained with an inexpensive stereo camera," and will prioritize suggesting a stereo camera for applications where cost efficiency is important. In the case of learning user preferences, it learns the operation history that "User A always changes the notification destination from Slack to email in response to suggestions from the generative AI," and will suggest email notifications to User A from the beginning.

[0040] <Details of the interface section> This section describes the processing conditions and actions in the interface unit 150. Processing conditions define the situations and judgment rules that trigger the workflow, and include four types: spatial conditions, sensing data conditions, time conditions, and combined conditions. Spatial conditions can be set to detect intrusion into a specific area (spatial ID), exceeding the time spent in a specific area, or detecting exit from a specific area. For example, by setting the condition "when a person enters spatial ID G05-G05", unauthorized entry into a restricted area can be detected. Sensing data conditions can be set to detect temperature anomalies (exceeding or falling below a threshold), crack detection by image analysis, noise level exceeding limits, and vibration anomaly detection. For example, by setting conditions "when the temperature exceeds 35℃" or "when a crack is detected by image analysis", equipment abnormalities can be automatically detected. Time conditions can be set to execute at set times (e.g., 5pm every day, 9am every Monday), execute after a certain period of time has elapsed (processing within 30 minutes of data acquisition), or execute within a specific time range (only during business hours). For example, by setting conditions such as "every day at 5 PM" and "within 30 minutes of data acquisition," regular inspections and rapid responses can be automated. Combined conditions allow for the definition of more detailed conditions by combining sensing data conditions, spatial conditions, and time conditions. For example, a combined condition such as "if the temperature exceeds 30°C in spatial ID G03-G03 between 9 AM and 5 PM on weekdays" allows for flexible settings that detect only abnormalities during business hours.

[0041] Actions define operations and processes that the system automatically executes when certain conditions are met, and include four types: notification actions, device control actions, data integration actions, and approval / branching actions. Notification actions can be configured to send notifications to Slack, LINE, email, SMS, etc., activate warning lights or alarm buzzers, and display alerts on the dashboard. For example, by setting the action to "Send a notification to the #alerts channel in Slack," information will be immediately communicated to team members when an anomaly occurs. Device control actions can be configured to start or stop robots or drones, send movement commands (to move to a specific spatial ID), and perform actions such as taking photos or measurements. For example, by setting the action to "Fly the drone to spatial ID G05-G05 and perform photography," detailed confirmation of the area where an anomaly occurred can be automated. Data integration actions can be configured to save data to cloud storage (Box, Google Drive, etc.), register records in business applications (kintone, Freee, etc.), and record data in a database. For example, by setting the action to "Save captured images to the inspection record folder in Box," audit trail management can be automated. The approval and branching processing actions can be configured to automatically trigger business approval workflows, branch processing based on conditions (IF-THEN-ELSE), and request human confirmation. For example, by setting an action such as "If the abnormality level is 80% or higher, trigger an approval request; if it is less than 80%, only record the information," the appropriate response based on the severity level will be automatically selected.

[0042] This section explains specific examples of workflow construction on the map UI. When a user accesses interface unit 150 via a web browser, a map of the target area is displayed. On the map, each spatial ID is visualized as a grid or polygon. For example, in the case of a construction site, a 100m x 100m area is divided into 10m x 10m grids, and each grid is assigned a spatial ID from "G01-G01" to "G10-G10". When a user clicks on the target area on the map (for example, building A, 3rd floor), the corresponding spatial ID (for example, A-3F-AREA01) is highlighted. The tool palette on the left side of the screen displays processing nodes such as "Data Acquisition," "Image Analysis," "Temperature Monitoring," "Notification," and "Drone Control" as icons. The user drags the "Temperature Monitoring" node and drops it onto the selected spatial ID "A-3F-AREA01." Clicking on a placed node displays a parameter setting screen where you can configure the threshold (30°C), acquisition frequency (10-second intervals), and anomaly detection conditions (threshold exceeded 3 times consecutively). Next, place a "Notification" node and set the notification destination (Slack's #alerts channel) and message template ("A temperature anomaly has occurred in space ID {space ID}. Current temperature: {temperature}°C"). Connecting the "Temperature Monitoring" node and the "Notification" node with an arrow (connection line) defines a data flow where a notification is sent when a temperature anomaly is detected. Furthermore, add a "Drone Control" node, set the target space ID (A-3F-AREA01) and operation details (photography with an infrared camera), and connect a connection line from the "Notification" node to the "Drone Control" node to complete a workflow where a drone automatically checks the site after a notification is sent.

[0043] Users can simulate their built workflow by clicking the "Preview" button. A replay function using data from the past week allows users to verify the system's behavior when an anomaly actually occurs. During simulation, the temperature of each spatial ID is displayed on the map in different colors (blue: normal, yellow: caution, red: anomaly). When an anomaly is detected, a notification node flashes, and an animation of a drone icon moving to the affected area is displayed. The processing time for each node, data flow, and the results of conditional branching are displayed in the log panel on the right side of the screen, allowing users to visually understand the workflow's behavior. If there are no problems, the workflow is saved by clicking the "Save" button, and real-time operation begins by clicking the "Start Execution" button. Workflows proposed by the generative artificial intelligence engine 140 are automatically displayed on the editing screen. The proposals include the reasons for the recommendation (e.g., "Because temperature anomalies have frequently occurred at this site in the past, we recommend monitoring at 10-second intervals"), allowing users to understand and decide whether to adopt or modify them. If a user modifies a suggestion (for example, changing the threshold from 30°C to 35°C), the modification is recorded in the learning and improvement module and used to improve the accuracy of future suggestions.

[0044] This document describes a specific application example of access control based on spatial ID in Security Management Department 180. Access control consists of four elements: user ID authentication, spatial ID, user location information, and authentication condition match determination. User ID authentication is basic authentication during login (ID and password, token, biometric authentication, etc.). Spatial ID is an identifier for a physical space unit to which a workflow or target of operation is associated (e.g., Building A-2F-Distribution Panel Room). User location information is the real-time current location obtained by GPS, BLE beacon, Wi-Fi RTT (Round Trip Time), QR code check-in, etc. Authentication condition match determination is a process that uses the actual presence of the user within the space belonging to the spatial ID as one of the authentication conditions. As an application example, consider an inspection approval operation at a construction site. The spatial ID is Building A-3F-ELEC, and the user's location is detected as being in Building A-3F from the smartphone's BLE and Wi-Fi. The authentication process requires users to first log in with their ID and password, and only if they are physically present at the site will the inspection approval button be activated. This mechanism eliminates the risk of accidental approval by someone not physically present. As an example of restricting operation in hazardous areas of plant equipment, the spatial ID is determined to be the tank room area (ID: PLANT-TK-02), and the user's location is determined to be outside the area based on the location beacon tag. As a result, robot control and manual operation buttons in the relevant area are hidden or disabled. This prevents accidental operation by someone who is not physically nearby.

[0045] As an example of drone deployment control on a farm, the spatial ID is determined to be the field area "Field-B-02," and the user's location is determined to be within the field boundary based on the tablet's GPS. As a result of authentication, drone deployment can be ordered. As a supplement, it is also possible to achieve three-factor authentication (ID authentication, location information, user type) by adding the user type (administrator) of the work supervisor as a condition. The method of acquiring location information can be selected depending on the environment. In outdoor environments, GPS is mainly used, with an accuracy of several meters to over ten meters. In indoor environments, BLE beacons are used, estimating the distance from the beacon's signal strength (RSSI), and determining the location by combining information from multiple beacons. The accuracy is about several meters. Wi-Fi RTT is a technology that measures distance from the round-trip radio time to the access point, and can achieve an accuracy of several tens of centimeters indoors. QR code check-in is a method of proving presence at a location by scanning a QR code installed on site with a smartphone, and is highly reliable but requires manual operation. To increase the reliability of location information, it is also possible to use multiple methods in combination. For example, a system is employed that obtains location information from both GPS and BLE beacons, and only allows authentication if both match. By recording the history of location information, it is also possible to track the user's movement and issue a warning if suspicious activity is detected. Access control policies are used in combination with role-based access control (RBAC). For example, access permissions can be set according to the user's role, such as general workers only being able to access the spatial IDs they are responsible for, site supervisors being able to access all spatial IDs, and external contractors being able to temporarily access only specific spatial IDs.

[0046] <Extension> The plugin mechanism in Interface Unit 150 will be explained in detail. The plugin mechanism is an extension architecture that allows external developers to add their own processing blocks, analysis engines, data linkage nodes, etc. Plugins are implemented in JavaScript or TypeScript and must conform to the plugin API provided by System 100. The plugin API includes interfaces for node registration, parameter definition, execution process implementation, and error handling. As an example of adding a processing block, consider the case of developing a database writing block for an internal system. The external developer defines database connection information (hostname, port, database name, table name, etc.) as parameters and implements the process of converting input data into an SQL query and writing it to the database. The implemented plugin is packaged as a ZIP file or npm package and uploaded from the plugin management screen of System 100. The uploaded plugin is automatically verified in a sandbox environment and added to the tool palette after it is confirmed that there are no security risks. As an example of adding an AI engine, consider the case of plugging in a custom image recognition model. The external developer includes a trained model (ONNX or TensorFlow format) and inference code in the plugin. The inference process is executed in a GPU environment and performs operations such as object detection, segmentation, and classification on the input image. The inference results are output in a standard format (JSON) and passed to subsequent processing nodes.

[0047] As an example of adding a data integration node, consider developing an integration node with a local government's facility management system. The external developer implements authentication processing, data transmission processing, error handling, etc., based on the specifications of the local government system's REST API. The integration node can have parameters such as facility ID, inspection results, and attached images set, and this information is sent to the local government system. As an example of customizing the UI or node attributes, parameter names can be set using industry-specific terminology. For example, in the construction industry, specialized terms such as "rebar cover thickness" and "slump value" can be used as parameter names to provide an interface that is easy for users to understand. Color coding can also be used to visually distinguish safety-related nodes in red and quality control-related nodes in blue. As an example of templating, the developed workflow configuration can be saved as a template and distributed to other users through the marketplace. For example, templates such as "Safety Monitoring Template for Construction Sites," "Predictive Maintenance Template for Plants," and "Cultivation Management Template for Agriculture" can be published along with ratings and reviews. Users can download the template from the marketplace and immediately start operation by simply adjusting the parameters to suit their own site. Plugin developers can receive rewards based on the number of downloads and ratings, promoting the activation of the ecosystem. The plugin's version control feature notifies users when a new version is released, allowing for one-click updates. If there are incompatible changes, a migration guide is provided to minimize the impact on existing workflows.

[0048] <Flexible definition of spatial ID> This section provides a detailed explanation of the definition units for spatial IDs, including specific examples for each industry. In the construction industry, spatial IDs are defined by building / structure units (Building A, Building B, work area, etc.), floor or level units (1F, 2F, rooftop, etc.), room / section units (conference room, workroom, warehouse, etc.), construction step units (foundation, structure, interior, finishing, etc.), and work zone units (protection area, hazardous area, restricted area, etc.). For example, spatial IDs are expressed hierarchically, such as "SITE-Building AB-3F-Workroom 301-Foundation work". This hierarchical structure allows for flexible analysis with varying scopes, from managing the progress of the entire building to managing the construction status of individual rooms. In the plant industry, spatial IDs are defined by process line units (reactions, distillation, cooling, etc. in the manufacturing process), equipment units (piping, valves, tanks, heat exchangers, etc.), control unit units (PLC or DCS control area), and building or floor units (operation building, equipment building, control room, etc.). For example, the spatial ID is expressed as "PLANT-B-Manufacturing Building-Reaction Line-Tank TK-05". This definition allows for tracking of inspection and anomaly history for specific equipment, improving the accuracy of predictive maintenance. In agriculture, spatial IDs are defined by field blocks (management areas for each crop, e.g., rice area A-1), cultivation bed or shelf numbers (hydroponics, plant factories, etc.), greenhouse numbers (per greenhouse, e.g., GH-03), and work zones (sowing area, fertilization area, harvesting area, etc.). For example, the spatial ID is expressed as "FARM-C-Field A-House GH-03-Shelf Number 12". This definition allows for the management of growth status and environmental data for each individual cultivation bed, enabling the search for optimal cultivation conditions.

[0049] In the railway industry, spatial IDs are defined by station / platform units (station ID and platform number, e.g., TKY-05-H2), track sections (section ID, e.g., Shinjuku to Nakano), vehicle / train sets (vehicle number, e.g., E235-501), and inspection zones (inspection pits, maintenance yards). For example, the spatial ID might be expressed as "RAIL-Tokyo Station-Track 5-Platform 2". This definition allows for tracking of each vehicle's operation and inspection history, enabling vehicle condition management and optimization of maintenance plans. In the logistics industry, spatial IDs are defined by shelf numbers or picking areas (shelf ID, e.g., A3-12-05), shipping lanes or berth numbers (truck entrances / exits, e.g., BAY-07), area zones (receiving, inspection, sorting, storage, shipping, etc.), and drone flight areas (warehouse flight zones, AGV driving zones). For example, the spatial ID might be expressed as "WH-Warehouse A-Storage Area-Shelf A3-12-05". This definition enables precise inventory location management, efficient picking operations, and automated control of AGVs and drones. In inspection operations (infrastructure, civil engineering, building management, etc.), spatial IDs are defined by equipment labels (e.g., Elevator No. 1, Distribution Panel A-3), building zones (e.g., multi-story parking garage, water pump room), inspection route sections (patrol order, e.g., rooftop → air conditioning room → elevator room), and anomaly detection areas (specific areas with a history of past anomalies). For example, the spatial ID might be expressed as "BLDG-Head Office Building-3F-Elevator No. 1". This definition streamlines patrol inspections of building and infrastructure equipment, preventing missed inspections and enabling early detection of anomalies. The hierarchical structure of spatial IDs can be dynamically changed depending on the application. For example, in the initial stages of a construction project, management can be done at a coarse granularity (building unit), and as construction progresses, it can be subdivided into finer granularity (room unit, construction step unit). Conversely, after project completion, it can be integrated into a coarser granularity for maintenance management.

[0050] As a second embodiment, an example of application to automated patrol inspections in a power plant (an example of a plant) will be described. A power plant has numerous facilities such as turbine buildings, boiler buildings, substations, and cooling towers, and each facility area can be assigned a spatial identifier (for example, in the format of spatial ID PLANT-A-TURBINE-01 to COOLING-TOWER-05). As sensing devices, surveillance cameras can be placed around main facilities and passageways to constantly monitor the appearance of the facilities and the movement of workers. Surveillance cameras can be used in combination with visible light cameras, enabling continuous monitoring day and night. Thermal imaging cameras can be placed around each main facility to constantly monitor the temperature distribution of the facility surface. Vibration sensors can be attached to rotating equipment such as turbines, generators, and pumps to detect signs of bearing deterioration or unbalance. Acoustic sensors can be placed around main piping and equipment to detect the occurrence of abnormal noises. Gas sensors can be placed around boilers and substations to detect leaks of flammable or toxic gases. Self-propelled inspection robots can be deployed to conduct patrol inspections of high-temperature areas and high places that are difficult for humans to access. These inspection robots are equipped with visible light cameras, infrared cameras, acoustic sensors, and gas sensors, enabling multifaceted data collection. Drones can also be deployed to inspect outdoor equipment (cooling towers, chimneys, power line connections, etc.). Data acquired from these devices can be transmitted to edge servers via industrial Ethernet or wireless networks within the power plant for real-time analysis. The edge servers are installed in the turbine and boiler buildings, enabling low-latency anomaly detection.

[0051] This document describes a specific example of a continuous monitoring workflow using surveillance cameras in a power plant. As a trigger, continuous monitoring can be set 24 hours a day, 365 days a year. The processing flow can be as follows: First, surveillance cameras continuously capture images of various areas within the power plant. Visible light cameras can monitor the external condition of equipment, the work status of workers, and whether or not intrusions into restricted areas have occurred. Infrared cameras can monitor the condition of people and equipment even at night or in areas with insufficient lighting. An image processing engine analyzes the captured images in real time. Object detection algorithms (e.g., YOLO series or Transformer-based models) can be used to detect workers, equipment, vehicles, foreign objects, etc. The system can automatically determine whether workers are wearing personal protective equipment (helmets, safety shoes, safety glasses, work clothes) and issue a warning if they are not. If intrusion into a restricted area is detected, an alert can be issued immediately. Specifically, if a person is detected in a restricted area defined as a spatial identifier such as "RESTRICTED-AREA-01," the spatial identifier and the person's tracking ID can be recorded, and a warning can be issued. Abnormal conditions in equipment (such as liquid leaks, smoke generation, equipment deformation, and corrosion progression) can be detected using image recognition. A deep learning model can be used to classify normal and abnormal conditions, and continuous learning can be implemented to improve classification accuracy.

[0052] In this example, the movement of workers may be tracked. Multiple camera images can be integrated, and a unique tracking ID (e.g., in the format WORKER-0001) can be assigned to each worker. A tracking algorithm (e.g., DeepSORT or ByteTrack) can be used to track the same person across cameras. Worker locations can be converted into spatial identifiers, allowing for real-time tracking of which workers are in which areas. For example, the movement of WORKER-0001 can be recorded, showing that they moved to the TURBINE-01 area at a specific time (e.g., 10:05) and then to the BOILER-02 area at another time (e.g., 10:20). If a worker's stay time is unusually long (e.g., a task that normally takes about 10 minutes takes more than 30 minutes), it can be determined that there may be a delay in work or some kind of problem, and the supervisor can be notified. Workers performing hazardous tasks (e.g., working at heights, working in high-temperature areas, working with electrical equipment, etc.) can be identified and monitored intensively. If an abnormal or dangerous situation is detected, immediate action can be taken. If intrusion into a restricted area is detected, a warning can be sent to the worker's wearable device, and the site supervisor can be notified via Slack or email. If personal protective equipment is not being worn, a warning can be sent to the worker to prompt corrective action. If a liquid leak or smoke is detected from equipment, an emergency alert can be issued to restrict access to the area. Specifically, the electronic lock installed at the entrance to the area can be locked, and a "no entry" sign can be illuminated on the warning panel. A message can also be sent to the worker's wearable device warning them of approaching the area.

[0053] This document describes a specific example of a periodic inspection workflow at a power plant. Automatic execution can be set as a trigger at specific times each day (e.g., 6:00, 14:00, 22:00). The processing flow can be as follows: First, the inspection robot is instructed on its inspection route (step S11). The inspection route is defined as a list of spatial identifiers (e.g., TURBINE-01, TURBINE-02, BOILER-01, BOILER-02, GENERATOR-01, etc.), and an efficient inspection sequence can be calculated using a shortest path algorithm. The inspection robot can then autonomously begin moving along the instructed route. Upon reaching the location of each spatial identifier, the robot performs the following data collection (step S12): A visible light camera captures the external appearance of the equipment, allowing for visual confirmation of abnormalities such as corrosion, discoloration, liquid leakage, and detached parts. An infrared camera measures the temperature distribution of the equipment surface, allowing for the detection of localized high-temperature and low-temperature areas. Acoustic sensors can record sounds emitted from the equipment and detect abnormal sound patterns (e.g., whistling, rattling, high-pitched noises). Gas sensors can measure the concentration of flammable gases (e.g., methane, propane, etc.) and toxic gases (e.g., hydrogen sulfide, carbon monoxide, etc.) in the surrounding air. The collected data is transmitted to an edge server and analyzed immediately (step S13). An image processing engine can quantitatively evaluate the corrosion area ratio, the area of ​​discolored regions, and the presence or absence of leaked liquid from visible light images. Deep learning models (e.g., architectures such as EfficientNet and ResNet) can be used to classify normal and abnormal states.

[0054] Temperature anomalies are detected by infrared image analysis (step S14). Using the normal temperature range (for example, a range of approximately 80°C to 100°C for the turbine bearing section) as a reference, areas deviating from the reference range can be determined as abnormal. Statistical methods (for example, a method using the mean plus three times the standard deviation as a threshold) and machine learning methods (for example, outlier detection using Isolation Forest) can be used in combination to determine temperature anomalies. Abnormal noises are detected by acoustic analysis (step S15). The acoustic spectrum during normal operation is recorded as a baseline, and changes in the spectrum can be detected. In particular, an increase in high-frequency components characteristic of bearing deterioration (for example, a frequency band of approximately 10kHz to 20kHz) and an increase in rotational frequency components characteristic of rotational imbalance can be detected. Fast Fourier Transform (FFT) and wavelet transform can be used for acoustic analysis. Gas leaks are detected by gas concentration analysis (step S16). An abnormality can be determined if the flammable gas concentration exceeds a certain percentage (e.g., around 25%) of the lower explosive limit (LEL), or if the toxic gas concentration exceeds the permissible concentration limit (TLV). Changes in gas concentration over time are also monitored, and a warning can be issued if there is a rapid upward trend. A list of spatial identifiers where abnormalities are detected is created (step S17). Each abnormality can be assigned an abnormality score (e.g., a numerical range from 0 to 100), an abnormality type (temperature abnormality, vibration abnormality, acoustic abnormality, gas leak, etc.), an estimated failure mode (e.g., bearing deterioration, pipe corrosion, valve leak, insulation deterioration, etc.), and an urgency level (e.g., four levels: emergency, high, medium, low). The urgency level can be determined by considering the abnormality score, past failure cases, and the importance of the equipment (whether or not it will lead to a power generation shutdown).

[0055] A report of the inspection results is sent to the maintenance personnel (Step S18). The report can include the date and time of the patrol inspection, a list of the inspected spatial identifiers, a list of detected anomalies (sorted in descending order of anomaly severity), detailed information on each anomaly (captured images, temperature distribution map, acoustic spectrum, gas concentration graph), and recommended actions (e.g., detailed inspection required, parts replacement required, emergency response required). The report is automatically generated in PDF format and sent via email, and can also be registered in a business application (e.g., kintone). If a high-priority anomaly (e.g., an anomaly score of 90 or higher) is detected, an alert is immediately issued (Step S19). The alert is posted to an emergency channel on Slack and can also be sent via SMS to the maintenance personnel's mobile phones. At the same time, a warning light and sound can be activated on the alarm panel in the control room. Measures to prohibit entry to the relevant spatial identifier can be automatically implemented. Specifically, the electronic lock installed at the entrance to the area can be locked, and a "No Entry" sign can be illuminated on the warning display board. Messages can also be sent to workers' wearable devices warning them of approaching the area. A drone is automatically flown to the designated spatial identifier to perform a detailed check. The drone can capture detailed images from multiple angles using high-resolution and infrared cameras. The captured images can be displayed in real time on monitors in the control room, allowing maintenance personnel to check the situation remotely. This workflow enables 24 / 7 continuous monitoring of the power plant, allowing for early detection and rapid response to anomalies.

[0056] This section describes a specific example of predictive maintenance using generative AI in a power plant. The generative artificial intelligence engine 140 can learn from data over a certain period (e.g., about 3 years) and make predictions. For the turbine bearing with spatial identifier "TURBINE-02", it can predict that there is a high probability (e.g., about 85%) of failure within a certain period (e.g., within 2 weeks). As the basis for the prediction, time-series analysis of vibration data confirms that high-frequency components characteristic of bearing deterioration (frequency components corresponding to the outer ring passing frequency BPFO) have gradually increased over a certain period (e.g., 6 months). It can also confirm that the effective value (RMS value) of vibration acceleration has risen from a certain period ago (e.g., 6 months ago) (e.g., about 2.5 millimeters per second squared) to the current value (e.g., about 5.8 millimeters per second squared). By comparing with the standards of international standards (e.g., ISO 10816-3), it can determine whether or not the level exceeds the warning level. Analysis of acoustic data confirms that the sound level generated from the bearing has risen by a certain value (e.g., about 10 decibels). Frequency spectrum analysis can reveal a significant increase in components in a specific frequency band (e.g., around 15 kHz), which may indicate a potential sign of inner ring separation in the bearing. Temperature data analysis can confirm that the bearing temperature has risen within a certain range (e.g., an average of 90°C to 105°C) over a certain period (e.g., 6 months). By comparing this to the temperature range during normal operation (e.g., 80°C to 100°C), it can be determined whether the upper limit has been exceeded. Past failure cases in similar equipment can be referenced, such as a case in the same type of turbine where bearing failure occurred after a typical period (e.g., about 14 days) following similar vibration, acoustic, and temperature patterns.

[0057] As recommended actions, several countermeasures can be proposed. First, it can be proposed to conduct a detailed inspection a certain number of days before the predicted failure date (e.g., 11 days prior). In the detailed inspection, a detailed vibration spectrum can be measured using a vibration measuring instrument (FFT analyzer), and the condition of the inside of the bearing can be non-destructively tested using an ultrasonic diagnostic device. Lubricant can be sampled and the amount of metal powder contamination can be analyzed. Second, if bearing deterioration is confirmed as a result of the detailed inspection, it can be proposed to carry out preventive maintenance work a certain number of days before the predicted failure date (e.g., 7 days prior). In the preventive maintenance work, the turbine can be shut down and the bearing can be replaced with a new one. At the same time, related bearing mounts, lubrication piping, and seal parts can also be inspected and replaced as necessary. The work period can be scheduled for a certain amount of time (e.g., about 24 hours). Third, after the bearing replacement, normal operation can be confirmed by vibration measurement. It can be confirmed that the RMS value of the vibration acceleration is below the standard value (e.g., below 3 millimeters per second squared) and the temperature is within the normal range (e.g., below 90°C). After conducting a trial run for a set period (e.g., one week) to confirm that vibration, sound, and temperature have stabilized, normal operation can be resumed. Once maintenance personnel review and approve this prediction and recommended action, the workflow can be automatically updated. Work instructions for detailed inspections can be automatically registered in a business application (e.g., kintone) and notified to the inspection personnel. The process of ordering parts for preventive maintenance work can be automatically initiated, and purchase orders for necessary parts (e.g., bearings, bearing seals, lubricants, etc.) can be automatically generated. A certain number of days before the work is to be carried out, the turbine shutdown plan can be automatically notified to the power system operations department, allowing for adjustments to the power supply plan. This predictive maintenance can, in some cases, avoid power generation shutdowns due to sudden turbine failures and minimize downtime through planned maintenance.

[0058] As a third embodiment, an example of application to automated patrol inspection of indoor and outdoor areas in a manufacturing plant will be described. The manufacturing plant can consist of an indoor manufacturing area, an outdoor material storage area, and an outdoor equipment area (power receiving equipment, cooling towers, compressor rooms, etc.). Each area can be assigned a spatial identifier (for example, in the format of spatial ID FACTORY-B-INDOOR-LINE01 to OUTDOOR-EQUIPMENT-05). As sensing devices, fixed-point cameras can be placed along the indoor manufacturing line and also in the outdoor area to enable 24-hour continuous monitoring. The cameras are high-resolution IP cameras, and data can be transmitted efficiently using video compression technology (for example, H.265). A mobile inspection robot can be deployed to patrol the manufacturing line, warehouse, and outdoor area. The inspection robot can be equipped with an omnidirectional camera (a camera with a 360-degree field of view), a LiDAR sensor (range sensor), a thermal imaging camera, and a gas sensor (a sensor capable of detecting flammable gases and organic solvent vapors). Drones can be deployed to perform high-altitude inspections (roofs, chimneys, elevated tanks, etc.) and wide-area monitoring in outdoor areas. Drones can be equipped with optical zoom cameras (e.g., cameras with approximately 30x zoom) and thermal cameras. Wearable devices can be distributed to employees to provide location information, vital signs (heart rate, body temperature), and fall detection functions. Environmental sensors can be placed indoors and outdoors to measure temperature, humidity, atmospheric pressure, illuminance, noise levels, and air quality (CO2 concentration, volatile organic compound concentration). Data acquired from these devices can be transmitted to edge servers via Gigabit Ethernet or wireless LAN (e.g., Wi-Fi 6) within the factory. Edge servers can be located in manufacturing buildings, warehouse buildings, and administration buildings and operated in coordination.

[0059] This document describes a specific example of a safety inspection workflow in a factory. As a trigger, automatic execution can be set to occur at a specific minute each hour (e.g., 0 minutes past the hour), allowing for multiple inspections per day (e.g., 24 times). The processing flow can be as follows: First, inspection robots and drones are instructed on their inspection routes (step S21). The inspection robot can be responsible for indoor and ground-level outdoor areas, while the drone can be responsible for high-altitude outdoor areas. The inspection route is defined as a list of spatial identifiers, and priorities can be dynamically adjusted based on the results of the previous inspection. For example, areas where abnormalities were detected in the previous inspection can be set to be inspected preferentially. Upon reaching the location of each spatial identifier, the following inspections are performed (step S22): Cameras can be used to check the operating status of manufacturing equipment, the arrangement of materials, the presence of obstacles in passageways, and the arrangement of safety equipment such as fire extinguishers. LiDAR sensors can be used to measure passageway width, the height of loaded items, and floor level differences to confirm whether safety standards (e.g., passageway width of 1.2 meters or more, loading height of 2.5 meters or less) are met. Thermal imaging cameras can detect temperature abnormalities in electrical panels, motors, piping, etc. Gas sensors can measure the concentration of flammable gases and organic solvent vapors and confirm whether they exceed a certain percentage of the lower explosive limit (e.g., around 25%) or the permissible concentration for the work environment. Environmental sensors can confirm whether the work environment (temperature, humidity, CO2 concentration) meets the standards of the Industrial Safety and Health Act. For example, it can confirm that the room temperature is below a certain temperature (e.g., below 28°C) and the CO2 concentration is below a certain concentration (e.g., below 1000 ppm).

[0060] The collected data is analyzed to detect anomalies (step S23). The image processing engine can automatically detect anomalies such as: obstacles in passageways (e.g., pallets, trolleys, tools, etc. left lying around), fire extinguishers not being installed or expired, items being placed in front of emergency exits, workers not wearing protective equipment (helmets, safety shoes, safety glasses), and liquid leaks (oil or water spreading on the floor). These anomaly patterns can be detected using deep learning models (e.g., object detection models such as YOLOv8). Thermal image analysis can detect temperature anomalies. Anomalies such as when the surface temperature of an electrical panel exceeds a certain temperature (e.g., exceeds 80°C), when the surface temperature of a motor exceeds a certain temperature (e.g., exceeds 100°C), and when there is a localized high-temperature area in a pipe connection (a part that is more than a certain temperature higher than the surrounding area, e.g., a part that is more than 20°C higher). Gas concentration analysis can detect abnormalities such as when the flammable gas concentration exceeds a certain percentage of the LEL (e.g., exceeding approximately 1.25 volume%), or when the organic solvent vapor concentration exceeds a certain percentage of the permissible concentration (e.g., exceeding 50%). Violations of environmental standards can be detected. Violations such as when the room temperature exceeds a certain temperature (e.g., exceeding 28°C), when the CO2 concentration exceeds a certain concentration (e.g., exceeding 1000 ppm), or when the noise level exceeds a certain level (e.g., exceeding 85 decibels) can be detected. If an abnormality is detected, the severity is determined (step S24). Severity can be evaluated in multiple stages (e.g., four stages: critical, high, medium, and low). Critical can be defined as requiring immediate action and posing a risk to human life, high as requiring action within a certain time (e.g., within 24 hours) and posing a significant safety risk, medium as requiring action within a certain period (e.g., within 1 week) and posing a safety risk, and low as requiring action within a certain period (e.g., within 1 month) and having room for improvement. The severity of an issue can be determined by considering the type of anomaly, its location, time, and surrounding work conditions.

[0061] If the severity is urgent or high, immediate action is taken (Step S25). The details of the anomaly can be posted to the safety management channel on Slack, and a notification can be sent via SMS to the safety manager's mobile phone. Warnings can be sent to the wearable devices of workers in the area, attracting their attention with vibration and warning sounds. Access to the area can be restricted. Specifically, entrances can be locked with electronic locks, warning signs can be illuminated to indicate no entry, and warning messages can be displayed on digital signage. If the flammable gas concentration exceeds a certain percentage of the LEL (e.g., more than 25%), the ventilation fans in the area can be automatically activated at maximum airflow to reduce the gas concentration. A drone can be flown to the area to check the detailed situation (Step S26). The drone's zoom camera can capture detailed images of the anomaly at high altitudes. The captured images can be displayed in real time on a monitor in the safety management room. A record of corrective actions is automatically created in a business application (e.g., kintone) (Step S27). Records can include the date and time of occurrence, spatial identifier, anomaly description, severity, captured images, response history, responsible person, and corrective action deadline. The progress of corrective actions can be visualized in real time on a dashboard, and unaddressed cases can be automatically escalated if the deadline is exceeded. If the severity is medium or low, it can be compiled into a daily report and reported periodically (step S28). At a specific time each day (e.g., 5 PM), a report summarizing all anomalies detected that day can be automatically generated and emailed to the safety manager. Weekly and monthly summary reports can also be generated to analyze anomaly occurrence trends. This workflow strengthens safety management within the factory and helps prevent workplace accidents.

[0062] This section describes a specific example of safety improvement suggestions using generative AI in a factory. The generative artificial intelligence engine 140 learns from data over a certain period (e.g., one year) and can obtain analysis results. In the spatial identifier "INDOOR-LINE03," it can detect that the frequency of obstacle detection in the aisle is higher (e.g., about three times higher) compared to other areas. Specifically, it can be confirmed that the average monthly number of detections (e.g., about 15) is significantly higher than the average number of detections in other areas (e.g., about 5). The breakdown of detected obstacles can be analyzed and classified into pallets, trolleys, toolboxes, and others. The time periods in which obstacles are detected can be analyzed and confirmed to be concentrated in specific time periods (e.g., from 2 PM to 4 PM). It can be confirmed that this time period coincides with the time when production line setup changes are performed. It can be understood that during setup changes, a large number of pallets and trolleys are moved and often temporarily placed in the aisle. Interviews with workers can be confirmed to show that there is insufficient space to place pallets and trolleys during setup changes. Furthermore, it can be confirmed that the time constraints of the setup changeover process have led to the habitual temporary placement of items in the passageways. The generated AI can recommend several countermeasures. These include creating a new temporary storage space, reviewing the standard time for setup changeovers, providing training to workers, and introducing a real-time warning system using AI cameras. Implementing these improvement suggestions may significantly reduce the frequency of obstacle detection in the area in question.

[0063] As a fourth embodiment, an example of application to management at a construction site will be described. A construction site has areas such as buildings under construction, material storage areas, heavy equipment areas, and worker offices, and each area can be assigned a spatial identifier (for example, in the format of spatial ID CONST-C-BUILDING-A, MATERIAL-AREA-01, EQUIPMENT-ZONE-02). As sensing devices, a fixed-point camera can be placed in a position that overlooks the entire site to monitor the construction status and the movement of workers. A drone can be deployed to perform 3D surveying and progress checks from above. The drone can be equipped with a high-resolution camera, a multispectral camera, and a LiDAR sensor to acquire 3D point cloud data. IoT sensors (vibration sensors, noise sensors, temperature sensors, humidity sensors) can be placed on site to monitor environmental conditions and construction quality. Wearable devices can be distributed to workers to acquire location information and vital signs (heart rate, body temperature). The data acquired from these devices can be transmitted to an edge server via a 5G line or Wi-Fi and analyzed in real time.

[0064] This section describes a concrete example of a progress management workflow at a construction site. As a trigger, automatic execution can be set at a specific time each day (e.g., 5 PM). The processing flow can be as follows: First, images taken over a certain period of time (e.g., 24 hours) are acquired from fixed-point cameras and drones (step S31). Next, an image processing engine recognizes the placement of materials and the construction status of structures (step S32). Using a deep learning model (e.g., YOLOv8), rebar, formwork, concrete, material storage areas, heavy machinery, etc., can be detected, and the position, quantity, and status (installed, in installation, not installed) of each object can be determined. Subsequently, the three-dimensional information processing engine 122 measures the volume and height from the point cloud data (step S33). Three-dimensional point cloud data is generated from oblique images acquired by the drone using SfM processing, and the amount of concrete poured, embankment, excavation, etc., at each spatial identifier can be automatically measured. The measurement results are aggregated by spatial identifier (step S34). The construction progress rate for each spatial identifier can be calculated and compared to the planned progress rate. A progress report is automatically created in a business application (e.g., kintone) (step S35). The report can include the construction progress rate for each spatial identifier, a list of spatial identifiers experiencing delays, photos, and 3D visualization data. A Slack notification is sent to the site supervisor for spatial identifiers experiencing delays (step S36). The notification message can include the relevant spatial identifier, planned progress rate, actual progress rate, and a link to detailed information. This workflow automates the daily progress check, allowing site supervisors to focus on addressing delay areas.

[0065] This section describes a specific example of a safety management workflow at a construction site. Real-time monitoring can be set as a trigger, allowing immediate processing when sensor values ​​exceed a threshold. The processing flow can be as follows: First, if the vibration sensor exceeds a specified value, the relevant spatial identifier is identified (step S41). For example, if the vibration level in a certain spatial identifier exceeds a certain value (e.g., around 50 gals), it can be determined to be an anomaly. Next, the location of nearby workers (wearers of wearable devices) is confirmed (step S42). A list of workers in the relevant spatial identifier and adjacent spatial identifiers can be obtained. If workers are in a hazardous area, a warning sound is emitted (step S43). The worker's wearable device can emit a warning sound and also transmit an alert through vibration. A real-time notification is sent to the site supervisor and safety manager (step S44). The notification message can include the relevant spatial identifier, vibration level, number of workers in the hazardous area, etc. A drone is automatically flown to the relevant spatial identifier to check the situation (step S45). The drone can move above a designated spatial identifier and take images using infrared and visible light cameras. The captured images are analyzed to determine if there are any anomalies (step S46). The image processing engine can automatically detect ground subsidence, structural tilt, crack occurrence, etc. The determination results are recorded, and a report is automatically created in the business application (step S47). The report can record the time of occurrence, spatial identifier, anomaly details, captured images, and response history. This workflow strengthens safety management and helps prevent accidents.

[0066] This section describes a specific example of optimization using generative AI at a construction site. After a certain period of time has elapsed since the start of operation (for example, about 3 months), the generative artificial intelligence engine 140 can analyze past log data. It can detect that progress delays are occurring frequently in a certain spatial identifier. It can confirm that the frequency of delays is higher (for example, more than twice as high) compared to other spatial identifiers. Analysis of camera footage from the area in question can confirm that shadows are generated due to sunlight during a specific time period (for example, from 2 to 4 pm), reducing image recognition accuracy. It can confirm that the image recognition accuracy is lower compared to other spatial identifiers (for example, around 60%, compared to over 90% for other spatial identifiers). It can confirm that the vibration sensor data acquisition frequency is at regular intervals (for example, every minute), and that instantaneous anomalies are not being detected. Analysis of past incident logs can confirm that there is a delay of a certain amount of time (for example, an average of about 2 minutes) between the occurrence of an anomaly and its detection. The generative AI can recommend multiple countermeasures. It can suggest things like installing additional cameras, changing the sensor sampling frequency, and adding HDR processing to the image processing engine. Once a user approves a proposal, the workflow can be automatically updated. After the new configuration is put into operation and an evaluation is conducted after a certain period, it may be possible to confirm effects such as a reduction in the frequency of progress delays in the relevant area, improved image recognition accuracy, and reduced delays in anomaly detection.

[0067] As a fifth embodiment, an example of application to management in a golf course will be described. The golf course 800 has multiple holes (e.g., 18 holes), a clubhouse, a practice area, cart paths, etc., and each area can be assigned a spatial identifier (e.g., spatial ID GOLF-D-HOLE-01 to HOLE-18, CLUBHOUSE, PRACTICE-RANGE). As sensing devices, fixed-point cameras can be placed around the teeing grounds and greens of each hole to monitor the progress of players and the course conditions. Drones can be placed to monitor the condition of the turf, bunkers, and water quality of ponds throughout the course. The drones can be equipped with high-resolution cameras, multispectral cameras, and thermal cameras to evaluate the health of the turf from multiple angles. Soil sensors can be placed on the fairways and greens of each hole to measure soil moisture, pH, and EC values. Weather sensors can be placed on the course to measure temperature, humidity, wind speed, wind direction, and precipitation. The data acquired from these devices can be transmitted to a server device 800 via the wireless network within the golf course and analyzed.

[0068] This document describes a specific example of a course management workflow at a golf course. The trigger can be set to automatically execute early each morning (e.g., 6:00 AM). The processing flow can be as follows: First, a drone flies over the entire course and photographs the condition of the grass at each hole (step S51). Visible light images can be acquired using a high-resolution camera, and near-infrared images using a multispectral camera. The vegetation index (NDVI) is calculated from the captured images (step S52). NDVI is an index that quantitatively evaluates the health of the grass; healthy grass shows a high value (e.g., around 0.7 to 0.9), while stressed grass shows a low value (e.g., around 0.3 to 0.5). The NDVI value for each spatial identifier is calculated and compared to a baseline value (step S53). Spatial identifiers with NDVI values ​​below the baseline can be extracted, and it can be determined that there is a problem with the health of the grass. Soil sensor data is analyzed to confirm whether the soil moisture is within the appropriate range (step S54). The system can extract spatial identifiers where soil moisture falls below a certain value (for example, spatial identifiers with a volumetric water content below 20%) and determine if irrigation is necessary. The system obtains the probability of precipitation for a certain period of time (e.g., 24 hours) from the weather forecast API (step S55). If the probability of precipitation is below a certain value (e.g., below 30%), it can be decided to irrigate. A report is sent to the course manager (step S56). The report can include the NDVI value, soil moisture, recommended actions (irrigation, fertilization, aeration, etc.) and drone images for each spatial identifier. The automated irrigation system is activated for spatial identifiers that require irrigation (step S57). The system can control the sprinklers corresponding to each spatial identifier and deliver the appropriate amount of irrigation. This workflow ensures that the course's turf condition is always good and provides players with a comfortable playing environment.

[0069] This document describes a specific example of a play progress management workflow at a golf course. The trigger is designed to run continuously during business hours, allowing for real-time monitoring of play progress. The processing flow can be as follows: First, camera equipment installed around the teeing grounds and greens of each hole detects player groups (step S61). An image processing engine uses an object detection algorithm to determine how many player groups are on each hole. The progress speed of each group is calculated (step S62). Travel time from the previous hole, time from tee shot to green, etc., are measured and compared to standard playing times. Groups with delays are identified (step S63). Groups significantly behind standard playing times (e.g., taking 25 minutes or more) compared to standard playing times (e.g., around 15 minutes per hole) can be extracted. The spatial identifier of the delayed group's location is recorded, and the marshal (play progress manager) is notified (step S64). The marshal can then proceed to the relevant hole and encourage play to proceed. The overall congestion status of the course is visualized on a dashboard (step S65). The system can display real-time information such as waiting times for each hole, the pace of play, and the estimated completion time. This workflow ensures smooth gameplay and improves player satisfaction.

[0070] This document describes a specific example of optimizing turf management at a golf course using generative AI. The generative artificial intelligence engine 140 learns from data over a certain period (e.g., one year) and can obtain analysis results. It can detect that the health of the turf at a certain spatial identifier (e.g., the green at HOLE-05) is lower compared to other holes. It can confirm that the NDVI value is lower (e.g., around 0.6) compared to the average value of other holes (e.g., around 0.8). Time-series analysis of past data confirms that the NDVI value decreases significantly during a specific season (e.g., summer). Correlation analysis with weather data confirms that the NDVI value tends to decrease when there are consecutive days with temperatures exceeding a certain level (e.g., days exceeding 35°C). Analysis of soil sensor data confirms that the soil moisture at the corresponding spatial identifier is lower compared to other holes (e.g., volumetric water content of around 15%, compared to around 25% for other holes). Investigation of soil type confirms that the corresponding spatial identifier is sandy soil with low water retention. The generative AI can recommend multiple countermeasures. We can propose measures such as increasing the frequency of irrigation (for example, from once a day to twice a day), applying soil conditioners (adding materials that improve water retention), and temporarily installing shade nets (during the hot summer months). If the course manager approves this proposal, the irrigation schedule can be automatically updated and the soil conditioners can be automatically ordered. This optimization may improve the NDVI value of the relevant spatial identifier (for example, from around 0.6 to around 0.75), and it is expected that the health of the turf will improve.

[0071] As a sixth embodiment, an example of application to agricultural management will be described. Farmland 900 is divided into multiple fields, and each field can be assigned a spatial identifier (for example, in the format of spatial ID FARM-E-FIELD-01 to FIELD-20). As sensing devices, soil sensors can be placed in each field to measure soil moisture, pH, and EC value. Each sensor can take measurements at multiple depths (for example, 10 cm, 30 cm, 50 cm) to obtain a detailed understanding of the moisture status of the rhizosphere. Weather sensors can be placed in the farmland to measure temperature, humidity, solar radiation, wind speed, and precipitation. Fixed-point cameras can be placed in each field to take time-lapse photographs of crop growth. A drone can be deployed to take multispectral photographs of a wide area of ​​the field and calculate the vegetation index (NDVI). The data acquired from these devices can be transmitted to a cloud server device 800 via LoRaWAN or a 4G line for analysis.

[0072] This section describes a specific example of an irrigation control workflow in agriculture. A trigger can be set to occur when soil moisture falls below a threshold. The processing flow can be as follows: First, soil sensor data for each spatial identifier is monitored (step S71). Soil moisture content can be acquired at regular intervals (e.g., every 10 minutes) and monitored in real time. Next, spatial identifiers whose moisture content falls below a set threshold are detected (step S72). For example, if the soil moisture content of a certain spatial identifier falls below a certain value (e.g., 20% volumetric water content), it can be determined that irrigation is necessary. The probability of precipitation for the next certain period (e.g., 24 hours) is obtained from a weather forecast API (step S73). The probability of precipitation, precipitation forecast, and temperature forecast for the target area can be obtained using the Japan Meteorological Agency's API or a private weather information service's API. If the probability of precipitation is below a certain value (e.g., 30% or less), the irrigation device is activated (step S74). If the probability of precipitation exceeds a certain value, irrigation can be postponed in the hope of natural rainfall. The amount of irrigation water is adjusted according to the soil conditions for each spatial identifier (step S75). The optimal amount of irrigation water can be calculated according to the type of soil (sandy, loamy, clayey), the type of crop (rice, vegetables, fruit trees), and the growth stage (sowing, growing, flowering, fruiting). For example, if a spatial identifier is sandy soil and it is the growing stage for tomatoes, a fixed amount of irrigation water per unit area (for example, about 5 liters per square meter) can be applied. Irrigation execution records are saved to cloud storage (step S76). The execution date and time, spatial identifier, amount of irrigation water, soil moisture content (before and after irrigation), weather conditions, etc. can be recorded. This workflow automates irrigation work, reduces the burden on workers, and enables the efficient use of water resources.

[0073] This section describes a specific example of a disease detection workflow in agriculture. A trigger can be set for weekly automatic execution (e.g., every Monday). The processing flow can be as follows: First, a drone flies over the entire field and acquires multispectral images (step S81). Visible light (RGB), near-infrared (NIR), and red edge images can be acquired. The vegetation index (NDVI) for each spatial identifier is calculated (step S82). NDVI is calculated using the formula (NIR - Red) / (NIR + Red), and healthy crops will show a high value (e.g., around 0.7 to 0.9), while diseased crops will show a low value (e.g., around 0.3 to 0.5). Spatial identifiers with NDVI values ​​below a baseline are extracted (step S83). Spatial identifiers whose NDVI values ​​are significantly lower than those of surrounding healthy crops (e.g., 10% or more lower) can be extracted. Detailed images of the corresponding spatial identifiers are analyzed (step S84). The image processing engine analyzes leaf color, shape, presence or absence of spots, etc., to estimate the type of disease (e.g., powdery mildew, downy mildew, brown spot disease, etc.). A deep learning model (e.g., convolutional neural network) can be used to classify disease patterns. A report is sent to the farm manager (step S85). The report can include a list of spatial identifiers suspected to be disease, the estimated type of disease, a graph of the NDVI value trend, and drone images. Recommended actions (e.g., pesticide application, removal of infected plants) can also be presented. This workflow enables early detection of diseases and minimizes the spread of damage.

[0074] This section describes a specific example of optimization using generative AI in agriculture. The generative artificial intelligence engine 140 learns from data over a certain period (e.g., one year) and can obtain analysis results. It can detect that the effect of improving plant growth after irrigation is lower in a given spatial identifier than in other fields. Specifically, it can confirm that the rate of increase in the vegetation index (NDVI) after irrigation is lower (e.g., around 8%) compared to the average value of other fields (e.g., around 15%). Image analysis can confirm that puddles remain on the surface of the area in question for a long time. This can be interpreted as suggesting a potential problem with drainage. Soil analysis can confirm that the soil in the area in question has a higher clay content (e.g., 35%, compared to an average of around 20%) and lower permeability compared to other fields. The generative AI can recommend several countermeasures. These include reducing the amount of irrigation (e.g., by 20%), implementing drainage improvement work (installation of subsurface or open drainage), and conducting detailed topographic surveys using drones. Once the farm manager approves this proposal, irrigation settings can be automatically changed and requests for quotes for drainage improvement work can be automatically issued. This optimization may improve yields in the relevant spatial identifiers.

[0075] As a seventh embodiment, an example of application to automated analysis and management in remote inspection of infrastructure facilities will be described. The inspection targets 1000 include infrastructure facilities such as bridges, tunnels, power transmission towers, and communication towers, and each facility can be assigned a spatial identifier (e.g., spatial ID in the format INFRA-F-BRIDGE-01, TUNNEL-02, TOWER-03). A drone can be deployed as a sensing device to inspect high places or places that are difficult for people to approach. The drone can be equipped with a high-resolution camera (e.g., 4K or higher resolution), a zoom camera (e.g., 30x or higher optical zoom), an infrared camera, and a LiDAR sensor to collect data from multiple angles. A ground-based high-resolution camera can be installed to take automatic photos at regular intervals. A vibration sensor can be installed on bridges, etc., to monitor the vibration characteristics of the structure. A tilt sensor can be installed to monitor the displacement of the structure. The data acquired from these devices can be transmitted to a cloud server device 800 via a 4G / 5G line for analysis.

[0076] This document describes a specific example of a periodic inspection workflow for infrastructure facilities. A trigger can be set for monthly or annual scheduled execution (e.g., the 1st of each month, or two specific days per year). The processing flow can be as follows: First, the drone is instructed to follow an inspection route (step S91). The inspection route is defined as a list of spatial identifiers (e.g., BRIDGE-01-PIER-A, BRIDGE-01-PIER-B, BRIDGE-01-GIRDER-01, etc.), and a flight path that allows for detailed photography of each part can be automatically generated. The drone can perform autonomous flight based on GPS information and 3D coordinate information linked to the spatial identifiers. Upon reaching the location of each spatial identifier, the following data collection is performed (step S92): A high-resolution camera can photograph the surface of the structure, recording deterioration conditions such as cracks, spalling, rust, and discoloration. A zoom camera can magnify and photograph distant or detailed deteriorated areas. An infrared camera can measure the surface temperature distribution, detecting internal voids and concrete spalling. LiDAR sensors can measure the three-dimensional shape of a structure, allowing for the detection of deformation and displacement. By comparing this with the three-dimensional shape from the previous inspection, changes over time can be quantitatively evaluated.

[0077] The collected data is analyzed to evaluate the deterioration status (step S93). Cracks can be automatically detected by the image processing engine. Using a deep learning model (e.g., a semantic segmentation model such as U-Net or SegNet), crack pixels can be extracted and the crack width, length, and density can be automatically measured. If the crack width exceeds a certain value (e.g., more than 0.3 millimeters) or the length exceeds a certain value (e.g., more than 1 meter), it can be determined to be significant deterioration. The progression of rust can be evaluated by hue analysis. The progression of corrosion can be quantitatively evaluated from the rust color (red rust, black rust, etc.) and area ratio. Concrete spalling can be detected by infrared image analysis. Internal voids and spalling can be estimated from abnormalities in the surface temperature distribution (areas that are more than a certain temperature higher or lower than the surroundings, e.g., areas with a difference of 5°C or more). The displacement of the structure can be evaluated by LiDAR data analysis. By calculating the difference between the current 3D shape and the previous measurement, if the displacement exceeds a certain value (for example, a displacement exceeding 10 millimeters), it can be determined that a structural problem may be occurring.

[0078] The soundness is evaluated based on the degree of deterioration (Step S94). A soundness rank (e.g., four levels: A, B, C, D) can be assigned to each spatial identifier. Rank A can be defined as sound and no action required, rank B as minor deterioration requiring observation, rank C as significant deterioration requiring early action, and rank D as requiring emergency action. The soundness rank can be determined by comprehensively evaluating the degree of cracking, the progression of rust, the presence or absence of peeling, the magnitude of displacement, etc. An inspection report is automatically generated (Step S95). The report can include the inspection date, a list of spatial identifiers to be inspected, the soundness rank of each spatial identifier, details of detected deterioration (type, location, size), captured images, and recommended countermeasures (priority of repair work, estimated cost). The report is automatically generated in PDF format and sent to the facility manager by email, and can also be registered in a business application (e.g., kintone). An alert is issued for spatial identifiers with a soundness rank of C or D (Step S96). The facility manager can be immediately notified and prompted to conduct a detailed investigation and develop a repair plan. In cases of high urgency (Rank D), access to or use of the affected area can be restricted. For example, in the case of a bridge, traffic restrictions can be imposed, and in the case of a power transmission tower, inspection work can be temporarily suspended.

[0079] A comparative analysis with past inspection data is performed (step S97). For the same spatial identifier, the results of multiple past inspections (e.g., a total of 6 inspections over the past 3 years) can be compared in chronological order to evaluate the rate of deterioration. For example, it can be confirmed that a crack that was 0.2 millimeters wide and 50 centimeters long at the time of the previous inspection has expanded to 0.4 millimeters wide and 80 centimeters long at the time of the current inspection. In this case, it can be determined that the crack is progressing rapidly (e.g., its width doubles and its length increases 1.6 times in six months), and that early repair is necessary. The results of the deterioration rate analysis are input into the generative AI to predict future deterioration (step S98). The generative artificial intelligence engine 140 learns past deterioration patterns and can predict the deterioration state after a certain period (e.g., 1 year later, 3 years later). Based on the prediction results, the optimal repair timing and repair method can be proposed. For example, assuming the current rate of deterioration continues, it is highly likely that the crack width will exceed a limit value (e.g., 1 millimeter) after a certain period (e.g., one year), so a recommendation can be made to carry out repairs within a certain period (e.g., within six months). This workflow enables efficient and comprehensive inspection of infrastructure facilities, allowing for early detection of deterioration and planned maintenance.

[0080] This section describes a specific example of an anomaly detection workflow in infrastructure facilities. As a trigger, continuous monitoring can be set up, and processing can be started immediately when the value of a vibration sensor or tilt sensor exceeds a threshold. The processing flow can be as follows: First, when a vibration sensor or tilt sensor detects an anomaly, the corresponding spatial identifier is identified (step S101). For example, if a vibration sensor installed on a bridge pier (spatial identifier BRIDGE-01-PIER-C) detects a value that significantly exceeds the normal vibration level (e.g., vibration more than three times the normal level), or if a tilt sensor detects a tilt exceeding a certain value (e.g., a tilt change exceeding 0.1 degrees), it can be determined to be an anomaly. The surrounding conditions of the spatial identifier where the anomaly was detected are checked (step S102). Weather conditions for the area (wind speed, precipitation, presence or absence of earthquake) can be obtained from the weather data API, and the cause of the anomaly can be estimated. For example, if strong winds (wind speed of 20 meters per second or more) are occurring, it can be determined that the vibration is due to wind. If an earthquake occurs, it can be determined that the anomaly is due to the effects of seismic motion.

[0081] Step S103: An emergency flight of the drone is made to the relevant spatial identifier. The drone follows a pre-set emergency inspection route and can take detailed photographs of the relevant spatial identifier and its surroundings. The high-resolution camera can detect the occurrence of new cracks, the enlargement of existing cracks, the detachment of members, deformation, etc. The infrared camera can check for the possibility of internal damage. The captured images are analyzed to determine whether or not there is structural damage (Step S104). The image processing engine can compare the image before and after the anomaly detection and automatically extract any newly occurring deterioration or changes. Difference analysis can detect changes at the pixel level. If structural damage is confirmed, an emergency alert is issued (Step S105). Facility managers, maintenance companies, and relevant organizations (e.g., road administrators, railway operators, etc.) can be notified immediately. The notification can include the relevant spatial identifier, the nature of the detected anomaly, the captured images, and the estimated level of risk. If necessary, measures such as restricting the use of the relevant facility or closing the road can be automatically implemented. For example, in the case of a bridge, a road closure notice can be displayed on an electronic signboard and the gates can be closed. The anomaly detection record is registered in the business application (step S106). The date and time of the anomaly, spatial identifier, type of anomaly, sensor value, captured image, implemented measures, and response history can be recorded. This workflow enables immediate detection of structural anomalies and rapid response.

[0082] This document describes a specific example of optimizing repair plans for infrastructure facilities using generative AI. The generative artificial intelligence engine 140 learns from data over a certain period (e.g., 10 years) and obtains analysis results. It can comprehensively analyze the deterioration progress, past repair history, and post-repair effects for multiple spatial identifiers. It can calculate repair priorities to achieve the greatest effect within a limited budget. Specifically, it can calculate a priority score considering the health rank, deterioration rate, social importance (traffic volume, availability of alternative routes, etc.) and repair costs for each spatial identifier. By carrying out repairs in order of spatial identifiers with high priority scores, optimal maintenance can be achieved under budget constraints. For example, if the repair budget for a given year is a fixed amount (e.g., 100 million yen), the repair plan can be automatically selected by incorporating spatial identifiers in order of their priority scores, and repair locations that can be implemented within the budget can be automatically selected. Spatial identifiers that are not repaired can be carried over to the repair plan for subsequent years and continuously monitored.

[0083] The generating AI can also propose repair methods. When multiple repair methods exist for each type of deterioration (for example, resin injection, surface coating, and cross-sectional repair for cracks), it can analyze past repair records and recommend the most effective and economical method. For example, for cracks approximately 0.5 millimeters wide, the AI ​​can recommend resin injection based on analysis showing it to be the most cost-effective method (repair cost is approximately a fixed amount per meter, with a service life of approximately 10 years). On the other hand, for larger cracks exceeding 1 millimeter wide, the AI ​​can recommend cross-sectional repair (repair cost is approximately a fixed amount per square meter, with a service life of approximately 20 years). The selection of a repair method can comprehensively consider the type and degree of deterioration, the type of structure, environmental conditions (coastal areas, mountainous areas, etc.), and past repair records. Once the facility manager reviews and approves this proposal, a repair plan can be automatically generated. The repair plan can include a list of spatial identifiers to be repaired, the deterioration status of each spatial identifier, recommended repair methods, estimated costs, implementation schedule, and expected effects (extension of service life). Based on the repair plan, the ordering process for repair work can be automatically initiated. This optimization allows for the most effective allocation of limited budgets and extends the lifespan of infrastructure facilities.

[0084] As a variation in remote inspection, the AI-powered automated diagnostic function can be further enhanced. Specifically, it is possible to comprehensively evaluate the structural integrity by integrating and analyzing multiple types of sensor data and image data. For example, by calculating the natural frequency of a structure from vibration sensor data and comparing it with the natural frequency at the time of design, the decrease in the structural rigidity can be quantitatively evaluated. If the natural frequency has decreased by a certain percentage or more from the design value (for example, by more than 10%), it can be determined that the structural rigidity has decreased and that repair or reinforcement is necessary. By monitoring the displacement of a structure over the long term from tilt sensor data, the rate of displacement progression can be evaluated. If the displacement is progressing continuously at a constant rate (for example, if the displacement is increasing by 5 millimeters per year), it can be determined that there is a possibility of problems such as ground settlement or buckling of the structure. By integrating and analyzing image data, vibration data, and tilt data, the cause of deterioration can be identified and more appropriate repair methods can be proposed. In this way, by combining multiple data sources, it is possible to elucidate complex deterioration mechanisms that are difficult to judge with a single data source and achieve more accurate diagnoses.

[0085] As a modification 1, a hybrid architecture that links edge computing and cloud computing will be described. In the first embodiment, a configuration in which the main processing is performed on a cloud server device 10 was described, but in this modification, a configuration is adopted in which processing requiring real time is performed on the edge server, and advanced analysis and long-term learning processing are performed on the cloud. The edge server is an industrial computer installed on-site (e.g., NVIDIA Jetson AGX Orin, Intel NUC Pro, etc.) and is equipped with a CPU, GPU, memory, and storage. On the edge server, preprocessing of sensing data (noise reduction, format conversion, etc.), real-time image analysis (object detection, anomaly detection, etc.), and transmission of emergency control commands (safety stop, warning issuance, etc.) are performed. For example, if a worker enters a dangerous area at a construction site, the edge server immediately detects the intrusion through image analysis and issues a warning sound within 100 milliseconds. On the cloud server device 10, complex 3D reconstruction processing, configuration proposals by generative AI, and long-term storage and analysis of large amounts of log data are performed. Data linkage between the edge and the cloud is realized by a streaming method using MQTT or Kafka. The edge server sends real-time analysis results (anomaly detection results, object detection results, etc.) to the cloud, and the cloud sends long-term trend analysis and optimization suggestions to the edge.

[0086] As a concrete example of edge-cloud integration, consider quality inspection on a manufacturing line. The edge server receives 30 frames per second of images from the line camera. The edge server uses an inference engine such as TensorRT to perform real-time defect detection on each frame. The inference time is 10 milliseconds per frame, achieving 30 FPS real-time processing. If a defect is detected, the edge server immediately sends a stop command to the manufacturing line to prevent defective products from being released. Simultaneously, it sends the defect image and detection results to the cloud. The cloud server device 10 analyzes the collected defect data and analyzes the defect occurrence trend (correlation with time of day, manufacturing conditions, raw material lot, etc.). The generating AI learns a model to predict defect occurrence and makes a suggestion such as, "There is a possibility that the defect occurrence rate will increase under manufacturing condition A tomorrow afternoon, so we recommend proactively changing to condition B." This suggestion is sent to the edge server, and the control parameters of the manufacturing line are automatically adjusted. Edge-cloud integration achieves both real-time performance and advanced analysis. Even in the event of a network failure, the edge server can continue to operate independently, improving system availability. The edge server stores data from the past 24 hours locally and sends it to the cloud after network recovery. This architecture corresponds to a configuration in which the execution control unit 160 and the analysis engine group 120 are distributed between the edge and the cloud, and has the advantage of achieving both low latency and high processing power.

[0087] As a second modification, a multimodal sensing integration function that integrates and analyzes multiple types of sensing modalities (image, acoustic, vibration, temperature, humidity, gas, etc.) will be described. In the first embodiment, a configuration in which each analysis engine processes sensing data individually was described, but in this modification, an integrated analysis engine is added that simultaneously receives multiple modalities as input and analyzes them complementaryly. The integrated analysis engine 126 uses a multimodal deep learning model (e.g., a Transformer-based multimodal encoder) to map image features, acoustic features, vibration features, etc., into a common feature space and process them integrally. As a specific example, consider a case in equipment anomaly detection where the image appears normal but the vibration data is abnormal, or conversely, the vibration data is normal but the image is abnormal. While this may be missed with analysis of a single modality, multimodal integration allows for the detection of anomalies by comprehensively judging multiple information sources. For example, in pump anomaly detection, a vibration sensor detects bearing deterioration, an acoustic sensor detects abnormal noise, and a thermal imaging camera detects a localized temperature rise. By integrating this information, it becomes possible to make a highly accurate prediction such as, "The bearing is making abnormal noises and generating heat due to deterioration, and there is a high probability of failure within 24 hours."

[0088] Another example of multimodal integration is disease detection in crops in agriculture. Visible light camera images show that the leaves have a slightly yellowish tint. Near-infrared camera images show that the reflectivity is reduced compared to healthy leaves. The NDVI (Normalized Density Index) calculated from multispectral camera data is 10% lower compared to surrounding healthy crops. By integrating this multiple pieces of information, it is determined that "there is a high probability of an early-stage disease occurring." Early-stage diseases that are difficult to judge with a single camera image alone can be detected early through multimodal integration. The integrated analysis engine 126 assigns a spatial ID and timestamp to the data of each modality, generating a spatiotemporalally synchronized dataset. Using this dataset as input, a Transformer-based attention mechanism learns the correlations between each modality. The trained model performs integrated judgments on new data, dynamically adjusting the importance of each modality. For example, adaptive integration is achieved by giving higher weight to acoustic and vibration data at night when image reliability is low, and higher weight to images during the day. This multimodal integration capability represents an extension of the analysis engine and offers the advantage of enabling highly accurate analysis that is difficult to achieve with a single modality.

[0089] As a third modification, we will describe a predictive workflow automatic generation function in which the generating AI predicts future events from past data and generates and executes a workflow in advance based on the prediction. In the first embodiment, a reactive configuration in which a workflow is executed after an event occurs was described, but in this modification, a predictive configuration is adopted in which preventive measures are taken before an event occurs. The prediction module uses a time series prediction model (LSTM, GRU, Transformer, Prophet, etc.) to predict future events from sensor data, weather data, work schedules, etc. As a specific example, consider weather forecasting at a construction site. Using weather forecast data obtained from a weather forecast API, real-time data from weather sensors installed at the site, and past weather pattern data as input, the AI ​​makes a prediction that "there is an 80% probability that strong winds (wind speed of 15 meters per second or more) will occur in spatial ID G05-G05 between 3pm and 4pm tomorrow." Based on this prediction, the generating AI generates the following workflow in advance: At 2pm tomorrow, temporarily suspend work in spatial ID G05-G05 and check the securing of materials. At 2:30 PM, a drone will be flown to check the securing of materials and the safety of the surrounding area. All workers will be evacuated to a safe location by 3:00 PM, when strong winds are predicted. After the strong winds subside at 4:00 PM, the drone will be used to assess the extent of the damage and determine whether work can resume. This workflow is automatically generated 24 hours before the event occurs and notified to the site supervisor. Once approved by the site supervisor, it will be automatically executed at the scheduled time.

[0090] As another example of predictive workflow automatic generation, consider failure prediction for plant equipment. Time-series analysis of vibration data predicts that "the bearing at spatial ID PUMP-05 has a 70% probability of failing in 7 days if the current deterioration rate continues." Based on this prediction, the generating AI pre-generates the following workflow: Three days later (four days before the predicted failure date), a detailed inspection is conducted to precisely check the condition of the bearing. If deterioration is confirmed as a result of the detailed inspection, preventive maintenance work is carried out five days later (two days before the predicted failure date). In the preventive maintenance work, the bearing is replaced with a new one, and related parts are inspected at the same time. After the work is completed, normal operation is confirmed by vibration measurement. This workflow avoids production stoppages due to sudden failures and enables planned maintenance. The effectiveness of predictive workflow automatic generation increases with improved prediction accuracy. Past prediction results are compared with actual occurrence results, and the prediction model is continuously improved. Even if the prediction is wrong (e.g., no strong winds occurred, no malfunction occurred), that information is stored as training data to improve the accuracy of future predictions. This predictive workflow automatic generation function is equivalent to an extension of automated workflow construction and has the advantage of enabling a shift from reactive to preventative responses.

[0091] As a fourth modification, a distributed autonomous cooperative control function is described in which multiple controlled devices communicate directly with each other and perform autonomous cooperative operations without waiting for instructions from a central server. In the first embodiment, a configuration was described in which the execution control unit 160 centrally transmits control commands. However, in this modification, a distributed configuration is adopted in which each device makes autonomous decisions and cooperates through inter-device communication (D2D, Device-to-Device). The distributed cooperative control module 165 places agent software on each controlled device 50 and achieves cooperative operation by exchanging messages between agents. DDS (Data Distribution Service), MQTT, or a custom protocol is used for inter-agent communication. As a specific example, consider the case where multiple AGVs perform transport work in a warehouse. In the conventional centralized system, the server instructs each AGV on its movement path, but only the server can know the relative positions of the AGVs, and communication with the server occurs frequently to avoid collisions. In distributed autonomous cooperative control, each AGV broadcasts its current position, target position, and speed, and other AGVs receive that information. Each AGV autonomously assesses the possibility of a collision based on the information it receives and adjusts its speed or changes its route as needed. For example, if AGV-01 and AGV-02 approach each other at an intersection, they exchange position information and autonomously perform a coordinated action where the one that arrives first has priority to pass, and the one that arrives later comes to a complete stop. Because communication with a server is not required, low-latency coordination is achieved, and operation can continue even in the event of a network failure.

[0092] Another example of distributed autonomous cooperative control is when multiple drones inspect a wide area. Because the inspection area is vast, it is necessary to divide the inspection among multiple drones. In a centralized system, a server pre-plans the flight paths of each drone, but it cannot flexibly respond to changes in on-site conditions (wind direction, detection of obstacles, etc.). In distributed autonomous cooperative control, each drone autonomously determines its assigned area and exchanges position information with other drones during flight. If one drone needs to return midway due to low battery power, the remaining drones autonomously re-divide their assigned areas to cover uninspected areas. Furthermore, if one drone detects an anomaly during inspection, it can request assistance from nearby drones and ask them to investigate further, demonstrating cooperative action. Multi-agent reinforcement learning is used in distributed autonomous cooperative control. Each agent (device) observes its own actions (movement, stopping, task execution, etc.) and the actions of other agents, and learns the optimal action to achieve the overall objective (minimizing task completion time, collision avoidance, etc.). The learned policy is deployed to each device and executed in real time. This distributed autonomous cooperative control function represents an extension of the cooperative control function and has the advantage of reducing dependence on a central server and improving scalability and fault tolerance.

[0093] As a fifth modification, we will describe a function that records important data such as sensing data, analysis results, and workflow execution history on a blockchain to achieve tamper-proof evidence management. In the first embodiment, a configuration in which log data is recorded in a database was described, but since the database can be tampered with by administrator privileges, there are challenges in terms of legal evidentiary value and audit compliance. In this modification, blockchain technology is utilized to guarantee the authenticity of the data. The blockchain integration module calculates the hash value (SHA-256, etc.) of important data and records it as a blockchain transaction. The blockchain used is either a public blockchain (Ethereum, Polygon, etc.) or a private blockchain (Hyperledger Fabric, Corda, etc.). As a specific example, consider construction records at a construction site. In concrete pouring work, data such as the start time of work, the completion time of work, the lot number of the ready-mix concrete used, the slump value, temperature, humidity, and captured images are collected. This data is stored in an integrated database, and the hash value of the data is recorded on the blockchain. The recorded hash value can prove that the data has not been tampered with later. For example, if a quality problem arises after construction and the data from the time of construction becomes the point of contention, comparing the hash value recorded on the blockchain with the hash value of the current data can objectively prove that the data has not been tampered with.

[0094] Another example of blockchain-based evidence management is the manufacturing history in pharmaceutical manufacturing. In pharmaceutical manufacturing, records compliant with GMP (Good Manufacturing Practice) are legally required. All data, including temperature, humidity, pressure, lot numbers of raw materials, workers, and inspection results during the manufacturing process, are recorded. By recording the hash values ​​of this data on the blockchain, tampering with manufacturing records becomes impossible, ensuring traceability for reporting to regulatory authorities and product recalls. Smart contract functionality allows for automation, such as automatically issuing alerts and triggering approval processes when specific conditions (e.g., temperature exceeding acceptable limits) are met. From the perspective of cost (transaction fees) and processing speed, the blockchain recording method employs a system that records only the hash values ​​of important data, rather than all data. The data itself is stored in a conventional database or cloud storage, and only the hash values ​​and metadata (timestamp, spatial ID, data type, etc.) are recorded on the blockchain. To verify the authenticity of the data, the hash value of the data obtained from the database is calculated and compared with the hash value recorded on the blockchain. If they match, it is guaranteed that the data has not been tampered with. This blockchain-based data trail management capability represents an extension of log keeping and has the advantage of improving legal evidentiary value and audit compliance capabilities.

[0095] While preferred embodiments of the present disclosure have been described in detail above with reference to the attached drawings, the technical scope of the present disclosure is not limited to such examples. It is clear to any person with ordinary skill in the art of the present disclosure that various modifications or alterations may be conceived within the scope of the technical idea set forth in the claims, and these will naturally also fall within the technical scope of the present disclosure.

[0096] The devices described herein may be implemented as a single device, or as a group of devices (e.g., cloud servers) that are partially or entirely connected by a network. For example, the control unit and storage of server 10 may be implemented as different servers connected to each other by a network.

[0097] The series of processes performed by the apparatus described herein may be implemented using software, hardware, or a combination of software and hardware. Computer programs for implementing each function of the server according to this embodiment can be created and implemented on a PC or the like. Furthermore, a computer-readable recording medium on which such a computer program is stored can also be provided. Examples of recording media include magnetic disks, optical disks, magneto-optical disks, and flash memory. Alternatively, the computer program may be distributed without using a recording medium, for example, via a network.

[0098] ( Furthermore, the processes described using flowcharts in this specification do not necessarily have to be executed in the order shown. Some processing steps may be executed in parallel. Additional processing steps may be adopted, and some processing steps may be omitted.

[0099] Furthermore, the effects described herein are merely descriptive or illustrative and not limiting. In other words, the technology relating to this disclosure may produce other effects that will be apparent to those skilled in the art from the description herein, in addition to or in lieu of the effects described herein.

[0100] Furthermore, the following configurations also fall within the technical scope of this disclosure. (Item 1) A spatial information integration unit that uses spatial identifiers to integrate and link data between multiple sensing devices, analysis engines, and business systems, A generative artificial intelligence engine that analyzes log data and business history to propose the configuration and workflow of the sensing device, the analysis engine, and the business system, An execution control unit that controls the sensing device, the analysis engine, and the business system based on the workflow described above, A spatial information integration platform system characterized by having the following features. (Item 2) The system according to claim 1, wherein the spatial identifier is generated based on location information and time information. (Item 3) The system according to claim 1, wherein the spatial information integration unit normalizes different sensing data spatiotemporally by assigning a spatial identifier to each data acquired from multiple sensing devices, unifying the acquisition time, transforming the coordinate system, and assigning the same event label. (Item 4) The system according to claim 1, wherein the analysis engine performs at least one of image processing, three-dimensional information processing, and anomaly detection processing. (Item 5) The system according to claim 4, wherein the analysis engine processes sensing data associated with the spatial identifier and assigns the same spatial identifier to the processing result. (Item 6) The system according to claim 1, wherein the generative artificial intelligence engine includes a processing configuration that analyzes past processing configurations and execution result logs for each spatial identifier, and proposes an optimal combination of sensing device, analysis engine, and business system according to environmental conditions, business content, and frequency of use. (Item 7) The system according to claim 6, wherein the generative artificial intelligence engine automatically constructs and visually displays a workflow based on the proposal. (Item 8) The system according to claim 6, wherein the generative artificial intelligence engine performs a learning process to improve the proposed configuration for subsequent times based on log information including the success or failure of execution of each component, processing time, user editing history, sensing data quality indicators, and business performance indicators. (Item 9) It includes an interface that allows the user to visually construct the workflow, and allows processing conditions and actions to be set for each spatial identifier. The processing conditions include spatial location conditions, sensing data conditions such as temperature or image analysis, and schedule conditions. The system according to claim 1, wherein the aforementioned actions include notification, device control, data storage, and activation of a business approval flow. (Item 10) The system according to claim 9, wherein the interface unit displays each process as a visually connectable block module. (Item 11) The system according to claim 9, wherein the interface unit visually displays the spatial identifier on a map display, and the user can place processing nodes on the target area to construct a workflow. (Item 12) The system according to claim 1, wherein the integration with the aforementioned business system is performed via an API with an external business application using the spatial identifier as a key. (Item 13) The system according to claim 12, wherein at least one of data storage, approval processing, and notification processing is performed using the spatial identifier as a key. (Item 14) The system according to claim 1, wherein the execution control unit transmits a control command to a movable controlled device based on the processing result based on the spatial identifier. (Item 15) The system according to claim 14, wherein the control command is transmitted by a communication protocol. (Item 16) The system according to claim 1, comprising a cooperative control function capable of controlling the coordinated operation between multiple controlled devices based on the spatial identifier. (Item 17) The system according to claim 1, which records processing logs, sensing data logs, and user operation history on a spatial identifier basis and uses them for training the generative artificial intelligence engine. (Item 18) The system according to claim 1, further comprising a state management function for recording and visualizing the execution status for each spatial identifier. (Item 19) The system according to claim 1, which provides access control that restricts or permits editing, execution, or approval operations of processes belonging to a target spatial identifier, by requiring, in addition to login authentication information, the current location of the user terminal to match the target spatial identifier in user authentication. (Item 20) The system according to claim 1, wherein security is ensured by encrypted communication during communication with external parties. (Item 21) The system according to claim 9, wherein the interface unit can be extended in a plug-in format with processing blocks, analysis engines, or data linkage nodes defined by external developers, and such extension elements support operation and template saving in the interface unit. (Item 22) The system according to claim 1, wherein the spatial identifier can be defined by a unit according to its application, and the spatial unit can be a hierarchy or construction step in the construction industry, an equipment unit or control zone in the plant industry, a field or shelf number in agriculture, a station section or vehicle in the railway industry, a shelf number or transport lane in the logistics industry, and an equipment label or inspection route in inspection work. (Item 23) A process that integrates and links data between multiple sensing devices, analysis engines, and business systems using spatial identifiers, The process involves analyzing log data and business history to propose the configuration and workflow of the sensing device, the analysis engine, and the business system using generative artificial intelligence, A process that controls the sensing device, the analysis engine, and the business system based on the workflow described above, An information processing method in a spatial information integration platform system, characterized by including the following: (Item 24) On the computer, A process that integrates and links data between multiple sensing devices, analysis engines, and business systems using spatial identifiers, The process involves analyzing log data and business history to propose the configuration and workflow of the sensing device, the analysis engine, and the business system using generative artificial intelligence, A process that controls the sensing device, the analysis engine, and the business system based on the workflow described above, A program to execute. [Explanation of Symbols]

[0101] 1 System 10 Server devices 20 Sensing Devices 30 External Business Systems 40 User terminals 50 Controlled Devices

Claims

1. A spatial information integration unit receives sensor data from multiple sensing devices, sets a spatial identifier based on the location information contained in the received sensor data, and stores the sensor data, the location information, and the time information when the sensor data was acquired in an integrated data storage unit in association with each other. An analysis engine that performs at least one of the following processes based on sensor data associated with the spatial identifier: image processing, three-dimensional information processing, and anomaly detection processing, and assigns the same spatial identifier to the data after processing. The business system integration unit uses the aforementioned spatial identifier as a key to connect with external business application systems via API, A generative artificial intelligence engine analyzes the sensor data stored in the integrated data storage unit, the analysis data obtained by the analysis engine, and the operational performance data including the business history data of the business application system, based on predetermined evaluation indicators, to generate combination information of the sensing device, the analysis engine, and the business application system, and a workflow based on the combination information. An execution control unit that controls the sensing device, the analysis engine, and the business application system based on the workflow described above, A spatial information integration platform system characterized by having the following features.

2. The system according to claim 1, wherein the location information included in the sensor data includes location information acquired by a location information device.

3. The system according to claim 1, wherein the spatial information integration unit normalizes sensor data acquired from multiple sensing devices in a spatiotemporal manner by associating a spatial identifier with each sensor data, unifying the acquisition time, transforming the coordinate system, and assigning the same event label.

4. The system according to claim 1, wherein the analysis engine performs image processing or three-dimensional information processing.

5. The system according to claim 1, wherein the analysis engine performs the anomaly detection process.

6. The system according to claim 1, wherein the generative artificial intelligence engine analyzes the processing configuration and execution result logs for each spatial identifier as past operational performance data, and evaluates information on environmental conditions, work content, and frequency of use based on the evaluation indicators, thereby proposing a combination of the sensing device, the analysis engine, and the business application system.

7. The system according to claim 1, wherein the generative artificial intelligence engine visually displays the workflow in the interface unit.

8. The system according to claim 6, wherein the generative artificial intelligence engine performs a learning process to improve the proposed configuration for the next time based on log information including the success or failure of execution of each component as past operational performance data, processing time, user editing history, sensor data quality indicators, and business performance indicators.

9. With the aforementioned workflow visually displayed on the interface, input information from the user is obtained to set processing conditions and actions for each spatial identifier. The processing conditions include spatial location conditions, sensing data conditions related to temperature or image analysis, and schedule conditions. The system according to claim 1, wherein the aforementioned actions include notification, device control, data storage, and activation of a business approval flow.

10. The system according to claim 9, wherein the interface unit displays each process as a visually connectable block module.

11. The system according to claim 9, wherein the interface unit visually displays the spatial identifier on a map display, and the user can construct a workflow by placing processing nodes on the target area in the interface unit.

12. The system according to claim 1, wherein at least one of data storage, approval processing, and notification processing is performed using the spatial identifier as a key.

13. The system according to claim 1, wherein the execution control unit transmits a control command to a movable controlled device based on the processing result based on the spatial identifier.

14. The system according to claim 13, wherein the control command is transmitted by a communication protocol.

15. The system according to claim 1, wherein the execution control unit includes a cooperative control function capable of controlling the coordinated operation between a plurality of controlled devices based on the spatial identifier.

16. The system according to claim 1, wherein the logging unit records processing logs, sensing data logs, and user operation history on a spatial identifier basis and uses them for training the generative artificial intelligence engine.

17. The system according to claim 1, further comprising a log recording unit that records and visualizes the execution status for each spatial identifier.

18. The system according to claim 1, further comprising a security management unit that, in user authentication, restricts or permits editing, execution, or approval operations of processes belonging to a target spatial identifier by requiring, in addition to login authentication information, the current location of the user terminal to match the target spatial identifier.

19. The system according to claim 1, further comprising a security management unit that uses an encryption protocol in communication with the outside world.

20. The system according to claim 9, wherein the interface unit can be extended in a plug-in format with processing blocks, analysis engines, or data linkage nodes defined by an external developer, and such extension elements correspond to operations and template saving in the interface unit.

21. The system according to claim 1, wherein the spatial identifier can be defined by a unit according to its application, and the spatial unit can be a hierarchy or construction step in the construction industry, an equipment unit or control zone in the plant industry, a field or shelf number in agriculture, a station section or vehicle in the railway industry, a shelf number or transport lane in the logistics industry, and an equipment label or inspection route in inspection work.

22. A spatial information integration process that receives sensor data from multiple sensing devices, sets a spatial identifier based on the location information contained in the received sensor data, associates the sensor data, the location information, and the time information when the sensor data was acquired, and stores them in an integrated data storage unit. Analysis using an analysis engine that performs at least one of the following processes based on sensor data associated with the spatial identifier, such as image processing, three-dimensional information processing, and anomaly detection processing, and assigns the same spatial identifier to the data after the processing, Using the aforementioned spatial identifier as a key, a business system integration process is performed to connect with an external business application system via an API, The process involves analyzing the sensor data stored in the integrated data storage unit, the analysis data obtained by the analysis engine, and the operational performance data including the business history data of the business application system, based on predetermined evaluation indicators, and proposing combination information of the sensing device, the analysis engine, and the business application system, and a workflow based on the combination information, using generative artificial intelligence. A process that controls the sensing device, the analysis engine, and the business application system based on the workflow described above, An information processing method in a spatial information integration platform system, characterized by including the following:

23. On the computer, A spatial information integration process that receives sensor data from multiple sensing devices, sets a spatial identifier based on the location information contained in the received sensor data, associates the sensor data, the location information, and the time information when the sensor data was acquired, and stores them in an integrated data storage unit. Analysis using an analysis engine that performs at least one of the following processes based on sensor data associated with the spatial identifier, such as image processing, three-dimensional information processing, and anomaly detection processing, and assigns the same spatial identifier to the data after the processing, Using the aforementioned spatial identifier as a key, a business system integration process is performed to connect with an external business application system via an API, The process involves analyzing the sensor data stored in the integrated data storage unit, the analysis data obtained by the analysis engine, and the operational performance data including the business history data of the business application system, based on predetermined evaluation indicators, and proposing combination information of the sensing device, the analysis engine, and the business application system, and a workflow based on the combination information, using generative artificial intelligence. A process that controls the sensing device, the analysis engine, and the business application system based on the workflow described above, A program to execute.

Images