System for providing object behavior analysis and personalized voice service in real time on basis of spatial information

Abstract

The present invention relates to an edge AI device for object tracking and collecting per-object spatial dwell information, the edge AI device comprising: a sensor unit for sensing an object in a designated area; a communication unit for communicating with other edge AI devices; a memory unit; and a processor unit, wherein the processor unit analyzes the object sensed by the sensor unit so as to output object analysis information, stores movement path information and dwell time information of the analyzed object in the memory unit, and creates the per-object spatial dwell information on the basis of the object analysis information that was output, the stored movement path information, and the stored dwell time information.

Description

Spatial Information-Based Real-Time Object Behavior Analysis and Personalized Voice Service Provision System

[0001] The present invention relates to a technology for managing vehicles on board a ship, and more specifically, to a vehicle management system and a method of operation thereof that enables real-time management of the entire process of vehicle entry, waiting, boarding, disembarking, and departure by utilizing Edge Artificial Intelligence (Edge AI) to identify and track vehicles boarding a ship.

[0002] The present invention relates to image-based object detection and behavior analysis technology. Specifically, it relates to a system and method for analyzing the behavior of an object in real time based on image data acquired through a plurality of edge AI devices, and providing voice relay data generated based on the analysis results to a user terminal. Furthermore, the present invention relates to a technology for analyzing information such as the spatial coordinates, relative position, movement patterns, and situational context of an object from various angles and converting it into voice content containing tactical meaning.

[0003] The present invention relates to a technical field for providing customized guidance services to visitors in indoor exhibition spaces such as museums, exhibition halls, and art galleries. More specifically, it relates to a personal AI guide system in which an edge AI device for spatial information analysis placed within an exhibition hall is linked with a personal edge AI device worn or carried by a visitor to provide multilingual voice guidance, interest-based question and answer, movement path guidance, and safety assistance functions in real time.

[0004] In large-scale complex facilities such as airports, it is important to analyze visitors' movement paths and dwell times at specific locations. Similarly, it is crucial to analyze dwell times at specific display shelves, such as how long visitors spent at specific items within a store. This is because such analysis facilitates the management of population density and the efficient operation of stores within the complex. However, existing visitor movement path analysis systems primarily collect information through centralized data processing and often rely on identity verification for object tracking, which can lead to privacy issues.

[0005] Modern security systems place great emphasis on the importance of identity verification and tracking. In particular, in facilities requiring a high level of security, such as airports, research institutes, and financial institutions, systems capable of accurately identifying the identities of entrants (people) and tracking their movement paths after entry are essential. Such security systems play a crucial role as a means to prevent potential threats and respond immediately to unexpected accidents or intrusions.

[0006] Traditional identity verification methods have utilized authentication techniques such as ID cards, passwords, and fingerprint recognition. However, these methods have limitations, such as requiring direct human intervention, relying heavily on external devices, and potentially being inefficient. Additionally, existing security systems may struggle to achieve sufficient speed and accuracy when tracking the movement paths of objects across multiple locations in real time.

[0007] Recently, with the introduction of Edge AI technology, it has become possible to process data on-site in real time and track objects without communication delays with a central server. Edge AI can analyze the location and movement path of objects in real time even if the devices are distributed, and devices installed at various locations within a security facility can cooperate to track objects more efficiently.

[0008] With the recent increase in demand for maritime transportation, such as passenger ships and ferries utilized as a means of public transportation, the efficient operation of systems for boarding vehicles is becoming increasingly important. In particular, to reduce congestion and delays during the boarding and disembarking of vehicles, technical means are required to efficiently manage vehicles within the dock by distinguishing between pre-booked and non-reserved vehicles, and to systematically track and control the entry, waiting, boarding, and disembarking processes.

[0009] Previously, vehicle verification relied mostly on manual human intervention or simple barcode-based identification, or methods determining eligibility for boarding were based solely on license plate recognition results at a single point. This approach makes it difficult to comprehensively track and manage the entire flow of a vehicle from its arrival at the dock to boarding; furthermore, failures or errors in license plate recognition delay reservation confirmation and boarding authorization; and the means for on-site managers and users to monitor the current status or communicate in real-time are limited. Consequently, there is a need for improvement in terms of vessel operational efficiency and user satisfaction.

[0010] Furthermore, in the existing structure that relies solely on a central server to collect and process vehicle information, issues such as data transmission delays, communication instability, and reduced real-time performance may occur, leading to difficulties in accurately allocating time for ship operations and establishing vehicle loading plans.

[0011] To address these issues, there is a need for an edge AI-based vehicle management system capable of detecting and analyzing vehicle information in real-time at the field level and providing customized guidance to users and administrators with minimal server connectivity when necessary. In particular, technical means are required to ensure system scalability and real-time responsiveness by deploying multiple edge AI devices at various locations—such as dock entrances, boarding points, disembarking points, and exits—and configuring these devices into a head-node structure.

[0012] With the recent advancements in image sensors and artificial intelligence, image-based object detection and behavior analysis technologies that recognize and analyze the movements of people or objects are being applied in various fields. In particular, there is a growing demand for technologies that can identify object movements in real time and infer their meaning in areas such as sports analysis, security surveillance, autonomous driving, healthcare, and assistance for the visually impaired.

[0013] Meanwhile, behavior-based voice relay systems are also being developed for users, such as the visually impaired, who have difficulty perceiving visual information. These systems aim to assist users in spatial perception and situational understanding by analyzing external video and conveying the situation in the form of text or voice.

[0014] Existing technologies have primarily adopted analysis structures based on a single video source or a central server, which means they fail to adequately reflect interactions between objects or tactical contexts in complex environments. Furthermore, real-time performance is degraded because all video data must be transmitted to a central server, stable relaying is difficult in the event of network delays or failures, and the summary results of visual information are simple or fail to accurately convey the situational context to the user.

[0015] Furthermore, technology that goes beyond simple object location or movement information to link high-level information inference—such as relationships between objects, behavioral intentions, tactical significance, and spatial context—with content generation is still lacking.

[0016] Recently, various guidance systems are being utilized in exhibition spaces such as museums, galleries, and art museums to enhance the visitor experience. Audio guide devices are a prime example, and visitors can listen to explanations about exhibits by renting a terminal at the entrance or installing an application on their personal smartphones. However, these conventional audio guide methods have limitations as they are limited to providing fixed audio content for each exhibit, making it difficult to offer personalized guidance services tailored to visitors' interests or movement paths.

[0017] Furthermore, some exhibition halls incorporate Augmented Reality (AR) or Virtual Reality (VR) technologies to visually enhance exhibits or provide location-based guidance in conjunction with mobile apps. However, since most of these technologies rely on central servers or cloud-based computing, it is difficult to guarantee seamless service in environments with network latency or congestion. In particular, visitors requiring detailed, personalized guidance—such as the visually impaired, children, and the elderly—demand features that go beyond simple exhibit descriptions, including real-time route guidance, companion management, and situational interactive assistance; however, existing technologies struggle to meet these demands.

[0018] Recently, with the advancement of AI-based voice assistants and multilingual translation systems, multilingual guidance and Q&A functions are becoming available in exhibition spaces. However, since most services operate on a single terminal or rely on a centralized server to process multiple users simultaneously, there are limitations in terms of personalized customization and the ability to reflect real-time situations within the space.

[0019] Therefore, there is a demand for a new type of personalized AI guide system capable of recognizing visitors' locations, movement patterns, surrounding congestion, and areas of interest in real time within spaces such as museums and exhibition halls, and providing customized voice guidance and Q&A services to individual visitors based on this information. In particular, such a system needs to feature a distributed structure in which edge AI devices for spatial information analysis distributed across the exhibition infrastructure cooperate with personal edge AI devices carried or worn by visitors. This enables multilingual, customized, and real-time guidance while minimizing issues such as network latency and server load.

[0020] This invention aims to process large-scale data in real time by resolving the performance degradation caused by existing centralized data processing for analyzing the movement paths and dwell times of specific individuals in specific areas of large-scale complex facilities.

[0021] The present invention aims to process large-scale data in real time by resolving the performance degradation caused by existing centralized data processing for analyzing the movement paths and dwell times of specific individuals within a store.

[0022] Since existing systems process data in a manner that identifies the identity of an object, which can lead to legal issues regarding data collection in public places, this invention aims to protect the personal information of an object during object tracking.

[0023] The present invention relates to a security system capable of continuously tracking an object whose identity has already been verified by utilizing object recognition and Edge AI technologies. It provides a technology that enables multiple devices within a security facility to communicate to track the movement path of an object and to detect illegal intrusions or abnormal behavior in real time to respond quickly.

[0024] The present invention aims to overcome the limitations of existing partial or manual-based identification methods by providing a system capable of tracking and managing the entire process of a vehicle from entry to disembarkation for boarding a ship in real time.

[0025] The present invention aims to provide a technology capable of systematically managing vehicle flow within an entire dock and ship operation environment by deploying a plurality of edge AI devices by zone and distributing and collecting vehicle identification information.

[0026] The present invention aims to implement a user / manager guidance information provision system that provides guidance information regarding remaining waiting time, boarding availability, ship location and status, etc., to users waiting to board, and enables managers to intuitively understand information such as boarding permission range, reservation status, and loading rate through a manager terminal.

[0027] The present invention aims to provide a vehicle management system capable of reducing network load and enabling edge-based real-time operation by ensuring that not all devices communicate directly with the server, but rather that information is aggregated from the node AI device to the head AI device and only the head AI device communicates with the server.

[0028] The present invention aims to provide a technology that effectively integrates multi-angle image information acquired through a plurality of edge AI devices and precisely analyzes the location, movement, and interaction of objects based thereon.

[0029] The present invention aims to provide a system capable of deriving high-level meaning by analyzing not only the individual actions of an object but also the tactical context and interrelationships between actions in real time.

[0030] The present invention aims to provide a technology that converts behavioral information analyzed in real time into voice relay content and effectively delivers it to users who have difficulty accessing visual information.

[0031] The present invention aims to provide a system that improves processing efficiency and stability by designing a distributed design for the functions of a head AI device and a node AI device so that it can operate stably even in the event of network delays or server failures.

[0032] The present invention aims to provide a behavior analysis framework capable of quantitatively inferring the intention, importance, and priority of an action, going beyond simple location information or object identification information.

[0033] The present invention aims to provide a technology capable of flexibly generating and transmitting voice content to be provided to a user terminal based on video-based behavioral analysis results, depending on the on-site or online broadcasting environment.

[0034] The present invention aims to solve the problem that existing audio guide systems provided within exhibition halls fail to reflect the individual interests and movement paths of visitors.

[0035] The present invention aims to solve the problem that central server-dependent guidance systems are vulnerable to network delays and congestion, making it difficult to provide real-time customized guidance.

[0036] The present invention aims to solve the problem of providing only uniform guidance of the same form to various visitors, such as the visually impaired, children, and the elderly, and to provide customized voice guidance for each target audience.

[0037] The present invention aims to solve the problem of difficulty in providing individually customized responses when a large number of visitors in an exhibition hall receive services simultaneously.

[0038] The present invention aims to solve the problem of difficulty in preventing companions from getting separated or children from getting lost during family or group viewings.

[0039] The present invention relates to an edge AI device for object tracking and collecting spatial dwell information per object, comprising: a sensor unit for sensing an object in a designated area; a communication unit for communicating with another edge AI device; a memory unit; and a processor unit; wherein the processor unit analyzes an object sensed by the sensor unit to output object analysis information, stores movement path information and dwell time information of the analyzed object in the memory unit, and is configured to create spatial dwell information per object based on the output object analysis information, the stored movement path information, and the stored dwell time information.

[0040] In addition, the processor unit is configured to analyze the object sensed by the sensor unit in real time and output object analysis information.

[0041] In addition, the object analysis information is information output based on at least one of the size information of the object, the shape information of the object, or the color information of the object.

[0042] In addition, the object analysis information is information composed of at least one of the age information of the object, the gender information of the object, or the race information of the object.

[0043] Additionally, the processor unit is configured to transmit at least one of the outputted object analysis information, the stored movement path information, the stored dwell time information, and the created object-specific spatial dwell information to another edge AI device via the communication unit when the object moves and the sensor unit can no longer sense the object in the designated area.

[0044] The present invention relates to an object tracking and spatial information collection system, wherein the system comprises a first edge AI device and a second edge AI device, and the first edge AI device comprises: a sensor unit for sensing an object in a designated area; a communication unit for communicating with the second edge AI device; a memory unit; and a processor unit; wherein the processor unit analyzes an object sensed by the sensor unit to output object analysis information, stores movement path information and dwell time information of the analyzed object in the memory unit, creates spatial dwell information for each object based on the output object analysis information, the stored movement path information, and the stored dwell time information, and is configured to transmit at least one of the output object analysis information, the stored movement path information, the stored dwell time information, and the created spatial information to the second edge AI device through the communication unit.

[0045] In addition, the processor unit is configured to analyze the object sensed by the sensor unit in real time and output object analysis information.

[0046] In addition, the object analysis information is information output based on at least one of the size information of the object, the external shape information of the object, or the color information of the object.

[0047] In addition, the object analysis information is information composed of at least one of the age information of the object, the gender information of the object, or the race information of the object.

[0048] Additionally, the processor unit is configured to transmit at least one of the generated object analysis information, the stored movement path information, the stored dwell time information, and the generated spatial information to the second edge AI device through the communication unit when the object moves and the sensor unit can no longer sense the object in the designated area.

[0049] The present invention relates to an edge AI device for object tracking and collecting spatial dwell information per object, comprising: at least one sensor unit for sensing an object in a designated area; a communication unit for communicating with a central server; a memory unit; and a processor unit; wherein the processor unit analyzes an object sensed by the at least one sensor unit and outputs object analysis information, stores movement path information and dwell time information of the analyzed object in the memory unit, creates spatial dwell information per object based on the output object analysis information, the stored movement path information and the stored dwell time information, and transmits the created spatial dwell information per object to a central server through the communication unit.

[0050] The present invention relates to an object tracking and object-specific spatial dwell information collection system, wherein the system comprises an edge AI device, and the edge AI device comprises: at least one sensor unit for sensing an object in a designated area; a communication unit for communicating with a central server; a memory unit; and a processor unit; wherein the processor unit analyzes an object sensed by the at least one sensor unit and outputs object analysis information, stores movement path information and dwell time information of the analyzed object in the memory unit, creates object-specific spatial dwell information based on the outputted object analysis information, the stored movement path information, and the stored dwell time information, and is configured to transmit at least one of the outputted object analysis information, the stored movement path information, the stored dwell time information, and the created spatial information to the central server through the communication unit.

[0051] The present invention relates to an object tracking and object-specific spatial dwell information collection system, wherein the system comprises a first edge AI device and a second edge AI device, and the first edge AI device comprises: at least one sensor unit for sensing an object in a designated area; a communication unit for communicating with the second edge AI device and a central server; a memory unit; and a processor unit; wherein the communication unit receives ID information regarding an object that has entered the space where the first edge AI device is located from the second edge AI device, and the processor unit analyzes the object sensed by the at least one sensor unit and outputs object analysis information, verifies whether the outputted object analysis information matches the received ID information, stores movement path information and dwell time information of the analyzed object in the memory unit, creates object-specific spatial dwell information based on the outputted object analysis information, the stored movement path information, and the stored dwell time information, and is configured to transmit at least one of the outputted object analysis information, the stored movement path information, the stored dwell time information, and the created spatial information to the central server through the communication unit.

[0052] In addition, the received ID information is at least one of the following: object analysis information for the object output by the second edge AI device, movement path information of the object stored in the second edge AI device, dwell time information of the object stored in the second edge AI device, or object-specific spatial dwell information of the object created by the second edge AI device.

[0053] The present invention relates to an edge AI device for object tracking and collecting spatial dwell information per object, comprising: a sensor unit for sensing an object in a designated area; a communication unit for communicating with another edge AI device or another terminal; a memory unit for storing unique information necessary for identifying the object; and a processor unit; wherein the processor unit is configured to match the unique information stored in the memory unit to an object sensed by the sensor unit, store movement path information and dwell time information of the matched object in the memory unit, and create spatial dwell information of the sensed object based on the unique information necessary for identifying the matched object, the stored movement path information, and the stored dwell time information.

[0054] In addition, the processor unit is configured to transmit information regarding external access to the other terminal or the external server through the communication unit when the unique information stored in the memory unit does not match the object sensed by the sensor unit.

[0055] Additionally, the processor unit is configured to transmit at least one of the unique information required for the identification of the matched object, the stored movement path information, the stored dwell time information, and the spatial dwell information of the created object to the other edge AI device through the communication unit when the matched object moves and the sensor unit can no longer sense the matched object in the designated area.

[0056] Additionally, the sensor unit senses the face of an object in the designated area, and the processor unit is configured to match the unique information stored in the memory unit with the sensed object based on the face of the object sensed by the sensor unit.

[0057] Additionally, the sensor unit senses the iris of an object in the designated area, and the processor unit is configured to match the unique information stored in the memory unit with the sensed object based on the iris of the object sensed by the sensor unit.

[0058] Additionally, the memory unit stores unique information necessary for identifying the object and access restriction information related to the unique information, and the processor unit is configured to determine whether the stored movement path information and dwell time information are included in the stored access restriction information.

[0059] In addition, the processor unit is configured to transmit an access restriction notification signal to the other terminal when it is determined that the stored movement path information and dwell time information are included in the stored access restriction information.

[0060] The present invention relates to an object tracking and spatial facility dwelling information collection system for said object, wherein the system comprises a first edge AI device and a second edge AI device, and the first edge AI device comprises: a sensor unit for sensing an object in a designated area; a communication unit for communicating with the second edge AI device or the spatial facility; a memory unit for storing unique information necessary for identifying said object; and a processor unit; wherein the processor unit is configured to match the unique information stored in the memory unit with an object sensed by the sensor unit, store movement path information and dwelling time information of said matched object in the memory unit, create spatial dwelling information of said sensed object based on the unique information necessary for identifying said matched object, said stored movement path information, and said stored dwelling time information, and transmit at least one of the unique information necessary for identifying said matched object, said stored movement path information, said stored dwelling time information, and said created spatial dwelling information to the second edge AI device through the communication unit.

[0061] In addition, the processor unit is configured to transmit information regarding external access to the other terminal or the external server through the communication unit when the unique information stored in the memory unit does not match the object sensed by the sensor unit.

[0062] Additionally, the processor unit is configured to transmit at least one of the unique information required for the identification of the matched object, the stored movement path information, the stored dwell time information, and the spatial dwell information of the created object to the other edge AI device through the communication unit when the matched object moves and the sensor unit can no longer sense the matched object in the designated area.

[0063] Additionally, the sensor unit senses the face of an object in the designated area, and the processor unit is configured to match the unique information stored in the memory unit with the sensed object based on the face of the object sensed by the sensor unit.

[0064] Additionally, the sensor unit senses the iris of an object in the designated area, and the processor unit is configured to match the unique information stored in the memory unit with the sensed object based on the iris of the object sensed by the sensor unit.

[0065] Additionally, the memory unit stores unique information necessary for identifying the object and access restriction information related to the unique information, and the processor unit is configured to determine whether the stored movement path information and dwell time information are included in the stored access restriction information.

[0066] In addition, the processor unit is configured to transmit an access restriction notification signal to the other terminal when it is determined that the stored movement path information and dwell time information are included in the stored access restriction information.

[0067] The present invention relates to an edge AI device for object tracking and collecting spatial dwell information per object, comprising: a sensor unit for sensing an object in a designated area; a communication unit for communicating with another edge AI device or another terminal; a memory unit for storing unique information necessary for identifying the object and access restriction information related to the unique information; and a processor unit; wherein the processor unit matches the unique information stored in the memory unit to an object sensed by the sensor unit, stores movement path information and dwell time information of the matched object in the memory unit, determines whether the stored movement path information and dwell time information are included in the stored access restriction information, and, if it is determined that the stored movement path information and dwell time information are included in the stored access restriction information, transmits an access restriction notification signal to the other terminal.

[0068] Additionally, the processor unit is configured to transmit at least one of the unique information required for the identification of the matched object, the stored movement path information, the stored dwell time information, and the spatial dwell information of the created object to the other edge AI device through the communication unit when the matched object moves and the sensor unit can no longer sense the matched object in the designated area.

[0069] Additionally, the sensor unit senses the face of an object in the designated area, and the processor unit is configured to match the unique information stored in the memory unit with the sensed object based on the face of the object sensed by the sensor unit.

[0070] Additionally, the sensor unit senses the iris of an object in the designated area, and the processor unit is configured to match the unique information stored in the memory unit with the sensed object based on the iris of the object sensed by the sensor unit.

[0071] In addition, in an object tracking and object-specific spatial dwell information collection system, the system comprises a first edge AI device and a second edge AI device, wherein the first edge AI device comprises: a sensor unit for sensing an object in a designated area; a communication unit for communicating with the second edge AI device; a memory unit for storing unique information required for the identification of the object and access restriction information related to the unique information; and a processor unit; wherein the processor unit matches the unique information stored in the memory unit to the object sensed by the sensor unit, stores movement path information and dwell time information of the matched object in the memory unit, determines whether the stored movement path information and dwell time information are included in the stored access restriction information, and if it is determined that the stored movement path information and dwell time information are included in the stored access restriction information, transmits an access restriction notification signal to the other terminal, and is configured to transmit at least one of the unique information required for the identification of the matched object, the stored movement path information, the stored dwell time information, and the created spatial dwell information to the second edge AI device through the communication unit.

[0072] Additionally, the processor unit is configured to transmit at least one of the unique information required for the identification of the matched object, the stored movement path information, the stored dwell time information, and the spatial dwell information of the created object to the other edge AI device through the communication unit when the matched object moves and the sensor unit can no longer sense the matched object in the designated area.

[0073] Additionally, the sensor unit senses the face of an object in the designated area, and the processor unit is configured to match the unique information stored in the memory unit with the sensed object based on the face of the object sensed by the sensor unit.

[0074] Additionally, the sensor unit senses the iris of an object in the designated area, and the processor unit is configured to match the unique information stored in the memory unit with the sensed object based on the iris of the object sensed by the sensor unit.

[0075] The present invention relates to a vehicle management system for identifying and tracking a vehicle on board a ship, comprising: an edge AI device positioned at least one of a dock entrance, a ship boarding point, a ship disembarking point, and a dock exit; a server device; and a user terminal; wherein the edge AI device is configured to generate vehicle identification information for the vehicle and transmit the generated vehicle identification information to the server device, and the server device is configured to provide user guidance information or administrator guidance information to the user terminal based on the vehicle identification information received from the edge AI device.

[0076] Additionally, the vehicle identification information includes at least one of the following: vehicle number information for the vehicle, vehicle type information for the vehicle, total vehicle length information for the vehicle, entry time information for the vehicle, reservation status information for the vehicle, and passenger count information for the vessel.

[0077] Additionally, the user guidance information includes at least one of information on whether the vehicle is boarded, information on the remaining waiting time expected until the vehicle can actually board the vessel, information on the time until the vessel's arrival, and information on the real-time loading status on the vessel.

[0078] In addition, the above-mentioned administrator guidance information includes at least one of the following: information on the reservation and reception status of the vehicle, information on vehicles waiting at the pier, information on the range of vehicles permitted to board the vessel, information on the loading rate of the vessel, information on the remaining space of the vessel, information on the list of vehicles eligible for priority boarding on the vessel, information on the arrival time of the vessel, and information on the departure time of the vessel.

[0079] Additionally, the edge AI device comprises a plurality of node AI devices and a head AI device communicating with the node AI devices, wherein the node AI device generates sensing data of the vehicle, and the head AI device is configured to receive the generated sensing data and generate vehicle identification information.

[0080] In addition, the edge AI device is configured to assign serial number information to the vehicle based on the generated vehicle identification information.

[0081] In addition, the above serial number information is configured to have different identification symbols or different number systems depending on whether the vehicle is reserved.

[0082] Additionally, the user terminal includes an output device implemented in the form of a mobile application, an in-vehicle display, or an electronic display board, and the user guidance information is configured to be visually displayed through the output device.

[0083] In addition, the priority boarding target vehicles included in the list information of priority boarding target vehicles for the said vessel are configured to be allowed to board the vessel even if they are not included in the range of the boarding permission vehicle range information.

[0084] The present invention relates to a vehicle management system for identifying and tracking a vehicle boarding a ship, comprising: a first edge AI device positioned at the entrance of a dock; a second edge AI device positioned at the exit of a dock; a third edge AI device positioned at a ship boarding point; and a fourth edge AI device positioned at a ship disembarking point. and a server device capable of communicating with at least some of the edge AI devices; wherein the first edge AI device is configured to generate vehicle identification information of a vehicle entering the dock and to assign serial number information to the vehicle based on the vehicle identification information; the second edge AI device is configured to verify the vehicle identification information and serial number information of a vehicle moving from the dock toward the exit of the dock; the third edge AI device is configured to verify the vehicle identification information and serial number information of a vehicle intending to board the vessel; the fourth edge AI device is configured to verify the vehicle identification information and serial number information of a vehicle disembarking from the vessel; and the server device is configured to track the movement flow of the vehicle based on the vehicle identification information and serial number information of the vehicle received from at least one of the first edge AI device to the fourth edge AI device.

[0085] Additionally, it further includes a user terminal configured to communicate with the server device; wherein the server device is configured to provide user guidance information or administrator guidance information to the user terminal based on the received vehicle identification information of the vehicle and the serial number information.

[0086] Additionally, the vehicle identification information includes at least one of the following: vehicle number information for the vehicle, vehicle type information for the vehicle, total vehicle length information for the vehicle, entry time information for the vehicle, reservation status information for the vehicle, and passenger count information for the vessel.

[0087] Additionally, the user guidance information includes at least one of information on whether the vehicle is boarded, information on the remaining waiting time expected until the vehicle can actually board the vessel, information on the time until the vessel's arrival, and information on the real-time loading status on the vessel.

[0088] In addition, the above-mentioned administrator guidance information includes at least one of the following: information on the reservation and reception status of the vehicle, information on vehicles waiting at the pier, information on the range of vehicles permitted to board the vessel, information on the loading rate of the vessel, information on the remaining space of the vessel, information on the list of vehicles eligible for priority boarding on the vessel, information on the arrival time of the vessel, and information on the departure time of the vessel.

[0089] In addition, at least one of the first edge AI device to the fourth edge AI device comprises a plurality of node AI devices and a head AI device communicating with the node AI devices, wherein the node AI device generates sensing data of the vehicle, and the head AI device is configured to receive the generated sensing data and generate vehicle identification information.

[0090] In addition, at least one of the first edge AI device to the fourth edge AI device is configured to assign serial number information to the vehicle based on the generated vehicle identification information.

[0091] Additionally, the user terminal includes an output device implemented in the form of a mobile application, an in-vehicle display, or an electronic display board, and the user guidance information is configured to be visually displayed through the output device.

[0092] In addition, the priority boarding target vehicles included in the list information of priority boarding target vehicles for the said vessel are configured to be allowed to board the vessel even if they are not included in the range of the boarding permission vehicle range information.

[0093] The present invention relates to an edge AI device located in a stadium that performs object-based voice relay, comprising: a memory unit; one or more sensor units for sensing the object; and a processor unit; wherein the one or more sensor units are each positioned at different locations within the stadium to acquire one or more image data, and the processor unit is configured to: analyze the image data to generate primary behavior analysis result data based on primary behavior analysis input data of the object, generate text data for the object based on the generated primary behavior analysis result data, and generate voice relay data based on the generated text data.

[0094] In addition, the primary behavior analysis input data of the object includes at least one of the spatial coordinate data of the object, the movement data of the object, the relative position data of the object, the interrelationship data of the object within the stadium, the pattern data of the object, and the context data of the object.

[0095] Additionally, the processor unit is configured to generate secondary behavior analysis result data for the object based on the generated primary behavior analysis result data, and the secondary behavior analysis result data is at least one of the tactical situation classification data of the object, the behavioral purpose and intention estimation data of the object, the behavioral importance and priority judgment data of the object, and the content extraction element data of the object.

[0096] In addition, the processor unit is configured to generate the text data to follow a sentence structure corresponding to the action of the object, and is configured so that the generated text data is matched with a template of a pre-secured voice sample library stored in the memory unit.

[0097] In addition, the generated voice relay data includes at least one meta-information among intonation information, speed information, and emotion expression information of the voice corresponding to the text data.

[0098] In addition, the memory unit is configured to store the generated primary behavior analysis result data and the voice relay data.

[0099] The present invention relates to an object-based voice relay system for performing object-based voice relay located in a stadium, comprising: an edge AI device; and a server device; wherein the edge AI device comprises: a communication unit configured to communicate with the server device; a memory unit; one or more sensor units for sensing the object; and a processor unit; wherein the one or more sensor units are each disposed at different locations within the stadium to acquire one or more image data, and the processor unit is configured to analyze the image data to generate primary behavior analysis result data based on primary behavior analysis input data of the object, generate text data for the object based on the generated primary behavior analysis result data, and generate voice relay data based on the generated text data.

[0100] Additionally, the processor unit is configured to generate secondary behavior analysis result data for the object based on the generated primary behavior analysis result data, and the secondary behavior analysis result data is at least one of the tactical situation classification data of the object, the behavioral purpose and intention estimation data of the object, the behavioral importance and priority judgment data of the object, and the content extraction element data of the object.

[0101] Additionally, the communication unit is configured to transmit the generated primary behavior analysis result data to the server device, and the server device is configured to generate secondary behavior analysis result data based on the primary behavior analysis result data received from the communication unit, and the secondary behavior analysis result data is at least one of the tactical situation classification data of the object, the behavior purpose and intention estimation data of the object, the behavior importance and priority determination data of the object, and the content extraction element data of the object.

[0102] In addition, the server device is configured to store primary behavior analysis result data received from the communication unit and secondary behavior analysis result data generated, and is configured to perform data analysis based on the stored data.

[0103] In addition, the server device is configured to perform at least one data analysis among team power analysis quantifying the strategic movements and power level of a specific team, player stat evaluation based on the behavioral characteristics of a specific player, and referee evaluation based on the referee's officiating tendencies.

[0104] The present invention relates to an object-based voice relay system for performing object-based voice relay located in a stadium, comprising: an edge AI device; and a user terminal; wherein the edge AI device comprises: a communication unit configured to communicate with the user terminal; a memory unit; one or more sensor units for sensing the object; and a processor unit; wherein the one or more sensor units are each disposed at different locations within the stadium to acquire one or more image data, and the processor unit is configured to analyze the image data to generate primary behavior analysis result data based on primary behavior analysis input data of the object, generate text data for the object based on the generated primary behavior analysis result data, and generate voice relay data based on the generated text data.

[0105] In addition, the processor unit is configured to transmit the generated voice relay data to the user terminal in real time.

[0106] In addition, the primary behavior analysis input data of the object includes at least one of the spatial coordinate data of the object, the movement data of the object, the relative position data of the object, the interrelationship data of the object within the stadium, the pattern data of the object, and the context data of the object.

[0107] Additionally, the processor unit is configured to generate secondary behavior analysis result data for the object based on the generated primary behavior analysis result data, and the secondary behavior analysis result data is at least one of the tactical situation classification data of the object, the behavioral purpose and intention estimation data of the object, the behavioral importance and priority judgment data of the object, and the content extraction element data of the object.

[0108] According to the present invention, an edge AI device can be installed in various locations within a large complex facility to provide a system that continuously tracks objects without identifying their identity.

[0109] According to the present invention, an edge AI device including multiple sensors is installed in a store, and a system can be provided that continuously tracks objects without identifying their identity.

[0110] According to the present invention, the same object can be continuously tracked without identifying the object's identity, thereby enhancing the protection of personal information.

[0111] According to the present invention, distributed processing using an edge AI device enables real-time data processing locally, resulting in reduced latency and reduced network bandwidth.

[0112] According to the present invention, by accumulating information on the movement path of an object and the time spent in a specific area, there is an effect of providing detailed pattern analysis such as specific time zones, age groups, or genders.

[0113] According to the present invention, an edge AI device can be installed in a security-related location such as a research institute to identify the identity of an object and then provide a system for continuously tracking the identified object.

[0114] According to the present invention, distributed processing using an edge AI device enables real-time data processing locally, resulting in reduced latency and reduced network bandwidth.

[0115] According to the present invention, by accumulating information on the movement path and dwell time in a specific area of ​​a specific object, there is an effect of providing a detailed pattern analysis of the specific object.

[0116] The present invention provides the effect of enabling real-time vehicle flow control while minimizing the communication burden with server devices by generating and processing vehicle identification information through edge AI devices deployed in each zone.

[0117] The present invention provides the effect of ensuring communication stability by hierarchically configuring a node AI device and a head AI device, and allowing only the head AI device to communicate with the server.

[0118] The present invention provides the effect of resolving user waiting inconvenience and improving user satisfaction by providing various information, such as boarding availability, waiting time, ship location, and status, in real time through a user terminal.

[0119] The present invention provides the effect of enhancing the manager's on-site responsiveness and the precision of operational judgment by providing the status of reservations and registrations, the status of vehicles waiting at the site, the loading rate of vessels, and a list of priority boarding vehicles through a manager terminal.

[0120] The present invention provides a vehicle management and operational effect that can flexibly respond to on-site situations by first exceptionally designating vehicles eligible for boarding and allowing boarding on ships.

[0121] The present invention provides the effect of analyzing the location, movement path, interaction, etc. of an object more precisely and quickly by integrating and processing multi-angle image data acquired from a plurality of node AI devices in a head AI device.

[0122] The present invention provides the effect of providing context-based voice content suitable for real-time situations by inferring the tactical meaning, intention, priority, etc., of analyzed object behaviors at a high level on the server.

[0123] The present invention provides the effect of efficiently distributing the computational burden and improving real-time performance and system stability by adopting a distributed processing structure that performs primary analysis at the edge and secondary analysis on the server.

[0124] The present invention provides the effect of enabling even users with difficulty recognizing visual information to intuitively understand the situation by transmitting voice relay data automatically generated based on analyzed results to a user terminal.

[0125] The present invention provides the effect of being able to perform precise behavioral analysis by capturing even the coordination or confrontation situation of multiple objects based on relative position and interaction information between objects.

[0126] The present invention provides the effect of being able to flexibly respond to various service environments, as the generated voice relay content can be selectively applied using a field-centered direct delivery method or an external relay server linkage method.

[0127] The present invention has the effect of providing an individually customized voice guidance service by analyzing the location, interests, and movement patterns of visitors within an exhibition hall in real time.

[0128] The present invention has the effect of minimizing network latency and providing stable guidance services even when multiple users use them simultaneously, by having edge AI devices distributed across exhibition infrastructure cooperate with personal edge AI devices.

[0129] The present invention has the effect of significantly improving the convenience of special visitors by providing guidance modes tailored to the characteristics of various users, such as the visually impaired, children, and the elderly.

[0130] The present invention has the effect of preventing children from getting lost and ensuring safety by detecting and notifying whether companions have separated during group or family viewing.

[0131] The present invention has the effect of enhancing visitor immersion and exhibition satisfaction by providing real-time voice guidance in multiple languages, as well as recommending exhibits and providing Q&A based on personal interests.

[0132] FIG. 1 is a diagram showing an example related to communication between an edge AI device and a plurality of edge AI devices according to an embodiment of the present invention.

[0133] FIG. 2 is a diagram showing an example of an edge AI device tracking an object within a large complex facility according to the present invention.

[0134] FIG. 3 is a diagram showing an example of tracking the movement path of an object within a large complex facility according to one embodiment of the present invention.

[0135] FIG. 4 is a diagram showing an example of tracking the movement path of an object within a store according to one embodiment of the present invention.

[0136] FIG. 5 is a diagram illustrating communication and object tracking between a plurality of edge AI devices according to an embodiment of the present invention.

[0137] FIG. 6 is a diagram illustrating an embodiment in which a processor unit according to the present invention cannot match an object sensed by a sensor unit with unique information stored in a memory unit.

[0138] FIG. 7 is a diagram illustrating an embodiment in which an access authorization authentication device according to the present invention communicates with an edge AI device.

[0139] FIG. 8a is a schematic diagram showing the communication structure and role sharing relationship of a head AI device (500) and a node AI device (500′) according to one embodiment of the present invention.

[0140] FIG. 8b is a drawing for explaining user guidance information and administrator guidance information according to an embodiment of the present invention.

[0141] FIG. 8c is a diagram illustrating the communication structure between the control server (410) and a plurality of head AI devices.

[0142] FIG. 9 is a drawing for explaining vehicle identification information and serial number information according to an embodiment of the present invention.

[0143] FIG. 10 is a diagram illustrating the arrangement structure and information flow of an edge AI device during the process of boarding and disembarking a vehicle from a ship according to one embodiment of the present invention.

[0144] FIG. 11 is a drawing illustrating an example of an application user interface (UI) of a user terminal (300) according to one embodiment of the present invention.

[0145] FIG. 12 is a drawing illustrating an example of a user interface (UI) of a manager terminal (320) according to one embodiment of the present invention.

[0146] FIG. 13 is a drawing showing an example of implementation of a user terminal (300) in the form of an electronic display board according to one embodiment of the present invention.

[0147] FIG. 14 is a drawing illustrating an example of a user terminal (300) that provides terminal waiting information through an upper electronic display board of a dock entry gate according to one embodiment of the present invention.

[0148] FIG. 15 is a diagram showing the arrangement and functional relationship of a head AI device (500) installed at a vehicle entry point (Input 1) at the entrance of a dock and a plurality of node AI devices (500') connected thereto, as an example of installation of a first edge AI device (100-1) according to an embodiment of the present invention.

[0149] FIG. 16 is a diagram illustrating a zone-based sensing structure of a node AI device (500') and an information linkage method with a head AI device (500) according to an embodiment of the present invention.

[0150] FIG. 17 is a drawing for illustrating an embodiment in which a plurality of head AI devices within a specific terminal of the present invention are arranged by zone.

[0151] FIG. 18 is a drawing for more specifically explaining the configuration and operation method of an edge AI device (100) that can be utilized in the first or second embodiment of the present invention.

[0152] FIG. 19a is a block diagram illustrating the overall processing structure and data flow of an object-based voice relay system according to one embodiment of the present invention.

[0153] FIG. 19b is a diagram illustrating an integrated video sequence and object-specific continuous tracking data of an object-based voice relay system according to an embodiment of the present invention.

[0154] FIG. 20 is a block diagram showing the detailed configuration of the object primary behavior analysis step (S230) according to an embodiment of the present invention.

[0155] FIG. 21 is a block diagram schematically illustrating the process of the secondary behavior analysis result data generation step (S22-1) according to an embodiment of the present invention.

[0156] FIG. 22a is a specific scene example for explaining the real-time object behavior recognition and relay processing flow according to an embodiment of the present invention, and is a drawing that visually represents an actual situation occurring in a soccer stadium.

[0157] FIG. 22b is a table-formatted drawing summarizing the specific configuration and example values ​​of spatial coordinates and movement data (3-1-1) utilized in the object primary behavior analysis step (S230) performed by the head AI device (500) of the present invention, based on the game scene (player number 11's ball stealing and breakthrough situation) described in FIG. 22a.

[0158] FIG. 22c is a table showing a specific configuration example of relative position and interrelationship data (3-1-2) used in the object primary behavior analysis step (S230).

[0159] FIG. 22d is a diagram summarizing the process of identifying the unique behavior style and repetitive movement patterns of a specific object, player H11, through specific items and example values ​​of the object pattern data (3-1-3) utilized in the present invention.

[0160] FIG. 22e is a diagram showing specific items and example analysis contents of context data (3-1-4) utilized by the head AI device (500) of the present invention in performing the object primary behavior analysis step (S230).

[0161] FIG. 23a is a diagram illustrating primary behavior analysis result data (3-2) output by a head AI device (500) using primary behavior analysis input data (3-1) as input according to an embodiment of the present invention.

[0162] FIG. 23b is a diagram showing an example in JSON format in which primary behavior analysis result data (3-2) generated by a head AI device (500) according to one embodiment of the present invention is output in a structured form.

[0163] FIG. 24a is a diagram illustrating an example of the configuration of tactical situation classification data (4-2-1) generated by a server device (400) based on primary behavior analysis result data (3-2) received from a head AI device (500) in a secondary behavior analysis result data generation step (S22-1) according to an embodiment of the present invention.

[0164] FIG. 24b is a diagram illustrating an example of the configuration of behavioral purpose / intention estimation data (4-2-2) generated by a server device (400) based on primary behavioral analysis result data (3-2) received from a head AI device (500) in a secondary behavioral analysis result data generation step (S22-1) according to an embodiment of the present invention.

[0165] FIG. 24c is a diagram illustrating an example of the configuration of behavior importance / priority judgment data (4-2-3) generated by a server device (400) based on the first behavior analysis result data (3-2) received from a head AI device (500) in the second behavior analysis result data generation step (S22-1) according to an embodiment of the present invention.

[0166] FIG. 24d is a diagram illustrating an example of the configuration of content extraction element data (4-2-4) generated by a server device (400) based on primary behavior analysis result data (3-2) received from a head AI device (500) in a secondary behavior analysis result data generation step (S22-1) according to an embodiment of the present invention.

[0167] FIG. 25 is a diagram illustrating voice relay data (19a-1) output by a server device (400) or a head AI device (500) according to an embodiment of the present invention.

[0168] FIG. 26 is a diagram illustrating the schematic layout of a distributed edge AI system for providing personalized AI guide services in indoor spaces such as museums and exhibition halls.

[0169] Specific details of the embodiments are included in the detailed description and drawings.

[0170] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims. Throughout the specification, the same reference numerals refer to the same components.

[0171] FIG. 1 is a diagram showing an example related to communication between an edge AI device and a plurality of edge AI devices according to one embodiment of the present invention.

[0172] As illustrated in FIG. 1(a), the edge AI device (100) according to the present invention includes a sensor unit (110), a communication unit (120), a processor unit (130), and a memory unit (140). Specifically, the sensor unit (110) is the most basic data input device in the edge AI device (100) and may include a visual sensor such as a camera. Additionally, the sensor unit (110) can sense objects (OB) in designated areas (200-1, 200-2) within a large complex facility.

[0173] According to the present invention, the edge AI device (100) may be a terminal device or a server and may include any type of device. The edge AI device (100) may be an electronic device having computational capabilities equipped with a processor and memory, such as a PC (Personal Computer), a laptop computer, a tablet PC, or a mobile phone.

[0174] The edge AI device (100) according to the present invention may include a PC (Personal Computer), a laptop computer, a mobile terminal, a smartphone, a tablet PC, an artificial intelligence (AI) speaker, an artificial intelligence TV, and a wearable device (e.g., a smart watch, a smart glass, a head-mounted display (HMD)), and may include all types of terminal devices capable of connecting to a wired / wireless network. Additionally, the edge device (300) may include any server implemented by at least one of an agent, an API (Application Programming Interface), and a plug-in. Additionally, the edge AI device (100) may include an application source and / or a client application.

[0175] According to one embodiment of the present invention, the sensor unit (110) may include various types of sensors such as a high-resolution camera, an infrared sensor (IR), and an optical sensor, and the sensor unit (110) may sense an object (OB) in a designated area (200-1, 200-2) within a large complex facility in real time.

[0176] According to the present invention, the memory unit (140) can store various programs or data required for the operation of the edge AI device (100). Additionally, an artificial intelligence model can be stored or implemented in the memory unit (140). Specifically, the memory unit (140) can store an artificial intelligence model learned based on field data. Additionally, the memory unit (140) can temporarily or permanently store training data for learning the stored artificial intelligence model. For example, the memory unit (140) can temporarily or permanently store field data for learning the artificial intelligence model. Additionally, the memory unit (140) can store an artificial intelligence model learned based on field data.

[0177] According to the present invention, the processor unit (130) can read a program stored in the memory unit (140) and perform a method according to one embodiment of the present invention. The processor unit (130) can execute one or more instructions to perform a method / operation according to various embodiments of the present invention. According to one embodiment of the present invention, the processor unit (130) can read a computer program stored in the memory unit (140) and perform data processing for additional learning of an artificial intelligence model according to the present invention. The processor unit (130) can perform calculations for learning a neural network, such as processing input data for learning in deep learning, extracting pitch from input data, calculating errors, and updating the weights of the neural network using backpropagation.

[0178] According to the present invention, at least one of the CPU, GPGPU, TPU, and NPU of the processor unit (130) can process the learning of a network function. For example, the CPU and GPGPU can together process the learning of a network function and data classification using the network function. In addition, in one embodiment of the present invention, the processor units of a plurality of user terminals can be used together to process the learning of a network function and / or data classification using the network function. Furthermore, a computer program executed on a user terminal according to one embodiment of the present invention may be a program executable on a CPU, GPGPU, TPU, or NPU.

[0179] According to the present invention, the communication unit (120) enables wireless communication between an edge AI device (100) and another edge AI device (100), and between an edge AI device (100) and a server device (400). Specifically, the communication unit (120) can transmit processing results by the processor unit (130) to the server device (400) or another edge AI device (100) via a network. Additionally, the communication unit (120) can receive information transmitted by another edge AI device (100) or the server device (400).

[0180] According to the present invention, the processor unit (130) can perform data learning or inference for providing a service according to the present invention based on an artificial intelligence model stored or implemented in the memory unit (140). Specifically, the processor unit (130) can read a computer program stored in the memory unit (140) and perform data processing for learning the artificial intelligence according to the present invention. The processor unit (130) can perform operations for learning the neural network.

[0181] The edge AI device (100) according to the present invention includes a processor unit (130). Specifically, the processor unit (130) can analyze data sensed by the sensor unit (110) in real time and determine the identity of an object (OB). Real-time object recognition and tracking can be performed by performing distributed processing within each edge AI device (100). For example, the processor unit (130) can analyze the visual characteristics of an object and, based thereon, continuously determine that the object (OB) being tracked is the same object (OB).

[0182] Accordingly, the processor unit (130) can maintain the identity of the object (OB) even without identifying the identity of the object (OB). For example, if the data sensed by the sensor unit (110) is data about a tall, slim woman wearing a white dress, the processor unit (130) can output object analysis information based on the size information of the object (OB) that the object (OB) is tall, the appearance information of the object (OB) that the object (OB) is slim, or the color information of the object (OB) that the object (OB) is wearing white clothes. Accordingly, the processor unit (130) can maintain the identity of the object (OB) based on the size information of the object (OB) that the object (OB) is tall, the appearance information of the object (OB) that the object (OB) is slim, or the color information of the object (OB) that the object (OB) is wearing white clothes, even without identifying the identity of the object (OB).

[0183] According to one embodiment of the present invention, the processor unit (130) can store information on the movement path of an object (OB) and information on the time spent in a specific area in the memory unit (140). For example, the processor unit (130) can maintain the identity of the object (OB) based on size information of the object (OB) such that the object (OB) is tall, appearance information of the object (OB) such that the object (OB) is slim, or color information of the object (OB) such that the object (OB) is wearing white clothes. In the case where the object (OB) stayed at store 1 for 10 minutes and moved to store 3 and stayed at store 3 for 30 minutes, the processor unit (130) can store information on the time spent in the memory unit (140) such that the object (OB) stayed at store 1 for 10 minutes and stayed at store 3 for 30 minutes. Additionally, the processor unit (130) can store information on the movement path of the object (OB) such that it moved from store 1 to store 3 in the memory unit (140).

[0184] The edge AI device (100) of the present invention is used to identify and track information of a vehicle (10) in a ship boarding / disembarking environment and may include a sensor unit (110), a communication unit (120), a processor unit (130), and a memory unit (140). Specifically, the sensor unit (110) may include one or more sensors to detect the approach of the vehicle (10) and to obtain various identification information such as a vehicle number, vehicle type, vehicle length, number of people, etc. For example, the sensor unit (110) may include a camera, a lidar, or an infrared sensor. That is, the sensor unit (110) may perform the role of enabling the edge AI device (100) to recognize the vehicle (10) and collect data related to the vehicle (10).

[0185] According to one embodiment of the present invention, the communication unit (120) can transmit and receive data between the edge AI device (100) and another edge AI device (100) or a server device (400) via a wireless or wired method. Hereinafter, an embodiment is described in which the edge AI device (100) communicates with the server device (400) when the edge AI device (100) corresponds to the head AI device (500). Through this, the communication unit (120) can share or update vehicle identification information (30) with another edge AI device (100) or a server (20).

[0186] According to one embodiment of the present invention, the processor unit (130) performs operations such as vehicle identification, serial number assignment, and determination of boarding status based on data collected from the sensor unit (110), and has an edge-based AI inference engine built in to enable fast determination and classification. In addition, the processor unit (130) can process the received data in real time and transmit the results to the server device (400) through the communication unit (120).

[0187] According to one embodiment of the present invention, the memory unit (140) is a storage device for temporarily or permanently storing data such as vehicle information, serial number, time information, and reservation status, and stores operational data required by the processor unit (130), and may also store learned AI model parameters. In addition, the memory unit (140) may be utilized to support advanced functions such as checking whether a vehicle has re-entered or making a history-based judgment in the future.

[0188] As illustrated in FIG. 1(b), the edge AI device (100) of the present invention can operate as a single independent device, and multiple edge AI devices (100-1, 100-2, 100-3, 100-4) can operate cooperatively by linking with each other through a network. Each edge AI device can acquire and process data individually, but through mutual information sharing and collaborative analysis, more precise object behavior determination becomes possible. This structure enables effective response to situations requiring multi-angle viewpoints and complex object behavior analysis in a wide area, such as a sports stadium, for example.

[0189] In the present invention, based on a communication structure between such edge AI devices (100-1, 100-2, 100-3, 100-4), a series of technical processes can be implemented to integrate data from multiple viewpoints obtained from each edge AI device (100-1, 100-2, 100-3, 100-4), analyze information on the location, movement, and interaction of an object, and then generate behavior recognition and meaning-based voice relay content based on this.

[0190] FIG. 1(b) is a diagram illustrating a structure of interlocking between a plurality of edge AI devices according to an embodiment of the present invention, showing that each edge AI device is positioned at a different location within a dock and a ship and performs a unique role. In particular, the first edge AI device (100-1) may be positioned at the entrance of the dock, the second edge AI device (100-2) at the exit of the dock, the third edge AI device (100-3) at the entrance of the ship, and the fourth edge AI device (100-4) at the exit of the ship.

[0191] According to one embodiment of the present invention, a first edge AI device (100-1) is positioned at the entrance of a dock and performs the role of initially recognizing and registering a vehicle (10) entering the dock. Specifically, the sensor unit (110-1) of the first edge AI device (100-1) acquires vehicle identification information (30) in real time, such as the vehicle number, vehicle type, vehicle length, entry time, number of people, and reservation status of the entering vehicle, and the processor unit (130-1) assigns a unique serial number to each vehicle based on the vehicle identification information (30). Subsequently, the communication unit (120-1) can generate initial data for tracking the subsequent path of the vehicle (10) by sharing the vehicle identification information (30) and serial number information (40) with a server device (400) or another edge AI device.

[0192] FIG. 1a(b) is a diagram illustrating a structure of interlocking between a plurality of edge AI devices according to an embodiment of the present invention, showing that each edge AI device is positioned at a different location within a dock and a ship and performs a unique role. In particular, the first edge AI device (100-1) may be positioned at the entrance of the dock, the second edge AI device (100-2) at the exit of the dock, the third edge AI device (100-3) at the entrance of the ship, and the fourth edge AI device (100-4) at the exit of the ship.

[0193] According to one embodiment of the present invention, a first edge AI device (100-1) is positioned at the entrance of a dock and performs the role of initially recognizing and registering a vehicle (10) entering the dock. Specifically, the sensor unit (110-1) of the first edge AI device (100-1) acquires vehicle identification information (30) in real time, such as the vehicle number, vehicle type, vehicle length, entry time, number of people, and reservation status of the entering vehicle, and the processor unit (130-1) assigns a unique serial number to each vehicle based on the vehicle identification information (30). Subsequently, the communication unit (120-1) can generate initial data for tracking the subsequent route of the vehicle (10) by sharing the vehicle identification information (30) and serial number information (40) with a server device (20) or another edge AI device.

[0194] FIG. 9 is a drawing for explaining vehicle identification information and serial number information according to an embodiment of the present invention.

[0195] According to one embodiment of the present invention, vehicle identification information (30) is information collected and generated by a first edge AI device (100-1) at the time when a vehicle (10) enters through the dock entrance, and refers to key identification data for distinguishing the vehicle (10) and tracking it in a subsequent path. For example, the vehicle identification information (30) may include vehicle number information (31), vehicle type information (32), vehicle length information (33), entry time information (34), reservation status information (35), and passenger count information (36).

[0196] As illustrated in FIG. 9, the vehicle identification information (30) may include vehicle number information (31), vehicle type information (32), vehicle length information (33), entry time information (34), reservation status information (35), passenger count information (36), etc. Here, the passenger count information (36) is information indicating the number of people inside the vehicle and can subsequently be used as reference data for the allocation of loading space on the ship, safety management, and cargo loading.

[0197] Additionally, according to an embodiment of the present invention, customer identification information corresponding to the driver or user of the vehicle, as well as information on the number of passengers, can be collected together. Through this, the customer's past usage history, frequently used terminals and time zones, and usage frequency can be verified and classified. This customer data is transmitted to a control server (30) via a server device (20) and can be utilized as basic data for providing customized customer-based services in the future. For example, if a specific customer repeatedly uses a specific time zone or route, this data can be used as a basis for providing differentiated services to that customer, such as priority boarding rights, discount benefits, and notification services.

[0198] According to one embodiment of the present invention, vehicle number information (31) is information obtained for the unique identification of a vehicle (10) and typically includes a registration number written on the license plate of the vehicle (10). Specifically, the vehicle number information (31) can be recognized through an Optical Character Recognition (OCR) algorithm included in the processor unit (130) and can be used for vehicle tracking and determining whether there is duplicate entry.

[0199] According to one embodiment of the present invention, vehicle type information (32) is information indicating the type to which the vehicle (10) belongs, such as a passenger car, a truck, a bus, or a trailer, and the processor unit (130) can classify the type of vehicle (10) based on various features such as external appearance recognition, vehicle body height, number of axles, and heating element pattern. In addition, the vehicle type information (32) can be used for calculating load distribution or setting boarding priorities when loaded on a ship, and can be used to establish an optimal layout strategy for the ship's interior space along with the length and number of people of the vehicle (10).

[0200] Additionally, the vehicle type information (32) that may be included in the vehicle identification information (30) is not limited to simply indicating the classification of vehicles such as passenger cars, trucks, and buses, but can be utilized in various aspects such as calculating load distribution when loaded on a ship, setting boarding priorities, cargo distribution strategies, and optimizing the layout of boarding spaces. In particular, when considered together with vehicle length information (33) and passenger count information (36), it is possible to establish an efficient layout strategy for the ship's internal space.

[0201] Additionally, according to some embodiments of the present invention, the vehicle type information (32) may include information that can estimate the total weight of the vehicle. For example, in the case of a cargo truck, the size and length of the loaded cargo are included, and in the case of a trailer vehicle, the total connection length of the entire trailer connected to the vehicle body and the total loading scale are identified together, thereby allowing the total weight of the vehicle to be indirectly determined. The vehicle type information (32) can be actively utilized for future weight distribution strategies for improving energy consumption efficiency within the vessel, securing safe load balance, and fuel-saving operation plans.

[0202] According to one embodiment of the present invention, vehicle length information (33) is information representing the total length of a vehicle (10) and can be obtained through a distance sensor, lidar, or image-based analysis included in an edge AI device (100). The vehicle length information (33) is used to identify vehicles exceeding the vessel's capacity in advance or to efficiently allocate cargo space. Additionally, the vehicle length information (33) can be applied to additional functions such as adjusting the spacing between waiting vehicles or optimizing the entry order.

[0203] According to one embodiment of the present invention, the vehicle length information (33) is not limited to the length of the vehicle body but is a concept that includes the total length occupied when the vehicle is actually boarded on the ship. For example, in the case of a cargo truck, it includes the length of the loaded cargo, and in the case of a trailer vehicle, it refers to the total connected length including the vehicle body and the trailer. This is essential information for space allocation and loading planning on the ship, and enables precise vehicle classification and loading optimization based on actual occupied space beyond simple vehicle type classification.

[0204] According to one embodiment of the present invention, the entry time information (34) is information that records the time when the vehicle (10) passes through the dock entrance in the form of a timestamp. The entry time information (34) can be used to determine the vehicle waiting order, review whether there is a delay compared to the reservation time, calculate the time of stay, etc., and is useful for dynamically controlling the flow of vehicles within the dock. In addition, the edge AI device (100) can calculate a queue weight based on the entry time information (34) or use it as a criterion for determining priority boarding conditions.

[0205] According to one embodiment of the present invention, the reservation status information (35) is information indicating whether the vehicle (10) has completed a boarding reservation in advance. The edge AI device (100) can determine the reservation status by linking with the server device (20) based on the vehicle number information (31), and the method of assigning serial number information (40) may vary depending on the reservation status information (35). For example, a serial number including a reservation identification number is assigned to a reserved vehicle, and a lower-priority serial number may be assigned to a non-reserved vehicle according to a queue management method.

[0206] According to one embodiment of the present invention, passenger count information (36) is information indicating the number of people on board the vehicle (10), and the passenger count information (36) can be measured by means such as a thermal imaging sensor, a door sensor, or a manager's verification. The passenger count information (36) can influence the securing of safety during operation, seat arrangement, and setting of ship loading standards, and is particularly used as basic data for identifying the number of passengers in the event of an emergency. In addition, the passenger count information (36) can also provide a function to verify whether the number of reserved passengers matches the actual number of passengers.

[0207] According to one embodiment of the present invention, vehicle identification information (30) is closely associated with the generation of serial number information (40) and is used as a reference point for tracking the entire route from when the vehicle (10) enters the dock until it boards and disembarks from the vessel. Serial number information (40) is an identification number uniquely assigned to the vehicle based on the vehicle identification information (30), and is generated one for each vehicle.

[0208] According to an embodiment of the present invention, serial number information (40) is not simply assigned randomly, but is configured to be distinguished in different ways depending on whether the vehicle is reserved. For example, in the case of a vehicle that has already been reserved, a serial number is assigned using a specific prefix or number system to distinguish it as a 'reserved vehicle,' while conversely, in the case of a general waiting vehicle that is not reserved, it can be set so that it is identified as an 'unreserved vehicle' through a separate number format. This method enables the administrator terminal or user application to immediately identify the reservation status of the vehicle using only the serial number, and allows for more efficient performance of determining boarding conditions, setting priorities, and managing queues.

[0209] As described above, the serial number information (40) is an identification code uniquely assigned to the vehicle based on the vehicle identification information (30), and is used as reference information to track the vehicle's entire movement process (entering the dock → boarding the ship → disembarking at the destination). The serial number information (40) is a unique value distinct from the vehicle number or type, and can be structured to include the reservation status, entry order, vehicle classification, etc. For example, the serial number "R001" assigned to the vehicle (10) consists of the prefix 'R' indicating that the vehicle is fully reserved and the vehicle's unique number (001). On the other hand, for a vehicle that is not reserved, the prefix "N (Non-reserved)" is used, so the serial number can be assigned in a format such as "N005". That is, the manager can immediately identify the vehicle's reservation status using only the serial number information (40), and it can be effectively utilized for various operations, such as setting queue priorities and determining whether to restrict boarding.

[0210] According to one embodiment of the present invention, a second edge AI device (100-2) is positioned at the exit of a dock and performs the role of identifying a vehicle and recording and verifying its movement path when a vehicle waiting within the dock moves out of the dock area to board a ship. The second edge AI device (100-2) determines whether it is the same vehicle based on serial number information (40) and vehicle identification information (30) assigned by the first edge AI device (100-1), and may approve passage through the exit only when appropriate conditions for boarding a ship, such as the arrival of the boarding time or confirmation of a reservation, are met.

[0211] According to one embodiment of the present invention, a third edge AI device (100-3) is positioned at the entrance of a ship and performs the role of verifying whether a vehicle that has left the dock actually boards the ship. The third edge AI device (100-3) utilizes serial number information (40) and vehicle identification information (30) received from the first edge AI device (100-1) and the second edge AI device (100-2) to perform identification and verification of the vehicle boarding the ship, and can record information such as the time of boarding completion, whether boarding has occurred, and the loading location within the ship. In addition, the third edge AI device (100-3) enables boarding history tracking by storing the serial number information (40) and vehicle identification information (30) in a memory unit (140-3) and simultaneously transmitting them to a server device (20) in real time.

[0212] According to one embodiment of the present invention, a fourth edge AI device (100-4) is positioned at the exit of a vessel that has arrived at a destination and performs the role of verifying the identity of a vehicle being disembarked and recording the end point of the entire journey. The fourth edge AI device (100-4) can verify the vehicle (10) once again based on the serial number information (40) and vehicle identification information (30) confirmed by the third edge AI device (100-3), and can record the time of disembarkation completion, the disembarkation location, etc. Additionally, the fourth edge AI device (100-4) can transmit the disembarkation information to a server device (20).

[0213] In this way, the first edge AI device (100-1) to the fourth edge AI device (100-4) are positioned at their respective locations to sequentially collect and verify information about the vehicle (10), thereby enabling automation and real-time tracking functions for the entire process of vehicle movement through the dock and the vessel. As a result, drivers and managers can efficiently identify and control the status, location, and boarding / disembarking status of each vehicle.

[0214] FIG. 2 is a diagram showing an example of an edge AI device tracking an object within a large complex facility according to the present invention.

[0215] As described above, the processor unit (130-1) of the first edge AI device (100-1) that senses the first area (200-1) of a large complex facility in real time can store information on the dwell time of an object (OB) in store 1, store 2, store 3, and the smoking room in the memory unit (140-1). Additionally, the processor unit (130-1) can store information on the movement path of an object (OB) within the first area (200-1) in the memory unit (140-1).

[0216] For example, if a short Asian woman in her 20s wearing a black top and blue bottoms enters the first area (200-1), the sensor unit (110-1) can sense the Asian woman in her 20s wearing a black top and blue bottoms within the first area (200-1) as an object (OB). In this case, the processor unit (130-1) can output object analysis information of the object (OB) based on size information of the object (OB) that the object (OB) is short, appearance information of the object (OB) that the object (OB) is plump, and color information of the object (OB) that the object (OB) has a black top and blue bottoms. Accordingly, the processor unit (130-1) can output object analysis information, which is information composed of at least one of the following: age information of the object (OB) being in its 20s, gender information of the object (OB) being female, and race information of the object (OB) being Asian.

[0217] According to one embodiment of the present invention, when a short Asian woman in her 20s wearing a black top and blue bottoms enters the first area (200-1), stays at store 1 (an eyewear store) for 10 minutes, and then stays at store 3 (a cosmetics store) for 40 minutes, the processor unit (130-1) can store in the memory unit (140-1) movement path information indicating that the object (OB) moved to store 3 after staying at store 1, and stay time information indicating that the object stayed at store 1 for 10 minutes and at store 3 for 40 minutes.

[0218] Accordingly, the processor unit (130-1) can create object-specific spatial stay information, such as “an Asian woman in her 20s stayed at the eyeglasses store for 10 minutes and stayed at the cosmetics store for 40 minutes,” based on object analysis information, which is composed of at least one of the age information of the object (OB) being in her 20s, gender information of the object (OB) being female, and race information of the object (OB) being Asian, movement path information, which indicates that the object (OB) stayed at store 1 and then moved to store 3, and stay time information, which indicates that the object (OB) stayed at store 1 for 10 minutes and at store 3 for 40 minutes.

[0219] Accordingly, the communication unit (120) can transmit spatial stay information by object to the server device (400), stating that "an Asian woman in her 20s stayed at the eyeglasses store for 10 minutes and stayed at the cosmetics store for 40 minutes," and the server device (400) can create a spatial stay information report by object group. Additionally, at least one of the following information can be transmitted to the server device (400): object analysis information, which is composed of at least one of the following: age information of the object (OB) being in her 20s; gender information of the object (OB) being female; and race information of the object (OB) being Asian; movement path information, which states that the object (OB) stayed at store 1 and then moved to store 3; and stay time information, which states that the object stayed at store 1 for 10 minutes and at store 3 for 40 minutes.

[0220] FIG. 3 is a diagram showing an example of tracking the movement path of an object within a large complex facility according to one embodiment of the present invention.

[0221] 1st object (OB1)

[0222] Area 1 (200-1)

[0223] As described, the first object (OB1) may stay at store 2 in the first area (200-1) for 30 minutes, move to restaurant 3 and stay at restaurant 3 for 1 hour, and move to the restroom and stay at the restroom for 10 minutes. In this case, the sensor unit (110-1) of the first edge AI device (100-1) can sense the first object (OB1) in the first area (200-1), which is the designated area. For a specific example, if the first object (OB1) is a tall, slim Western male in his 30s wearing a blue dress shirt, the processor unit (130-1) of the first edge AI device (100-1) can output object analysis information. At this time, the object analysis information for the first object (OB1) may be information output based on at least one of the size information of the first object (OB1) that the first object (OB1) is tall, the appearance information of the first object (OB1) that the first object (OB1) has a slim body shape, or the color information of the first object (OB1) that the first object (OB1) is wearing a blue top.

[0224] According to one embodiment of the present invention, if the first object (OB1) is a tall, slim Western male in his 30s wearing a blue dress shirt, the object analysis information for the first object (OB1) may be information output based on at least one of the size information of the first object (OB1) that the first object (OB1) is tall, the appearance information of the first object (OB1) that the appearance of the first object (OB1) is slim, or the color information of the first object (OB1) that the first object (OB1) is wearing a blue top. Accordingly, the processor unit (130-1) may output object analysis information for the first object (OB1) based on the above information, which is information composed of at least one of the age information of the first object (OB1) that the first object (OB1) is in his 30s, the gender information of the first object (OB1) that the first object (OB1) is male, and the race information of the first object (OB1) that the first object (OB1) is a Westerner.

[0225] As described above, when the first object (OB1) moves to store 2 (①-1), stays in store 2 for 30 minutes, and then moves toward the second area (200-2) (②-1), the processor unit (130-1) can store the movement path information (①-1, ②-1) of the first object (OB1) and the stay time information indicating that it stayed in store 2 for 30 minutes in the memory unit (140-1). Accordingly, the processor unit (130-1) can create object-specific spatial stay information stating that a Western male in his 30s stayed at store 2 based on object analysis information for the first object (OB1), which is composed of at least one of the age information of the first object (OB1) stating that the first object (OB1) is in his 30s, gender information of the first object (OB1) stating that the first object (OB1) is male, and race information of the first object (OB1) stating that the first object (OB1) is a Westerner, movement path information (①-1, ②-1) of the first object (OB1), and stay time information stating that he stayed at store 2 for 30 minutes. Furthermore, if store 2 is a store that sells men's watches, object-specific spatial stay information stating that "a Western male in his 30s stayed at a store that sells men's watches for 30 minutes" can be created.

[0226] Area 2 (200-2)

[0227] According to one embodiment of the present invention, the first object (OB1) moves so that the sensor unit (110-1) can no longer sense the first object (OB1) in the first area (200-1). That is, if the processor unit (130-1) can no longer obtain the movement path information (①-1, ②-1) of the first object (OB1), the processor unit (130-1) may use another edge AI device, such as the second edge AI device (100-2), the third edge AI device (100-3), or the fourth edge AI device (100-4), to obtain the object analysis information generated by the processor unit (130-1) (object analysis information for the first object (OB1) which is composed of at least one of the age information of the first object (OB1) being in its 30s, the gender information of the first object (OB1) being male, and the race information of the first object (OB1) being a Westerner), the movement path information (①-1, ②-1) stored in the memory (140-1), the stay time information stating that the person stayed at store 2 for 30 minutes, and the fact that a Western male in his 30s stayed for an average of 30 minutes The communication unit (120-1) can be controlled to transmit at least one of the spatial information, such as staying at a store selling men's watches. Additionally, the processor unit (130-1) may assign an ID to the first object (OB1), record the movement path information (①-1, ②-1) of the first object (OB1) and the stay time information, such as staying at store 2 for 30 minutes, in the ID assigned to the first object (OB1), and transmit the ID along with the record to another edge AI device.

[0228] According to another embodiment of the present invention, the processor unit (130-1) may control the communication unit (120-1) to transmit object analysis information generated by other edge AI devices, such as a second edge AI device (100-2), a third edge AI device (100-3), or a fourth edge AI device (100-4), and movement path information (①-1, ②-1) stored in memory (140-1), but the communication unit (120-1) may also control the communication unit (120-1) to transmit object analysis information generated and movement path information (①-1, ②-1) stored in memory (140-1) only to another edge AI device (100-2) that senses a second area (200-2) located in the direction where the movement path of the first object (OB1) is no longer sensed, by determining the direction in which the movement path of the first object (OB1) is no longer sensed.

[0229] In this case, the second edge AI device (100-2) can generate new object-specific spatial dwell information based on object analysis information generated by the first edge AI device (100-1) received for the first object (OB1), movement path information (①-1, ②-1) stored in memory (140-1), etc., and object analysis information generated by the second edge AI device (100-2) itself for the first object (OB1). Additionally, the processor unit (130-1) may assign an ID to the first object (OB1), record the movement path information (①-1, ②-1) of the first object (OB1) and dwell time information, such as that the object stayed in store 2 for 30 minutes, on the ID assigned to the first object (OB1), and transmit the ID along with the record only to another edge AI device (100-2) sensing the second area (200-2).

[0230] According to one embodiment of the present invention, the sensor unit (110-2) of the second edge AI device (100-2) can sense the first object (OB1) in the second area (200-2), which is a designated area. For example, if the first object (OB1) is a tall, slim Western male in his 30s wearing a blue dress shirt, the processor unit (130-2) can store in the memory unit (140-2) movement path information (③-1, ④-1) that the first object (OB1) entered the second area (200-2) from the direction where the first edge AI device (100-1) is located, visited Restaurant 3, and then moved to the restroom, and dwell time information that the first object stayed at Restaurant 3 for 1 hour and stayed at the restroom for 10 minutes.

[0231] According to one embodiment of the present invention, the processor unit (130-2) of the second edge AI device (100-2) can generate object-specific spatial stay information stating "a Western male in his 30s stayed for about 1 hour at a Western restaurant with a modern atmosphere" based on: object analysis information for the first object (OB1), which is composed of at least one of age information of the first object (OB1) being in his 30s, gender information of the first object (OB1) being male, and race information of the first object (OB1) being a Westerner, when Restaurant 3 is a "Western restaurant with a modern atmosphere"; movement path information (③-1, ④-1) stating that the first object (OB1) entered the second area (200-2) from the direction where the first edge AI device (100-1) is located, visited Restaurant 3, and then moved to the restroom; and stay time information stating that the first object (OB1) stayed at Restaurant 3 for 1 hour and stayed in the restroom for 10 minutes. there is.

[0232] Accordingly, the communication unit (120-2) can transmit object-specific space stay information, such as “a Western male in his 30s stayed for about 1 hour at a Western-style restaurant with a modern atmosphere,” to the server device (400), and the communication unit (120-2) can generate a report of object-specific space stay information to the server device (400). Additionally, the first object (OB1) may transmit to the server device (400) object analysis information, which is composed of at least one of the following: age information of the first object (OB1) being in their 30s, gender information of the first object (OB1) being male, and race information of the first object (OB1) being a Westerner; movement path information (③-1, ④-1), which indicates that the first object (OB1) entered the second area (200-2) from the direction where the first edge AI device (100-1) is located, visited restaurant 3, and then moved to the restroom; and stay time information, which indicates that the first object (OB1) stayed at restaurant 3 for 1 hour and stayed at the restroom for 10 minutes.

[0233] 2nd object

[0234] Area 3 and Area 4 (200-3, 200-4)

[0235] As described, the second object (OB2) may stay at store 5 in the third area (200-3) for 20 minutes, move to store 4 and stay at store 4 for 10 minutes, move to the smoking room and stay in the smoking room for 5 minutes, and move to restaurant 1 and stay at restaurant 1 for 40 minutes. In this case, the sensor unit (110-3) of the third edge AI device (100-3) can sense the second object (OB2) in the third area (200-3), which is the designated area. For a specific example, if the second object (OB2) is a short, slim Asian woman in her 40s wearing a red dress, the processor unit (130-3) of the third edge AI device (100-3) can output object analysis information. At this time, the object analysis information for the second object (OB2) may be information output based on at least one of the size information of the second object (OB2) that the second object (OB2) is short, the appearance information of the second object (OB2) that the second object (OB2) has a slim body shape, or the color information of the second object (OB2) that the second object (OB2) is wearing a red dress.

[0236] According to one embodiment of the present invention, if the second object (OB2) is a slim Asian woman in her 40s wearing a short red dress, the object analysis information for the second object (OB2) may be information output based on at least one of the following: size information of the second object (OB2) being short, appearance information of the second object (OB2) being slim, or color information of the second object (OB2) being wearing a red dress. Accordingly, the processor unit (130-3) may output object analysis information for the second object (OB2) based on the above information, which is information composed of at least one of the following: age information of the second object (OB2) being in her 40s, gender information of the second object (OB2) being female, and race information of the second object (OB2) being Asian.

[0237] As described above, when the second object (OB2) moves to store 5 (①-2), stays at store 5 for 20 minutes, moves to store 4 (②-2) and stays for 10 minutes, and then moves toward the fourth area (200-4) (③-2), the processor unit (130-3) can store the movement path information (①-2, ②-2, ③-2) of the second object (OB2) and the stay time information that it stayed at store 5 for 20 minutes and at store 4 for 10 minutes in the memory unit (140-3).

[0238] Accordingly, the processor unit (130-3) can create object-specific spatial stay information stating "an Asian woman in her 40s stayed at store 5 and store 4" based on object analysis information for the second object (OB2), which is composed of at least one of the following: age information of the second object (OB2) being in her 40s, gender information of the second object (OB2) being female, and race information of the second object (OB2) being Asian; movement path information of the second object (OB2) (①-2, ②-2, ③-2); and stay time information stating that she stayed at store 5 for 20 minutes and at store 4 for 10 minutes. Furthermore, if store 5 is a store selling women's cosmetics, object-specific spatial stay information stating "an Asian woman in her 40s stayed at the store selling women's cosmetics for 20 minutes" can be created.

[0239] FIG. 4 is a diagram showing an example of tracking the movement path of an object within a store according to one embodiment of the present invention.

[0240] As described above, when the second object (OB2) is a short, slim Asian woman in her 40s wearing a red dress and the second object (OB2) enters store 5, the edge AI device (100-5) located in store 5 may include at least one sensor unit. For example, the first sensor unit (110-5①), the second sensor unit (110-5②), and the third sensor unit (110-5③) of the edge AI device (100-5) may be located at each display stand inside store 5, and each of the first sensor unit (110-5①), the second sensor unit (110-5②), and the third sensor unit (110-5③) may sense the second object (OB2) located in each of the first area (210-1), the second area (210-2), and the third area (210-3).

[0241] Accordingly, the processor unit (130-5) can output object analysis information for the second object (OB2), and the object analysis information for the second object (OB2) may be information output based on at least one of the size information of the second object (OB2) that the second object (OB2) is short, the appearance information of the second object (OB2) that the second object (OB2) has a slim body shape, or the color information of the second object (OB2) that the second object (OB2) is wearing a red dress. Accordingly, the processor unit (130-3) can output object analysis information for the second object (OB2) that is composed of at least one of the age information of the second object (OB2) that the second object (OB2) is in its 40s, the gender information of the second object (OB2) that the second object (OB2) is female, and the race information of the second object (OB2) that the second object (OB2) is Asian.

[0242] According to one embodiment of the present invention, a second object (OB2) may enter store 5, stay at shelf 1 (5-1) for 3 minutes without staying at shelf 3 (5-3), stay at shelf 2 (5-2) for 15 minutes, and then exit. In this case, the processor unit (130-5) may store movement path information (⑤-1, ⑤-2 or ⑤-6) of the second object (OB2) and stay time information indicating that it did not stay at shelf 3 (5-3) in the memory unit (140-5) based on the sensing data of the second object (OB2) sensed by the third sensor unit (110-5③) in the third area (210-3). Additionally, the processor unit (130-5) can store movement path information (⑤-2, ⑤-3) of the second object (OB2) and dwell time information indicating that it stayed at display stand 1 (5-1) for 3 minutes in the memory unit (140-5) based on the sensing data of the second object (OB2) sensed by the first sensor unit (110-5①) in the first area (210-1). Additionally, the processor unit (130-5) can store movement path information (⑤-3, ⑤-4, or ⑤-5) of the second object (OB2) and dwell time information indicating that it stayed at display stand 2 (5-2) for 15 minutes in the memory unit (140-5) based on the sensing data of the second object (OB2) sensed by the second sensor unit (110-5②) in the second area (210-2).

[0243] According to one embodiment of the present invention, the processor unit (130-5) [describes] object analysis information for the second object (OB2), which is composed of at least one of age information of the second object (OB2) that the second object (OB2) is in its 40s, gender information of the second object (OB2) that the second object (OB2) is female, and race information of the second object (OB2) that the second object (OB2) is Asian, movement path information of the second object (OB2) (⑤-1, ⑤-2, ⑤-3, ⑤-3, ⑤-4, or ⑤-5), dwell time information that the second object (OB2) did not stay at shelf 3 (5-3), dwell time information that the second object stayed at shelf 1 (5-1) for 3 minutes, and dwell time information that the second object stayed at shelf 2 (5-2) for 15 minutes, and [describes] that "an Asian woman in her 40s stayed at a shelf selling color cosmetics for 3 minutes, did not stay at a shelf selling skin care products, and stayed at a shelf selling functional cosmetics for 15 minutes." You can create space dwell information for each object.

[0244] Accordingly, the processor unit (130-5) can transmit object-specific spatial stay information to the central server (not shown) via the communication unit (120-5), stating that "an Asian woman in her 40s stayed at a display stand selling color cosmetics for 3 minutes, did not stay at a display stand selling skin care products, and stayed at a display stand selling functional cosmetics for 15 minutes." Additionally, the processor unit (130-5) can transmit object analysis information for the second object (OB2), which is composed of at least one of age information of the second object (OB2) being in their 40s, gender information of the second object (OB2) being female, and race information of the second object (OB2) being Asian, movement path information of the second object (OB2) (⑤-1, ⑤-2, ⑤-3, ⑤-3, ⑤-4 or ⑤-5), dwell time information indicating that the second object (OB2) did not stay at display stand 3 (5-3), dwell time information indicating that the second object stayed at display stand 1 (5-1) for 3 minutes, and dwell time information indicating that the second object stayed at display stand 2 (5-2) for 15 minutes, to a central server (not shown) through the communication unit (120-5).

[0245] According to one embodiment of the present invention, an object tracking and object-specific spatial dwell information collection system may include a third edge AI device (100-3) that senses a third area (200-3) including store 5, and an edge AI device (100-5) located at store 5. In this case, the third edge AI device (100-3) may recognize that a second object (OB2) is entering store 5, and may assign an ID to the second object (OB2) entering store 5 and store ID information for the second object (OB2) in a memory (140-3). The ID information for the second object (OB2) stored in the memory (140-3) may be at least one of the following: object analysis information for the second object (OB2) output by the third edge AI device (100-3), movement path information for the second object (OB2), dwell time information for the second object (OB2) in the third area (200-3), and object-specific spatial dwell information for the second object (OB2), but is not limited thereto.

[0246] FIG. 5 is a diagram illustrating communication and object tracking between a plurality of edge AI devices according to an embodiment of the present invention.

[0247] Unique information matching of the processor section

[0248] The edge AI device (100) according to the present invention includes a processor unit (130). Specifically, the processor unit (130) can match unique information stored in the memory unit (140) with an object sensed by the sensor unit (110). For a specific example, if the object sensed by the sensor unit (110) is a first object (OB1) and the first object (OB1) has previously sensed its face with the sensor unit (110), then the face sensing information of the first object (OB1) and unique information required for the identification of the first object (OB1) can be stored in pairs in the memory unit (140) as described above. In this case, since the face sensing information of the first object (OB1) is stored in the memory unit (140), the processor unit (130) can match the object sensed by the sensor unit (110) with the unique information required for identification stored in pairs with the stored face sensing information. That is, if the unique information required for identification stored in pairs with the stored face sensing information is the unique ID information 1234-5678, the processor unit (130) can match the unique ID information 1234-5678 to the object sensed by the sensor unit (110).

[0249] Unable to match unique information

[0250] FIG. 6 is a diagram illustrating an embodiment in which a processor unit according to the present invention cannot match an object sensed by a sensor unit with unique information stored in a memory unit.

[0251] The edge AI device (100) according to the present invention includes a processor unit (130). Specifically, the processor unit (130) can match unique information stored in the memory unit (140) with an object sensed by the sensor unit (110). However, there are cases where the processor unit (130) cannot match an object sensed by the sensor unit (110) with unique information stored in the memory unit (140). In this case, the processor unit (130) can transmit information regarding external access to a user terminal (300), a server device (400), or other edge AI devices (100-2, 100-3, 100-4, 100-5) through the communication unit (120). As a specific example, the sensor unit (110-1) of the first edge AI device (100-1) can sense a designated area, which is the first area (200-1).

[0252] Information regarding outsider access - appearance information, movement path information, duration of stay information

[0253] As illustrated in FIG. 6, an Asian male in his 30s wearing a black top and blue bottoms may be located in the designated area, the first area (200-1). In this case, the sensor unit (110-1) of the first edge AI device (100-1) can sense a third object (OB3), which is an Asian male in his 30s wearing a black top and blue bottoms, in the designated area, the first area (200-1). Accordingly, the processor unit (130-1) can match unique information stored in the memory unit (140-1) with the third object (OB3) sensed by the sensor unit (110-1).

[0254] However, if the processor unit (130-1) cannot match the unique information stored in the memory unit (140-1) with the third object (OB3), it may transmit information related to external access to the user terminal (300), server device (400), or other edge AI devices (100-2, 100-3, 100-4, 100-5) through the communication unit (120-1). Specifically, the processor unit (130-1) may transmit movement path information and dwell time information of the third object (OB3) to the user terminal (300), server device (400), or other edge AI devices (100-2, 100-3, 100-4, 100-5) through the communication unit (120-1). For example, if a third object (OB3) is located in the first area (200-1) at 1:41 PM on November 25, 2024, and stays there for 18 minutes until 1:59 PM on November 25, 2024, the processor unit (130-1) can transmit information to a user terminal (300), a server device (400), or other edge AI devices (100-2, 100-3, 100-4, 100-5) via the communication unit (120-1) that the third object (OB3) is located in the first area (200-1) and stayed in the first area (200-1) for 18 minutes from 1:41 PM on November 25, 2024, until 1:59 PM on November 25, 2024.

[0255] According to one embodiment of the present invention, the processor unit (130-1) can transmit appearance information of a third object (OB3) to a user terminal (300), a server device (400), or other edge AI devices (100-2, 100-3, 100-4, 100-5) through the communication unit (120-1). Specifically, the appearance information may include information that the third object (OB3) is an Asian male in his 30s wearing a black top and blue bottoms. Additionally, the appearance information may include information that the third object (OB3) is wearing a black top and blue bottoms. Additionally, the appearance information may include information that the third object (OB3) is an Asian male in his 30s, but is not limited thereto.

[0256] Door Locking Status Retention Notification

[0257] According to the present invention, the processor unit (130) can match unique information stored in the memory unit (140) with an object sensed by the sensor unit (110). However, there are cases where the processor unit (130) cannot match an object sensed by the sensor unit (110) with unique information stored in the memory unit (140). In this case, the processor unit (130-1) can transmit a notification to the user terminal (300) or server device (400) via the communication unit (120-1) to maintain the locking state of the entrance door of the space facility so as not to open the entrance door of the space facility. Specifically, the processor unit (130-1) can transmit a signal via the communication unit (120-1) to the user terminal (300) or server device (400) to detect an event when the entrance door of the space facility is opened.

[0258] The movement path and dwell time information of the matched object are stored in the memory.

[0259] According to one embodiment of the present invention, the processor unit (130) can match unique information stored in the memory unit (140) to an object sensed by the sensor unit (110). Accordingly, the processor unit (130) can store movement path information and dwell time information of the matched object (OB) in the memory unit (140). Hereinafter, the example of FIG. 5 will be described.

[0260] As illustrated in FIG. 5, the first edge AI device (100-1) may be located outside the security facility. When the first object (OB1) is located in the first area (200-1), the sensor unit (110-1) of the first edge AI device (100-1) may sense the face or iris of the first object (OB1) located in the first area (200-1). When the sensor unit (110-1) senses the face or iris of the first object (OB1) located in the first area (200-1), the processor unit (130-1) may check whether the face sensing information or iris sensing information of the first object (OB1) is stored in the memory unit (140-1). Accordingly, when the processor unit (130-1) confirms that face sensing information of the first object (OB1) is stored in the memory unit (140-1), it can match the unique number '1234-5678' stored in the memory unit (140-1) to the first object (OB1).

[0261] According to one embodiment of the present invention, the processor unit (130-1) can store movement path information and dwell time information of the first object (OB1) having unique information of '1234-5678' in the memory unit (140-1). As described, if the first object (OB1) having unique information '1234-5678' moved from the first area (200-1) to the first door (IN1) at 3:48 PM on October 21, 2024 (①-1), entered the first door (IN1) at 3:49 PM on October 21, 2024, and exited the first door (IN1) at 7:49 PM on October 21, 2024 (⑦-1), the processor unit (130-1) [describes] that the first object (OB1) having unique information '1234-5678' in the space facility [dated] from 3:49 PM to 7:49 PM on October 21, 2024 Information on the duration of stay for 4 hours can be stored in the memory unit (140-1). Additionally, the processor unit (130-1) can store in the memory unit (140-1) information on the movement path of the first object (OB1) having unique information '1234-5678', which entered through the first door (IN1) and exited from the first door (IN1).

[0262] According to another embodiment of the present invention, the processor unit (130-1) can store movement path information and dwell time information of a second object (OB2) having unique information of '1234-5555' in the memory unit (140-1). As described, if a second object (OB2) having unique information '1234-5555' moved from the first area (200-1) to the second door (IN2) at 7:48 AM on November 21, 2024 (①-2), entered the second door (IN2) at 7:49 AM on November 21, 2024, and exited from the first door (IN1) at 2:49 PM on November 21, 2024 (⑧-2), the processor unit (130-1) [describes] that in the space facility, the second object (OB2) having unique information '1234-5555' [did not enter] from 7:49 AM to 2:49 PM on November 21, 2024 Information on the duration of stay for 6 hours can be stored in the memory unit (140-1). Additionally, the processor unit (130-1) can store in the memory unit (140-1) information on the movement path of a second object (OB2) having unique information '1234-5555', which entered through the second door (IN2) and exited through the first door (IN1).

[0263] Area exit

[0264] As illustrated in FIG. 2, there is a case where the first object (OB1) having unique information '1234-5678' moves out of the first area (200-1) and the sensor unit (110-1) can no longer sense the first object (OB1). In this case, the unique information of the first object (OB1) having unique information '1234-5678', the '1234-5678' information, the stored movement path information (①-1), and the stored dwell time information (entered through the first door (IN1) at 3:49 PM on October 21, 2024) can be transmitted to the second edge AI device (100-2) via the communication unit (120-1).

[0265] For a specific example, the second edge AI device (100-1) can receive from the first edge AI device (100-1) the unique information of the first object (OB1) having the unique information of 1234-5678, the '1234-5678' information, the stored movement path information (①-1), and the stored dwell time information (entering the first door (IN1) at 3:49 PM on October 21, 2024), and the second edge AI device (100-1) can track the first object (OB1) having the unique information of 1234-5678 in the second area (200-2). Accordingly, the second edge AI device (100-1) can store the movement path information (②-1, via A1 and ③-1) of the first object (OB1) in the second area (200-2) in the memory unit (140-2), following the movement path information and dwell time information (entering through the first door (IN1) at 3:49 PM on October 21, 2024) stored in the memory unit (140-1).

[0266] According to one embodiment of the present invention, a second edge AI device (100-1) can track a first object (OB1) having unique information of 1234-5678' in a second area (200-2). For example, there is a case where the first object (OB1) having unique information of 1234-5678' entered the first entrance (IN1) at 3:49 PM on October 21, 2024, and then stayed in Area 1 (A1) from 3:52 PM on October 21, 2024, to 4:32 PM on October 21, 2024. At this time, the second edge AI device (100-2) can store in the memory unit (140-2) information on the movement path of the first object (OB1) having unique information of '1234-5678' in the second area (200-2) moving (②-1) to the area 1 (A1) and then moving again (③-1), and information on the time of stay of the first object (OB1) staying in the area 1 (A1) for 40 minutes from 3:52 PM on October 21, 2024 to 4:32 PM on October 21, 2024.

[0267] According to one embodiment of the present invention, when the matching object moves and the sensor unit (110-2) can no longer sense the matching object in the designated area (200-2), the processor unit (130-2) can transmit at least one of the unique information required for identification of the matching object, the stored movement path information, the stored dwell time information, and the created space dwell information to another AI device through the communication unit (120-2).

[0268] As described above, when the first object (OB1) having unique information of '1234-5678' moves (③-1) and the first object (OB1) can no longer be sensed in the second area (200-2), the processor unit (130-2) can transmit to the third edge AI device (100-3) via the communication unit (120-2) information on the movement path of the first object (OB1) having unique information of '1234-5678' moving (②-1) from the second area (200-2) to be located in Zone 1 (A1), and then moving again (③-1), and information on the time of stay of the first object (OB1) staying in Zone 1 (A1) for 40 minutes from 3:52 PM on October 21, 2024 to 4:32 PM on October 21, 2024.

[0269] Create space stay information

[0270] The following describes the information regarding the stay in the space.

[0271] According to one embodiment of the present invention, the processor unit (130) can create spatial dwelling information of a sensed object based on unique information required for identifying a matched object, stored movement path information, and stored dwelling time information. For example, the second edge AI device (100-2) can store in the memory unit (140-2) movement path information that a first object (OB1) having unique information '1234-5678' in the second area (200-2) moved (②-1) to area 1 (A1) and then moved again (③-1), and dwelling time information that the object stayed in area 1 (A1) for 40 minutes from 3:52 PM on October 21, 2024 to 4:32 PM on October 21, 2024.

[0272] In this case, if Zone 1 (A1) is a security facility related to memory semiconductors, the processor unit (130-2) can create space stay information indicating that the first object (OB1) having the unique information '1234-5678' is performing work related to the security facility related to memory semiconductors. Additionally, if Zone 2 (A2) is a security facility related to AI semiconductors and the first object (OB1) having the unique information '1234-5678' moves (④-1) and stays in Zone 2 (A2) for about 2 hours, the processor unit (130-2) can create space stay information indicating that the first object (OB1) having the unique information '1234-5678' is performing work related to memory semiconductors and AI semiconductors, but is spending more time on work related to AI semiconductors.

[0273] FIG. 7 is a diagram illustrating an embodiment in which an access authorization authentication device according to the present invention communicates with an edge AI device.

[0274] According to the present invention, an edge AI device (100) can determine object tracking and access rights to an object within a spatial facility. Specifically, a communication unit (120) can communicate with another edge AI device, a user terminal (300), or a server device (400), and the communication unit (120) can receive unique information necessary for identifying an object and information restricting access to an object from the user terminal (300) or the server device (400). As a specific example, the communication unit (120) can receive at least one of the following from the user terminal (300) or the server device (400): appearance information including photos of ‘A Research Institute employees’, age information, name information, unique ID information, job title information, information on accessible areas, information on time limits for areas accessible only for a limited time, and information on tasks currently being performed.

[0275] As illustrated in FIG. 7, when object 1 (OB1) is located in the first area (200-1), the processor unit (130-1) of the first edge AI device (100-1) can match object 1 (OB1) with unique information required for object identification received from the user terminal (300) or server device (400) by the communication unit (120). For example, if the unique information required for object identification received from the user terminal (300) or server device (400) by the communication unit (120) is the external appearance information of object 1 (OB1) and the unique ID information of '1234-5678', the processor unit (130-1) can match object 1 (OB1) with the unique ID information of '1234-5678'. In this case, the processor unit (130-1) can store in the memory unit (140-1) movement path information (①-1 and ⑦-1) of object 1 (OB1) having unique ID information of '1234-5678' within the first area (200-1) and dwell time information indicating that object 1 (OB1) having unique ID information of '1234-5678' was located within the first area (200-1) at 2:17 PM on December 5, 2024 and at 5:17 PM on December 5, 2024.

[0276] According to one embodiment of the present invention, the access restriction information of an object received from a user terminal (300) or a server device (400) by the communication unit (120-2) of the second edge AI device (100-2) may be information that the access rights of object 1 (OB1) having the unique ID information '1234-5678' are restricted in area 1 (A1).

[0277] At this time, since object 1 (OB1) has moved (②-1) and is staying in zone 1 (A1), the processor unit (130-2) can determine whether the stored movement path information (②-1, staying in A1, ③-1) and the stored stay time information (staying in zone 1 (A1) for 1 hour from 2:47 PM on December 5, 2024 to 3:47 PM on December 5, 2024) are included in the access restriction information of the received object, “information that the access rights of object 1 (OB1) having the unique ID information of ‘1234-5678’ are restricted in zone 1 (A1).”

[0278] Accordingly, when the processor unit (130-2) determines that the “information that access rights for object 1 (OB1) having unique ID information of ‘1234-5678’ are restricted in Zone 1 (A1)” includes movement path information (②-1, stay in A1, ③-1) and stored stay time information (stayed in Zone 1 (A1) for 1 hour from 2:47 PM on December 5, 2024 to 3:47 PM on December 5, 2024), it can transmit an access restriction notification signal to other edge AI devices (100-1, 100-3, 100-4, 100-5), user terminals (300), or server devices (400) through the communication unit (120-2).

[0279] According to one embodiment of the present invention, a communication unit (120) may receive access restriction information for an object from a user terminal (300) or a server device (400). Specifically, the access restriction information for an object may include at least one of access restriction information in an object-specific spatial facility having unique information, access restriction information for a predetermined area inside an object-specific spatial facility having unique information, access restriction information for a time exceeding a threshold in an object-specific spatial facility having unique information, and access restriction information for a time exceeding a threshold in a predetermined area inside an object-specific spatial facility having unique information.

[0280] According to one embodiment of the present invention, access restriction information for an object may include access restriction information in a spatial facility specific to the object having unique information. For example, access restriction information in a spatial facility specific to the object having unique information may be access restriction information in a spatial facility stating that the object possesses unique information but has currently resigned and can no longer access the spatial facility.

[0281] According to one embodiment of the present invention, access restriction information for an object may include access restriction information for a predetermined area within an object-specific spatial facility having unique information. For example, access restriction information for a predetermined area within an object-specific spatial facility having unique information may be access restriction information within the spatial facility stating that the object has access rights to the spatial facility, but access rights to Zone 1 and Zone 4 are restricted.

[0282] According to one embodiment of the present invention, access restriction information for an object may include access restriction information for a time exceeding a threshold in an object-specific space facility having unique information. For example, if the object visits with the authority of a visitor rather than an employee of the space facility, the access restriction information for a time exceeding a threshold in the object-specific space facility may be access restriction information within the space facility stating that although the object has access rights to the space facility, it cannot stay in the space facility for more than one hour.

[0283] According to one embodiment of the present invention, access restriction information for an object may include access restriction information for a time exceeding a threshold in a predetermined area within an object-specific spatial facility having unique information. For example, if staying for more than one hour is prohibited because radiation emitted from Zone 1 is fatal to the human body, the access restriction information for a time exceeding a threshold in a predetermined area within an object-specific spatial facility may be access restriction information within the spatial facility stating that one cannot stay in Zone 1 for more than one hour.

[0284] As illustrated in FIG. 7, an edge AI device (100) can communicate with an access authorization authentication device (50). Specifically, the communication unit (120) of the edge AI device (100) can receive at least one of access authorization authentication information, access authorization non-authentication information, unique information required for identifying an object, and access restriction information of an object from the access authorization authentication device (50).

[0285] According to one embodiment of the present invention, an access authorization authentication device (50-1) can receive at least one of the following from an object: external appearance information of the object, fingerprint information of the object, vein information of the object, card key information owned by the object, password information of the access authorization authentication device, and voice information of the object, and create access authorization authentication information of the object or access authorization non-authentication information of the object.

[0286] For example, if Object 1 (OB1) recognizes a fingerprint on the access authorization authentication device (50-1), the access authorization authentication device (50-1) can create access authorization authentication information for Object 1 (OB1). Accordingly, the access authorization authentication device (50-1) can transmit at least one of the access authorization authentication information of Object 1 (OB1), unique information required for the identification of Object 1 (OB1), and access restriction information of Object 1 (OB1) to the first edge AI device (100-1). The access authorization authentication device (50-1) may also transmit only the access authorization authentication information of Object 1 (OB1) to the first edge AI device (100-1). In this case, the communication unit (120-1) of the first edge AI device (100-1) can receive access authorization authentication information from the access authorization authentication device (50-1) and can receive at least one of unique information required for identifying object 1 (OB1) and access restriction information for object 1 (OB1) from the user terminal (300) or server device (400).

[0287] FIG. 8a is a diagram schematically illustrating the communication structure and role sharing relationship of a head AI device (500) and a node AI device (500′) according to one embodiment of the present invention.

[0288] As described above, one or more head AI devices (500) are deployed in each area, such as a dock entrance or a ship boarding gate, and the head AI device (500) is connected to multiple node AI devices (500') to perform a central role in intensively collecting, processing, and managing vehicle information in the area. Specifically, the head AI device (500) includes a sensor unit (510), a communication unit (520), a processor unit (530), and a memory unit (540), and each component is functionally similar to the configuration of the existing edge AI device (100), but can perform a higher level of integrated judgment function. In particular, the communication unit (520) receives vehicle identification information (30) from multiple adjacent node AI devices (500'), analyzes and refines it through the processor unit (530), and then performs data exchange with the server device (400).

[0289] According to one embodiment of the present invention, the head AI device (500) is not merely a simple data relay device, but can perform local control functions such as comprehensively identifying the state of vehicle flow within each zone, determining priorities according to the situation, or adjusting the traffic flow of reserved and non-reserved vehicles. As a result, the information sensed by the node AI devices (500′) installed on-site is not merely transmitted to the server device (400), but is transmitted to the server device (400) after undergoing pre-processing in the head AI device (500), thereby significantly improving data processing efficiency and response speed.

[0290] According to one embodiment of the present invention, the head AI device (500) is configured to communicate with other head AI devices (500) in neighboring areas, thereby enabling integrated control and prediction of the entire dock or the entire ship boarding flow beyond a single area. For example, the head AI device (500) located at the entrance of the dock can perform distributed coordination control, such as checking the vehicle loading status of the ship boarding area in real time, adjusting the entry speed of waiting vehicles, or providing guidance information through an electronic display board.

[0291] According to one embodiment of the present invention, a node AI device (500′) performs the function of generating various sensing data for vehicles within a specific monitoring area in which it is deployed. The node AI device (500′) may typically be deployed in multiple locations where vehicle traffic occurs, such as vehicle entry lanes, waiting areas, and stopping points before boarding a ship, and each device detects information such as license plate recognition, vehicle size (length), location coordinates, whether it is stopped, movement speed, time of passage, and vehicle type for vehicles passing through or stopping at the location.

[0292] To this end, the node AI device (500′) may include a sensor unit such as a camera, a distance sensor, an infrared sensor, or a lidar sensor, and may incorporate an image-based object recognition algorithm or a neural network-based inference module to extract characteristic information of the vehicle in real time. The sensing data generated in this way is first processed in the processor unit within the node AI device (500′) and then transmitted to the head AI device (500) located in the upper layer. Subsequently, the head AI device (500) synthesizes the sensing data collected from multiple node AI devices (500′) to generate final vehicle identification information (30) and transmits it to the server device (400).

[0293] According to one embodiment of the present invention, the edge AI device (100) does not communicate directly with the server device (400), and only the head AI device (500) is connected to the server device (400) or can communicate with the head AI device (500) in another area. Through this, the edge AI device (100) can minimize communication costs and improve the processing efficiency of edge-based data, and in terms of security, it has the advantage of preventing the direct communication exposure of the node AI device (500'). Additionally, the node AI device (500') may include sensor functions and some processing functions itself, but forms a single control structure (hierarchical control) through a one-to-many connection with the head AI device (500), and may not perform direct communication with other nodes or other edge AI devices. As a result, each head AI device (500) can maintain a consistent flow of information even when inter-zone linkage is required by integrating the sensing results of the node AI devices (500′) under its jurisdiction and transmitting them to the server device (400) or sharing them with other head AI devices (500).

[0294] In the present invention, a plurality of node AI devices (500′) and a head AI device (500) are connected to each other via a network, and data transmission and reception between them can be performed through a communication unit (520) included in each device.

[0295] According to one embodiment of the present invention, the node AI device (500′) is positioned at different locations to individually acquire image data and transmit it to the head AI device (500). The head AI device (500) integrates the received image data and performs complex operations such as object detection, multi-viewpoint-based tracking, and primary behavior analysis to process it into high-dimensional spatiotemporal information.

[0296] Additionally, the head AI device (500) may also be connected to a server device (400), and the server device (400) may perform advanced processing such as additional secondary behavioral analysis, natural language text generation, and voice content generation based on the primary analysis results received from the head AI device (500). The processed information may be transmitted to a user terminal (300) and provided in the form of actual voice relay data or visual content.

[0297] As such, the present invention is configured to efficiently perform the entire process from distributed collection of object-based image data acquired in real-time at the field to integrated analysis and content generation through an edge-head-server hierarchical structure. In particular, by having each node AI device operate as a simple image acquisition node and performing analysis and comprehensive processing intensively in the head AI device and server, the invention provides a structure capable of simultaneously achieving network efficiency and distribution of processing load.

[0298] According to one embodiment of the present invention, the first edge AI device (100-1) to the fourth edge AI device (100-4) can each be implemented as a head AI device (500). Specifically, the first edge AI device (100-1) can be placed at the entrance of a dock, the second edge AI device (100-2) at the exit of a dock, the third edge AI device (100-3) at the boarding port of a ship, and the fourth edge AI device (100-4) at the disembarking port of a ship, and can operate as a higher-level control device capable of integratedly managing vehicle identification information (30) and serial number information (40) collected while communicating with a plurality of node AI devices (500′) installed in the respective areas.

[0299] When each edge AI device (100-1 to 100-4) is configured as a head AI device (500), they receive data from a plurality of node AI devices (500′) in the corresponding area through a communication unit (220), analyze the data in a processor unit (230), and then communicate with a server device (400) or a head AI device (500) in another area to link and transmit the vehicle (10)'s movement history, status, reservation information, etc., in real time.

[0300] As described above, the head AI device (500) can be configured to communicate with the server device (400), and the server device (400) acts as a medium to provide necessary information to the user by communicating with the user terminal (300). Specifically, the head AI device (500) can transmit data such as vehicle identification information (30), serial number information (40), boarding status, and waiting status collected and processed to the server device (400), and the server device (400) can process and relay the received information to provide it to the user terminal (300). That is, the user terminal (300) receives information indirectly through the server device (400), and the user checks the information through a visual output means such as a graphical user interface (GUI) of an application or an electronic display board. The server device (400) integrates and manages information received from a plurality of head AI devices (500) and is configured to provide real-time status information, waiting order, boarding availability, and estimated arrival time of the vessel to the user in response to a request from the user terminal (300).

[0301] FIG. 8b is a drawing for explaining user guidance information and administrator guidance information according to an embodiment of the present invention.

[0302] According to one embodiment of the present invention, user guidance information (22) is generated by a server device (400) or a head AI device (500) and transmitted to a user terminal (300), thereby providing real-time information related to ship operation and vehicle boarding to a driver or manager. The type and scope of information included in the user guidance information (22) may vary depending on the use of the user terminal (300) and the level of user authority, and may be broadly classified into information for general users and information for managers.

[0303] User guidance information (22) displayed on a general user terminal (310), such as a vehicle driver's mobile application or vehicle display, includes information that helps the user intuitively understand vehicle boarding status information (22-1) indicating whether their vehicle is currently available for boarding, remaining waiting time information (22-2), time until ship arrival information (22-3), and real-time ship loading status information (22-4). Such user guidance information (22) serves to guide the user to move their vehicle effectively without causing unnecessary waiting or congestion within the dock. Additionally, user guidance information (22) can also be visually provided through a public general user terminal (310), such as an electronic display board.

[0304] According to one embodiment of the present invention, vehicle boarding status information (22-1) is a sub-component of user guidance information (22) and is information intended to provide real-time guidance to a driver via a general user terminal (310) on whether their vehicle is currently available for boarding on a vessel. The vehicle boarding status information (22-1) is determined by combining multiple conditions such as vehicle serial number information (40), reservation status, entry time, waiting order, and the loading status of the vessel, and the user can immediately check whether their vehicle is available for boarding in this round or if they need to wait for the next round.

[0305] According to one embodiment of the present invention, vehicle boarding status information (22-1) is calculated based on a serial number range. For example, if information such as "boarding possible up to serial number 0299" is displayed on the user terminal for the current vessel, and the user's vehicle has a serial number such as "R0123," it is determined to be in a boarding status based on that information. Specifically, vehicle boarding status information (22-1) can be visually output through various user terminals (300), such as an application screen, an in-vehicle display, or an electronic display board, and can be provided as graphics such as text, a display, or a mark indicating boarding possible / impossible. Additionally, vehicle boarding status information (22-1) is updated in real time and can change according to the estimated arrival time of the vessel, the number of vehicles already boarded, and the entry progress of currently waiting vehicles. In some embodiments, a push notification may be provided to the user terminal (300) at the time of change of boarding status information, or the vehicle status may be dynamically switched in a map-based UI.

[0306] According to one embodiment of the present invention, the remaining waiting time information (22-2) is a sub-component of the user guidance information (22) and is information intended to guide the driver, via a general user terminal (310), the expected remaining waiting time until the vehicle can actually board the vessel. The remaining waiting time information (22-2) prevents the user from waiting unnecessarily at the current location and provides practical decision-making assistance data to efficiently guide the flow of vehicles within the dock.

[0307] According to one embodiment of the present invention, the remaining waiting time information (22-2) is calculated by combining multiple elements such as the vehicle serial number information (40), the current boarding progress rate of the vessel, the estimated time of arrival of the vessel, the number of vehicles ahead of the order, and the estimated time required for boarding. For example, if there are 300 vehicles waiting to board at the dock and vehicles up to serial number 0300 are currently eligible for boarding, the vehicle with serial number 0500 may be informed as "Waiting time prediction: Approximately 30 minutes after boarding begins."

[0308] According to one embodiment of the present invention, the remaining waiting time information (22-2) may be displayed as text or graphics in an application UI or on an electronic display board within a general user terminal (310), and in some embodiments, the time may be dynamically counted down according to a real-time update cycle, or the waiting stage may be visualized as a color or icon. For example, as the waiting time approaches, the color may change to green or automatically switch to a 'ready to board' state, thereby intuitively assisting the driver's judgment.

[0309] Additionally, since the remaining waiting time information (22-2) can be calculated individually based on different conditions for each vehicle (whether it is a reserved vehicle, vehicle type, priority status, etc.), the remaining waiting time information (22-2) provided through the general user terminal (310) functions as guidance information customized to the individual driver's vehicle conditions. In some embodiments, pop-up notifications, entry guidance notifications, and boarding call messages based on the remaining waiting time information (22-2) may be provided in conjunction.

[0310] According to one embodiment of the present invention, the time information until ship arrival (22-3) is a sub-component of the user guidance information (22) and is information that informs the driver, via a general user terminal (310), of the estimated time remaining until the ship arrives at the dock based on the current location. Specifically, the time information until ship arrival (22-3) can provide the user with a sense of time for waiting or preparing to move when the ship has not yet arrived or is in port. Additionally, the time information until ship arrival (22-3) may include time information for the ship to arrive at the destination dock after the vehicle has boarded the ship.

[0311] According to one embodiment of the present invention, time information (22-3) until the arrival of a ship is calculated by a server device (400) by comprehensively considering the ship's real-time location information, route progress speed, current weather and sea conditions, and scheduled arrival timetable, and is updated at regular intervals and transmitted to a user terminal (300).

[0312] According to one embodiment of the present invention, real-time coordinate data is received from a GPS or location transmitting device installed on the ship, and the remaining time until arrival is calculated based on this data and provided to the user in the form of "approximately 18 minutes until arrival of the ship." Additionally, the time information (22-3) until arrival of the ship can be displayed as text on the user terminal (300) or implemented as visual elements according to the flow of time, such as a progress bar, color change, or icon flashing. In particular, it is utilized so that users waiting to board can intuitively understand the time of arrival of the ship even on a fixed user terminal such as an electronic display board. For example, when arrival is imminent in 5 minutes, a notification may be sent in the form of a push or highlighted and displayed at the top of the user screen.

[0313] According to one embodiment of the present invention, real-time ship loading status information (22-4) is a sub-component of user guidance information (22) and is information intended to guide the driver in real-time by visualizing the loading status of vehicles currently loaded inside the ship through a general user terminal (310). Specifically, the real-time ship loading status information (22-4) can serve as a practical criterion for the driver to intuitively recognize the loading progress inside the ship and to determine whether their vehicle will soon be eligible for boarding or if they will have to wait for additional time.

[0314] According to one embodiment of the present invention, real-time ship loading status information (22-4) is based on data collected from cameras, vehicle sensors, weight detection devices, etc. installed on the ship, and is collected and aggregated through a server device (400) or a head AI device (500) and transmitted to a user terminal (300). The real-time ship loading status information (22-4) generally consists of the floor structure within the ship, such as 4F and 5F, the number of vehicles loaded on each floor, remaining loading space, and the use of dedicated areas such as a motorcycle area and a large vehicle-only area.

[0315] According to the present invention, real-time ship loading status information (22-4) can ideally be provided based on data collected in real time through equipment such as cameras, vehicle sensors, and weight detection devices installed inside the ship. However, since equipment such as cameras, vehicle sensors, and weight detection devices installed inside the ship is often not installed, it is practically difficult to directly sense this.

[0316] Accordingly, based on detection information from an edge AI device (third edge AI device (100-3)) positioned at a location capable of detecting vehicle flow in the ship loading area, the system can be configured to estimate and visually represent an arbitrary loading state based on the number of vehicles entering, vehicle length, etc. This configuration is a real-time vehicle flow-based estimation method and has the effect of intuitively providing predictive information regarding the ship loading state to a user or manager.

[0317] According to one embodiment of the present invention, a graphic representing the cross-sectional structure of a ship is displayed on a general user terminal (310), and the ratio (130 / 300) of the number of vehicles loaded in each zone and the total number of vehicles that can be accommodated is displayed. In some embodiments, a visual effect is implemented in which the remaining space is gradually filled through color changes or animations, such as blue -> yellow -> red, allowing the user to intuitively understand the current progress. In addition, real-time ship loading status information (22-4) plays an important role in inducing psychological stability and predictive behavior in the user. For example, if there is enough remaining space, the user can wait comfortably, and conversely, if the space is nearly full, the user can make an immediate decision to move the vehicle. In addition, real-time ship loading status information (22-4) can be displayed in parallel on a fixed general user terminal (310), such as an electronic display board, as well as on a user app, and can also be utilized as detailed statistics for each floor on a user terminal (320) for administrators.

[0318] On the other hand, the administrator's user terminal (320) provides more advanced administrator guidance information (23). In this case, the administrator guidance information (23) may include judgment information necessary for managing overall operations, such as reservation and reception status information (23-1) indicating the number of reserved vehicles and reception completion status, on-site waiting vehicle information (23-2), range of vehicles permitted to board (23-3), ship loading rate and remaining space information (23-4), list of vehicles eligible for priority boarding (23-5), vehicle flow statistics information (23-6), and ship arrival / departure time information (23-7). According to some embodiments, the administrator terminal may also perform control functions such as setting the vehicle entry permission range, changing priorities, and delivering internal messages based on the administrator guidance information (23).

[0319] According to one embodiment of the present invention, the reservation and reception status information (23-1) is a sub-component of the administrator guidance information (23) and is information for checking and managing vehicle reservation information for boarding a ship and the status of acceptance of said reservation in real time through a user terminal (320) for the administrator. The reservation and reception status information (23-1) allows the ship operator or the pier manager to intuitively understand the total number of reserved vehicles, whether acceptance is complete, the status of acceptance delays, and the distribution of reservation time slots, thereby providing reference data for boarding flow control and queue management.

[0320] In this embodiment, the reservation and reception status information (23-1) may include information such as the total number of reserved vehicles, such as 250 vehicles reserved as of today; the number of vehicles with completed reception, such as 180 vehicles that actually arrived at the dock and were confirmed; the number of reserved vehicles that have not yet arrived, such as 70 reserved vehicles that have not yet entered; the distribution of arrivals by reservation time slot, such as concentration between 14:00 and 15:00; and the classification of reservation types, such as general vehicles, large cargo trucks, and emergency loading vehicles.

[0321] According to one embodiment of the present invention, reservation and reception status information (23-1) is generated in real time by matching reservation data from a server device (400) or a head AI device (500) with entry information, such as vehicle identification information (30) received from a first edge AI device (100-1), and the reservation and reception status information (23-1) can be visually checked in the form of a table, graph, bar chart, or color-coded list on a user terminal (320) for an administrator. In addition, the system can be configured to provide a warning notification (Popup or Notification) to the administrator if the reservation and reception status information (23-1) is delayed beyond a certain threshold or if a reserved vehicle is omitted, and it is also possible to execute control commands such as adjusting entry priority, calling a specific vehicle, or setting priority boarding in conjunction with this.

[0322] According to one embodiment of the present invention, on-site waiting vehicle information (23-2) is a sub-component of manager guidance information (23), and is an information item configured to check real-time information about vehicles currently waiting in a dock or waiting area through a manager user terminal (320), and to effectively determine the boarding order, guidance path, and waiting space distribution based on this information.

[0323] According to one embodiment of the present invention, on-site waiting vehicle information (23-2) is collected through a plurality of node AI devices (500') and head AI devices (500), including identification information, location information, waiting time, reservation status, vehicle type and size, etc. of vehicles present in the waiting area, and then integratedly processed by a server device (400) and transmitted to an administrator terminal. Specifically, the on-site waiting vehicle information (23-2) is implemented to count the number of waiting vehicles in real time and display them as "Current waiting vehicles: 312," or to separate reserved vehicles and non-reserved vehicles to distinguish and sort the queue.

[0324] In addition, the on-site waiting vehicle information (23-2) can be sorted by serial number for each vehicle or visually displayed as a distribution within designated areas such as A1, B2, and C3. Furthermore, the location of the vehicle can be visualized as a pin or block on a map-based UI, and the status of each vehicle, such as whether it is reserved, arrival time, or available for boarding, can be displayed using colors or icons, allowing the manager to intuitively grasp the overall waiting flow. Accordingly, the manager can make judgments such as which vehicles to allow entry to, whether a specific area is overcrowded or underutilized, whether the ratio of reserved vehicles to non-reserved vehicles is appropriate, and where to place priority when calling passengers.

[0325] According to one embodiment of the present invention, the vehicle range information (23-3) authorized for boarding is a sub-component of the administrator guidance information (23) and is configured to display and set the serial number range or conditions of vehicles currently allowed to board the vessel through the administrator's user terminal (320). The vehicle range information (23-3) clearly informs the administrator which vehicles among the vehicles waiting on-site can actually board this voyage, and allows the range to be manually adjusted or restricted.

[0326] According to one embodiment of the present invention, the vehicle range information (23-3) for boarding permission can be automatically calculated by comprehensively considering the remaining cargo space within the vessel, total vehicle length, loading rate by floor, priority of reserved vehicles, and operation delay information collected in real time by the server device (400) or the head AI device (500). Basically, among the vehicles sorted by serial number, the boarding candidates are determined based on the order of arrival or reservation priority. For example, information such as "Current boarding permission range: Serial number R0001 ~ R0250" may be displayed on the administrator user terminal (320).

[0327] According to one embodiment of the present invention, a UI is provided that allows an administrator to manually adjust the range of permitted boarding. Depending on specific situations, such as prioritizing boarding for emergency vehicles, responding to missed reservations, or excluding specific vehicles, boarding conditions can be set as exceptions for individual vehicles or prioritized and excluded on a group basis. Additionally, the information on the range of permitted boarding vehicles (23-3) can be transmitted to a general user terminal (310) and utilized as information on whether boarding is possible (22-1). It can also be reflected in real-time on an electronic display board, etc., and functions as a standard for intuitively guiding waiting vehicles on whether boarding is possible.

[0328] According to one embodiment of the present invention, the ship loading rate and remaining space information (23-4) is a sub-component of the manager guidance information (23) and is information provided to enable the manager to determine the real-time status of the number of vehicles currently loaded on the ship and the available loading space through a user terminal (320) for the manager. Specifically, the ship loading rate and remaining space information (23-4) quantitatively indicates the status of space utilization inside the ship, and is used as reference data for the manager to calculate the number of vehicles that can be boarded and to set the boarding permission range information (23-3).

[0329] According to one embodiment of the present invention, the ship loading rate and remaining space information (23-4) is calculated based on the loading status by floor within the ship collected by the server device (400) or the head AI device (500), the length and type of each vehicle, the use of pre-assigned dedicated areas such as a motorcycle area and a dedicated area for disabled vehicles, etc. Additionally, the ship loading rate and remaining space information (23-4) may include information such as the current usage relative to the total loading space, a loading rate by floor such as 85% for the 4th floor and 62% for the 5th floor, the number of remaining vehicles that can be loaded such as about 40, an estimated loading time, or an estimated closing time.

[0330] According to one embodiment of the present invention, the ship loading rate and remaining space information (23-4) can be visually displayed on a user terminal (320) for an administrator using a diagram of the ship's cross-section, bar graph, colors, etc., and if the loading rate exceeds a certain standard, a warning display or an automatic boarding restriction suggestion function can be linked. In addition, the ship loading rate and remaining space information (23-4) is useful for immediately identifying discrepancies between the actual loading status and the reserved vehicles, the unused status of specific areas, and whether specific floors are overloaded, and provides the administrator with a basis for real-time judgment to adjust boarding flow or modify ship loading strategies. Furthermore, the ship loading rate and remaining space information (23-4) can be utilized in establishing a plan for the next voyage, and the server device (400) can be linked with high-dimensional analysis functions such as predicting loading patterns and space optimization modeling based on accumulated data.

[0331] According to one embodiment of the present invention, the priority boarding target vehicle list information (23-5) is a sub-component of the administrator guidance information (23), and is information that selects vehicles to be boarded onto the vessel as a priority from among all waiting vehicles through the administrator's user terminal (320) and displays them in the form of a list. The priority boarding target vehicle list information (23-5) is used to identify and manage vehicles that are exceptionally designated separately from the range of vehicles permitted to board (23-3) for vehicles that satisfy specific conditions or require priority processing according to the administrator's judgment.

[0332] According to one embodiment of the present invention, a vehicle eligible for priority boarding may be one of the following: a vehicle with an urgent priority request at the time of reservation, a special purpose vehicle such as an emergency vehicle, a military support vehicle, or a vehicle for the disabled, a vehicle eligible for delay compensation, a vehicle that was omitted from a previous boarding, or a vehicle recommended for priority boarding through analysis by a server device (400). That is, the server device (400) or the head AI device (500) can select vehicles eligible for priority boarding based on the above conditions, and the administrator can directly review the list through the UI on the administrator's user terminal (320) and perform manual addition or exclusion settings if necessary.

[0333] According to one embodiment of the present invention, the priority boarding vehicle list information (23-5) is generally provided to the administrator in a table format including the vehicle number, serial number, vehicle type, reason for request or priority designation, and time of designation. In some embodiments, the vehicle is visually distinguished using highlighting, a separate color, or an icon to enable quick recognition within the administrator interface. Additionally, the priority boarding vehicle list information (23-5) is linked with an exception handling logic when setting the boarding permission vehicle range information (23-3), so that even if a vehicle exceeds the current range, boarding the vessel is permitted if it is included in the priority boarding list. In some embodiments, the user terminal (300) may also be notified that it is a "priority boarding target" in the form of a pop-up notification or a push message.

[0334] According to one embodiment of the present invention, the priority boarding vehicle list information (23-5) is information that identifies vehicles that are exceptionally allowed to board a ship based on the judgment of an administrator or server device (400) among vehicles outside the existing range of vehicles permitted for boarding (23-3), generates a list thereof, and allows the administrator to check it in real time through a terminal. Such priority boarding vehicles may include vehicles that are not reserved or within the normal range but are eligible for delay compensation, vehicles with emergency transport purposes such as military, fire, police, or emergency vehicles, vehicles carrying disabled or elderly passengers where priority boarding is required according to ship operation policies, vehicles determined by the server device (400) to be priority boarding vehicles after continuous waiting, or exceptional vehicles directly designated by the administrator considering the on-site situation.

[0335] Vehicles designated as priority boarding vehicles may be exceptionally allowed to board a vessel even if they fall outside the current permitted boarding vehicle range information (23-3). For example, even if the vehicle's serial number is ahead or behind the current boarding range, if it is included in the priority boarding list, it is processed to be included as a boarding target without being filtered during the vessel loading process. Additionally, the priority boarding vehicle list information (23-5) includes items such as the vehicle number, serial number, vehicle type, reason for priority designation, and time of designation, and can be distinguished by visualization using a separate color or highlighting on the UI of the administrator terminal. Furthermore, a pop-up notification or electronic display guide may be displayed on the user terminal (300) so that the person scheduled to board the vehicle can recognize that it is a priority boarding target.

[0336] According to one embodiment of the present invention, the ship arrival / departure time information (23-7) is a sub-component of the manager guidance information (23) and is information provided to allow the manager to check the real-time or planned sailing schedule, including the scheduled arrival and departure times of the ship, the actual arrival / departure times, and whether there is a delay, through a user terminal (320) for the manager. The ship arrival / departure time information (23-7) functions as a time-based operational indicator that serves as a basis for various judgments, such as setting the boarding allowance range, controlling entry, dispersing queues, and preparing for the next voyage, by clearly allowing the manager to recognize the reference point of the entire ship operation flow.

[0337] According to one embodiment of the present invention, the ship arrival / departure time information (23-7) is calculated by integrating the ship's GPS-based real-time location data, a sailing schedule table, and navigation status information received from an AI device or a sailing control system within the ship at the server device (400). The administrator user terminal (320) may include items such as the scheduled arrival time / actual arrival time, the scheduled departure time / actual departure time, the arrival or departure standard delay time, and the estimated arrival time for the next voyage. Additionally, the ship arrival / departure time information (23-7) is provided in the UI of the administrator user terminal (320) in a graphic form, such as text, a timeline, a color change, or a delay notification display. When a delay is detected, suggestions for adjusting the boarding permission range, judgments for accepting additional waiting vehicles, and notifications for adjusting the next voyage may be provided in conjunction.

[0338] According to one embodiment of the present invention, the ship arrival / departure time information (23-7) is linked with on-site control and user guidance information (22), so that if the arrival is delayed, the waiting time guidance is automatically updated to the user, and if the departure time is changed, the boarding guidance time is automatically adjusted. In addition, the ship arrival / departure time information (23-7) can be accumulated as a log and utilized as subsequent analysis data, such as statistics on operational punctuality and analysis of operational efficiency by voyage.

[0339] FIG. 8c is a diagram illustrating the communication structure between the control server (410) and a plurality of head AI devices.

[0340] As described above, the control server (410) may be configured to communicate with the first head AI device (500-1), the second head AI device (500-2), and the third head AI device (500-3) installed at each of the plurality of terminals. For example, the first head AI device (500-1) is installed at Terminal A, the second head AI device (500-2) is installed at Terminal B, and the third head AI device (500-3) is installed at Terminal C, respectively, and aggregates data collected from the node AI devices (500') of each terminal and transmits it to the control server (410). Specifically, the first head AI device (500-1), the second head AI device (500-2), and the third head AI device (500-3) installed at each of the plurality of terminals may collect and organize vehicle status information (411) within the respective terminal, that is, information such as the type, quantity, location, and identification number of the vehicle currently located at the terminal, and transmit it. Additionally, the results of analyzing the terminal's real-time entry queue, average waiting time, distance between vehicles, and throughput may be included as waiting and congestion information (412), and the number of vehicles that have completed boarding on the vessel, total loading length, loading rate, etc., may be transmitted to the control server (410) as loading status information (413).

[0341] According to one embodiment of the present invention, loading status information (413) refers to status information regarding vehicles actually loaded on a ship, and is generated based on data collected within the jurisdictional area by a first head AI device (500-1), a second head AI device (500-2), and a third head AI device (500-3) installed at each terminal. Specifically, the loading status information (413) may include whether a vehicle has actually been loaded on the ship at the ship boarding point, the number of vehicles that have been loaded, the type and length of the vehicle, and information on the loaded location.

[0342] According to one embodiment of the present invention, in cases where no sensor is installed inside the vessel, the loading status information (413) is generated by detecting the entry of a vehicle through a camera, sensor, etc., at the time the vehicle passes the boarding point of the vessel, and thereby considering the loading to be complete. Additionally, the list of loaded vehicles can be updated in real time based on the vehicle serial number information (40) and vehicle identification information (30).

[0343] According to one embodiment of the present invention, the control server (410) can centrally control the real-time operational status of each terminal and perform efficient resource distribution and operational strategies by comprehensively analyzing information received from a plurality of head AI devices (500). For example, according to the waiting and congestion information (412) received through the first head AI device (500-1), if the number of waiting vehicles at Terminal A increases rapidly and the average waiting time exceeds a threshold, the control server (410) may determine that measures are needed to reduce the congestion of Terminal A.

[0344] On the other hand, information regarding the waiting and congestion levels (412) of Terminal B received through the second head AI device (500-2) can be confirmed to be relatively relaxed, with few vehicles currently waiting to board and sufficient processing capacity. In this case, the control server (410) can transmit a command to the server device (400) to distribute some of the vehicles to Terminal B. In accordance with this command, the server device (400) can provide a guidance message to some of the waiting vehicles saying "Please move to Terminal B" through the application of the user terminal (300) or the on-site electronic display, or directly transmit instructions to the on-site operator through the manager terminal (320).

[0345] In this way, the control server (410) can go beyond simply passive data reception functions and perform the role of a centralized operational control tower, such as active judgment and instruction execution based on real-time information, including vehicle flow control, guidance of distributed boarding between terminals, and readjustment of priorities. Through this, the entire system can prevent unbalanced vehicle density and maintain a more efficient waiting environment for boarding ships.

[0346] According to one embodiment of the present invention, real-time boarding status information (20-1) by terminal is information that is synthesized and organized based on loading-related data received by a server device (400) from a first head AI device (500-1), a second head AI device (500-2), and a third head AI device (500-3) installed at each of a plurality of terminals, and is integrated information that reflects the ship boarding progress status of each terminal in real time. Specifically, the first head AI device (500-1), the second head AI device (500-2), and the third head AI device (500-3) transmit vehicle identification information (30), loading determination information, a list of vehicles that have completed boarding, and vehicle lengths, etc., collected from their respective node AI devices (500') to the server device (400), and the server device (400) can integrate this to calculate the boarding progress status organized by terminal. Real-time boarding status information (20-1) by terminal may include the cumulative number of boarded vehicles by terminal, total length loaded, boarding rate, number of remaining vehicles available, list of vehicles waiting to board, etc., and may be updated in real time including changes over time.

[0347] According to one embodiment of the present invention, vehicle statistical information (20-2) is statistical data generated by a server device (400) based on vehicle-related information collected from a plurality of terminals, and is provided to a control server (410) so that it can be utilized for the operation patterns of each terminal, customer usage behavior, resource allocation, and the establishment of future operation plans. Specifically, the vehicle statistical information (20-2) may include information such as the number of vehicles entering by terminal and period, the ratio of non-reserved vehicles to reserved vehicles, peak time analysis and vehicle concentration analysis by time of day, the boarding ratio by vehicle type (passenger car, truck, trailer, etc.), vehicle length distribution and average length, boarding completion rate and boarding failure rate, and the repeat visit history of a specific customer (for identifying regular customers).

[0348] According to one embodiment of the present invention, a server device (400) can receive detailed data such as vehicle identification information (30), serial number information (40), and boarding status in real time from a first head AI device (500-1), a second head AI device (500-2), and a third head AI device (500-3) of each terminal and a node AI device (500') connected to the first head AI device (500-1), the second head AI device (500-2), and the third head AI device (500-3), and classify and accumulate the data to perform statistical analysis.

[0349] According to one embodiment of the present invention, the reservation and reception processing status information (20-3) is information generated by integrating reservation information, on-site reception information, and reception progress status collected by the server device (400) for each terminal, and represents the flow of reservation-based operations of each terminal in real time. Specifically, the server device (400) classifies the number of vehicles with completed reservations, the distribution of reservation time zones, vehicles with reservations that have not arrived, the number of vehicles received on-site, the number of vehicles waiting for reception, and information on vehicles rejected for reception, in conjunction with vehicle identification information (30) and serial number information (40), and can organize the current reservation and reception progress status based on this. In addition, the reservation and reception processing status information (20-3) is transmitted to the control server (410) and utilized as basic data for operational decision-making, such as predicting reservation demand, adjusting reception processing priorities, providing administrator notifications, and determining early reservation closure for specific terminals. Furthermore, it can contribute to improving terminal operational efficiency through the analysis of reservation concentration in specific time zones and the identification of reception bottleneck sections.

[0350] According to one embodiment of the present invention, the estimated waiting time and congestion information (20-4) refers to information that predicts the waiting time and spatial congestion within the terminal at the present or near future point in time, based on the result of analysis by the server device (400) based on real-time vehicle flow data received from the first head AI device (500-1), second head AI device (500-2), and third head AI device (500-3) and each node AI device (500') of each terminal. Specifically, the estimated time for how long each vehicle must wait until boarding is calculated by comprehensively considering factors such as vehicle entry time information (34), the current number of waiting vehicles, the total length of vehicles, the speed of boarding progress, the estimated time of arrival of the vessel, and the dispatch interval, and the congestion can be quantified by analyzing the density of the waiting space, the lack of space, and the vehicle congestion sections.

[0351] According to one embodiment of the present invention, the server device (400) transmits the estimated waiting time and congestion level information (20-4) to the control server (410), thereby enabling central monitoring of congestion conditions, distribution measures between terminals, and readjustment of the boarding allowance range, and supports real-time guidance or control through the user terminal (300) and the administrator terminal (320). In addition, by visually providing the current estimated waiting time or congestion level to the user through an electronic display or app notification, both user satisfaction and on-site responsiveness can be improved.

[0352] According to one embodiment of the present invention, user feedback / status information (20-5) refers to direct input information or status change data of a user transmitted to a server device (400) via a user terminal (300) or an administrator terminal (320). Specifically, user feedback / status information (20-5) may include user feedback information on the app, such as dissatisfaction with waiting time, reporting of app errors, and confirmation of boarding completion; user status selection information, such as "arriving soon" or "expected to be late"; user location information collected from the user terminal (300); and app usage log information.

[0353] According to one embodiment of the present invention, the server device (400) collects user feedback / status information (20-5) and transmits it to the control server (410), thereby enabling real-time improvement of on-site response speed, adjustment of expected waiting time, redistribution of boarding priority, adjustment of electronic display messages, etc.

[0354] According to one embodiment of the present invention, emergency notification information (20-6) is information containing a notification message regarding exceptional and urgent situations that may occur at a dock, terminal, ship, traffic, weather conditions, etc., and is generated by a server device (400) or a control server (410) and transmitted to a user terminal (300), an administrator terminal (320), an electronic display board, etc. For example, there may be delays in ship departure due to bad weather, accidents within a terminal, reservation system errors, security issues, power outages, and communication failures, and when such situations occur, they are delivered to the user in the form of an emergency pop-up notification, an emergency message on an electronic display board, an app push notification, etc. That is, the server device (400) stores the sending log and response results of the emergency notification, and the control server (410) can centrally control follow-up measures according to the situation.

[0355] According to one embodiment of the present invention, the priority boarding target processing result information (20-7) is information that records and analyzes whether a vehicle was actually boarded normally with respect to a list of priority boarding target vehicles generated in advance by a server device (400) or a control server (410). Specifically, the priority boarding target processing result information (20-7) includes the entry time of the vehicle included in the list, the time of boarding the vessel, the assigned vessel information, whether boarding actually took place, the reason for refusal (such as insufficient space), and details of the action taken.

[0356] According to one embodiment of the present invention, the electronic display / app guidance message record information (20-8) includes historical data such as electronic display messages, app push notifications, and pop-up messages generated and transmitted by the server device (400) or the control server (410). Specifically, the electronic display / app guidance message record information (20-8) may include the time of transmission of the message, the recipient (electronic display location, user ID), the message type (general / urgent), the exposure time, whether there is a user response, etc.

[0357] As described above, the control server (410) can receive various data, such as vehicle status information (411), waiting and congestion information (412), and loading status information (413), from the first to nth head AI devices (500-1, 500-2, ..., 500-n) and the server device (400) installed at each of the multiple terminals, and can determine the terminal operation status in real time, and can derive advanced operation analysis information by comprehensively analyzing this information.

[0358] Specifically, the control server (410) can derive operational efficiency analysis information for each terminal based on the average loading rate, vehicle entry / exit processing speed, waiting time, and trends in congestion levels of each terminal, and can generate statistical information based on user patterns by analyzing data such as the user's repeated usage history, preferred time slots, and routes. In addition, it is possible to extract regional dispersion effect analysis information by comparing the results of dispersing some vehicles to nearby terminals in a situation of excessive congestion at a specific terminal.

[0359] According to one embodiment of the present invention, the terminal-specific operational efficiency analysis information refers to information calculated by comprehensively analyzing the vehicle entry / exit processing speed, vessel loading rate, frequency of congestion occurrence, and turnover rate at each terminal based on various real-time and historical data received by the control server (410) from the first to nth head AI devices (200-1, 200-2, ...) and the server device (400). For example, there may be cases where a specific terminal maintains a low level of congestion while processing more vehicles within the same time period, and such data can be used as a basis for comparing operational efficiency by terminal, optimizing the allocation of manpower and resources, and determining future expansion or policy changes.

[0360] According to one embodiment of the present invention, user pattern-based statistical information refers to statistical information that analyzes the behavioral patterns and trends of regular customers based on the preferences, usage frequency, and preferred time periods of individual vehicles and users tracked through a user terminal (300) or vehicle identification information (30). The control server (410) learns whether a specific vehicle repeatedly uses the same time period or the same route, and whether usage is concentrated according to a specific day of the week or season, and can utilize this to establish customized service policies such as priority boarding policies for regular customers, enhanced advance notifications, and the provision of mileage-based benefits. In addition, user pattern-based statistical information can also contribute to reviewing whether to improve or abolish inefficient sections by identifying time periods or routes with low usage rates.

[0361] According to one embodiment of the present invention, the regional dispersion effect analysis information refers to the result of analyzing the effect of dispersing vehicles according to the congestion and waiting conditions between multiple terminals, and is derived based on the number of waiting vehicles, loading rate, actual vehicle movement history, etc., received by the control server (410) from each head AI device and server device. For example, if the congestion level of Terminal A is excessively high, the effectiveness of the dispersion induction, the effect of improving loading speed, and the degree of reduction in waiting time can be quantitatively determined by guiding some vehicles to Terminal B and comparing the real-time changes in congestion levels between the two terminals. Such regional dispersion effect analysis information can be utilized in policy design, such as setting automated guidance policies when similar situations occur in the future or automatically sending dispersion recommendation messages to user terminals (300), and contributes to establishing a preemptive strategy based on central control for balanced operation between regions.

[0362] FIG. 10 is a diagram illustrating the arrangement structure and information flow of an edge AI device during the process of boarding and disembarking a vehicle from a ship according to one embodiment of the present invention.

[0363] As described above, the vehicle enters from the dock on the left and travels via ship to the destination dock on the right, and edge AI devices (100) are deployed stepwise at key points along the travel path to continuously track and manage the status of the vehicle. Specifically, the first edge AI device (100-1) corresponding to INPUT 1 (10-1) is installed at the entrance of the dock to first recognize the vehicle entering the dock, obtain vehicle identification information (30), and then assign a unique serial number information (40) to the vehicle. Subsequently, when the vehicle (10) passes through the interior of the dock to board the ship, the second edge AI device (100-2) corresponding to OUTPUT 1 (10-2) recognizes the vehicle (10) again, thereby verifying whether the vehicle (10) has moved to the actual ship boarding area and reporting to the server device (400).

[0364] Just before boarding the vessel, a third edge AI device (100-3) corresponding to INPUT 2 (10-3) is installed to re-verify the vehicle number information (31) and serial number information (40) at the moment the vehicle (10) enters the vessel, and to record the boarding time and whether boarding is complete. After the vessel arrives at the destination, a fourth edge AI device (100-4) corresponding to OUTPUT 2 (10-4) detects the time when the vehicle (10) disembarks, processes the entire movement path of the vehicle (10) to terminate, and can provide final disembarkation information to the server device (400).

[0365] Meanwhile, a third edge AI device (100-3) is positioned at the ship boarding point on the drawing, and the third edge AI device (100-3) is installed as a single device to detect the flow of vehicles at the ship boarding point. That is, instead of configuring separate entry and exit devices in a dual manner, it can be implemented in a way that a single edge AI device identifies and tracks all vehicles passing through the section. This configuration is efficient in terms of installation space and cost, and in the actual processing, data classification is performed by logically distinguishing the direction in which vehicles enter (Input) and the direction in which they exit after disembarking (Output) at the ship boarding point.

[0366] Accordingly, the third edge AI device (100-3) is installed as a single hardware device, and by recognizing and separating the direction of data (boarding / disembarking) in the internal processor or server linkage logic, it can accurately track and manage the flow of vehicles according to boarding and disembarking of the ship.

[0367] FIG. 11 is a drawing illustrating an example of an application user interface (UI) of a user terminal (300) according to one embodiment of the present invention.

[0368] As illustrated, the user terminal (300) may be a mobile device, an in-vehicle display, or other portable electronic device, and can visually output information regarding boarding of a vehicle in real time based on data received from the server device (400). The illustrated information "Boarding available up to serial number 0299" clearly indicates the range of vehicles currently available for boarding on the vessel, thereby allowing the user to immediately determine whether they are eligible to board based on their serial number. Additionally, the illustrated item "Time remaining until arrival: 20 minutes" indicates the real-time location and estimated arrival time of the vessel at the current dock, and is based on navigation status information collected from the edge AI device (100) and the server device (400).

[0369] In addition, the displayed information, "Waiting Estimation: 30 minutes after boarding begins," allows users to recognize the expected remaining waiting time and adjust their movement or preparation accordingly. Furthermore, the bottom section includes visualizations of the boarding status for each deck of the ship, providing a diagrammatic representation of the usage status of vehicle loading spaces. For example, the "Motorcycle Zone" displayed on the 5th deck indicates that space has been pre-allocated for a specific group of vehicles, while the number of loaded vehicles ("130 / 300") represents the ship's current capacity. Through this, users can intuitively understand information such as which zone their vehicle can be placed in and how much spare space remains.

[0370] According to one embodiment of the present invention, the user terminal (300) can be implemented to receive real-time notifications regarding whether a ride has been called for the vehicle, the arrival of a waiting turn, and emergency announcements through a pop-up notification of an app. In parallel, the same information is also provided in parallel to a user terminal (300) in the form of an electronic display installed inside the dock, so that users who do not use a mobile app can also visually perceive the same information.

[0371] FIG. 12 is a drawing illustrating an example of a user interface (UI) of a manager terminal (320) according to one embodiment of the present invention.

[0372] As described, the administrator terminal (320) is a device used by a system administrator, such as a dock operator or a ship operation manager, for the purpose of monitoring and determining in real time the number of reserved vehicles, on-site waiting vehicles, available vehicles, and ship status. Meanwhile, the driver terminal (320) may also be included as an example of a user terminal (300). That is, the user terminal (300) can be implemented as an application for general drivers as well as an operation terminal for administrators, and the range and level of information displayed may be differentiated according to the user's role. This structure provides a flexible user environment that can satisfy the needs of various users while maintaining consistency in terminal configuration.

[0373] The illustrated screen example visually represents key information that an administrator can check through the administrator terminal (320). For example, the number of reserved vehicles (120) and the number of vehicles that have been accepted (80) are displayed at the top, allowing the administrator to check the status of reservation acceptance in real time. Subsequently, the number of vehicles waiting on-site (300) and the number of vehicles currently available for boarding (180) are provided together, enabling the administrator to quickly decide how many vehicles to allow entry to and how to adjust the queue. Additionally, the real-time loading status of each deck (4F, 5F) of the vessel is provided at the bottom in a diagrammatic form, allowing the administrator to immediately check the remaining space on the vessel, vehicle loading density, and the status of dedicated area allocation (motorcycle-only area).

[0374] FIG. 13 is a drawing showing an example of implementation of a user terminal (300) in the form of an electronic display board according to one embodiment of the present invention.

[0375] As described above, an electronic display board installed on the approach road to the dock provides real-time guidance to drivers approaching via the route by visually displaying information such as the real-time terminal waiting status, the range of vehicles available for boarding, the time remaining until the ship's arrival, and the estimated waiting time. Specifically, the electronic display board may also be included as an example of a user terminal (300). The user terminal (300) is generally implemented in the form of a mobile application, but is not limited thereto, and may include various visual output means such as a fixed display device, an electronic notification board, or a roadside guide indicator. Such a form allows the driver to intuitively recognize whether their vehicle is available for boarding and the status of ship operations without opening an app or without the application being installed.

[0376] For example, the display board of FIG. 13 sequentially displays information such as “Roading available up to serial number 0299”, “20 minutes until arrival of the ship”, and “Waiting time prediction - 30 minutes after boarding begins”, and the displayed information can be updated by receiving it from the server device (400). This can be implemented as a flow in which data collected and analyzed by the head AI device (500) is relayed to the display board through the server device (400) to provide real-time information to the driver.

[0377] Such an electronic display-based user terminal (300) functions as a means of determining the waiting strategy of entering vehicles, preventing unnecessary congestion, and providing information that can be autonomously determined without instructions from an administrator, and can be utilized as a mutually complementary information provision channel with a mobile app-based user terminal. In addition, by providing information from the electronic display and the mobile app in a synchronized manner, the reliability of the information and user satisfaction can be improved together.

[0378] FIG. 14 is a drawing illustrating an example of a user terminal (300) that provides terminal waiting information through an upper electronic display board of a dock entry gate according to one embodiment of the present invention.

[0379] As described above, before a vehicle intending to enter the dock enters the gate lane, multiple electronic displays are installed on the upper front of the vehicle, and through these displays, information such as the serial number of currently available vehicles, the estimated time of ship arrival, predicted waiting times, and real-time boarding status is displayed in real time.

[0380] According to one embodiment of the present invention, a gate display board within the dock may also be included as an example of a user terminal (300). The display board of FIG. 14 is updated based on information received from a server device (400) and can perform the function of providing visual guidance so that the user (driver) can immediately check whether they are eligible to board, the status of the ship's operation, the remaining waiting time, etc. Specifically, in the embodiment of FIG. 14, the real-time boarding status is displayed in parallel on the front of the vehicle in a diagrammatic form along with the display board information, and information such as remaining space and specific areas (motorcycle-only areas) inside the 4th and 5th floors of the ship is also provided, thereby providing the driver with the opportunity to reconfirm the boarding conditions and make an appropriate judgment at the stage immediately before entry.

[0381] According to one embodiment of the present invention, a gate electronic display-based user terminal (300) can be utilized as a supplementary channel synchronized with information provided by a mobile application or in-vehicle navigation, and can function as an effective means to provide boarding information of an equivalent level even to non-app users, foreign visitors, or environments where data communication is limited. In addition, from the perspective of an administrator, the effect of naturally controlling vehicle flow and preventing queue congestion can be obtained through the terminal in front of the gate.

[0382] FIG. 15 is a diagram showing the arrangement and functional relationship of a head AI device (500) installed at a vehicle entry point (Input 1) at the entrance of a dock and a plurality of node AI devices (500') connected thereto, as an example of installation of a first edge AI device (100-1) according to an embodiment of the present invention.

[0383] As described, the head AI device (500) is composed of a main control device including a fixed camera installed on the upper structure of the dock entrance, and performs the function of comprehensively collecting and processing vehicle identification information (30) for all vehicles entering from that location. The head AI device (500) corresponds to the first edge AI device (100-1) of the present invention and functions as a higher control device for the dock entrance area.

[0384] Meanwhile, the node AI device (500') can be installed on the side of the entrance, near the ground, or on an individual lane basis, and each node AI device (500') has a camera or sensor module built in to collect detailed data such as license plate recognition, length measurement, and location identification for individual vehicles. That is, the node AI device (500') can transmit the collected information to the head AI device (500) positioned above it.

[0385] According to the present invention, the head AI device (500) can integrate and analyze information received from a plurality of node AI devices (500'), assign a unique serial number information (40) to each vehicle (10), and store identification information such as whether the vehicle (10) is reserved and the entry time in a memory unit (540) or transmit it to a server device (400).

[0386] According to one embodiment of the present invention, the illustrated first edge AI device (100-1) can measure the total length of a vehicle by utilizing reference coordinates in virtual space or corrected distance information based on an image at the moment the vehicle (10) enters. Specifically, after detecting the position from the point where the front wheels of the vehicle enter to the rear end, an accurate vehicle length measurement value can be calculated by utilizing a preset virtual calibration line or lane-based scale information.

[0387] This method enables separate recognition of connection units and summation of total lengths even in cases where the structure is connected, such as with trailer vehicles, and floor markings at positions 1 through 6 can serve as reference points. This virtual space-based measurement can provide highly reliable length information despite the absence of fixed sensors and can be utilized as basic data for future real-time load distribution strategies, ship space layout planning, and fuel optimization operations.

[0388] FIG. 16 is a diagram illustrating a zone-based sensing structure of a node AI device (500') and an information linkage method with a head AI device (500) according to an embodiment of the present invention.

[0389] As described above, a plurality of node AI devices (500') include surveillance cameras installed inside the dock or above the waiting area, and each node AI device (500') can monitor the location, number, stopping status, movement path, etc. of a vehicle in real time based on a specific assigned surveillance area and generate vehicle identification information (30).

[0390] As described above, each node AI device (500') may have an individual field of view and monitoring area, and each node AI device (500') may be configured to recognize only vehicles (10) within a certain range. This method has the advantage of reducing errors in distinguishing between vehicles (10) and minimizing interference between sensors. Additionally, each node AI device (500') may acquire vehicle identification information (30) and transmit it to a head AI device (500) located at a higher level. Furthermore, the head AI device (500) may perform various high-level decisions, such as determining whether a vehicle (10) can board within the dock, setting boarding priorities, inducing congestion relief, and distributing loading areas, by integrating and analyzing the information received from multiple node AI devices (500') in real time.

[0391] FIG. 17 is a drawing for illustrating an embodiment in which a plurality of head AI devices within a specific terminal of the present invention are arranged by zone.

[0392] The embodiment illustrated in FIG. 17 indicates that multiple head AI devices can be arranged by zone within a single terminal. For example, in Terminal A, a first head AI device (501), a second head AI device (502), and a third head AI device (503) may be installed at different locations to manage their respective assigned zones. The first head AI device (501), the second head AI device (502), and the third head AI device (503) do not merely perform the role of collecting data, but can refine and analyze information from each zone and transmit it to a server device (400) or a control server (410). In particular, as multiple head AI devices are operated in parallel, detailed zone-specific situations can be independently determined within a single terminal, and boarding strategies optimized for each zone, such as vehicle alignment priority and boarding guidance, can be executed.

[0393] FIG. 18 is a drawing for more specifically explaining the configuration and operation method of an edge AI device (100) that can be utilized in the first or second embodiment of the present invention.

[0394] As described, the configuration shows that a plurality of sensor units (111 to 114) included in the edge AI device (100) are installed at various locations within the stadium to independently collect video data. While FIG. 8a focuses on the relationship and collaboration structure between the head AI device (500) and a plurality of node AI devices, FIG. 18 highlights an embodiment in which a single edge AI device (100) itself includes a plurality of sensor units and can collect video from various viewpoints.

[0395] Specifically, the edge AI device (100) includes a plurality of sensors constituting the sensor unit (110), such as the first sensor unit (111) to the fourth sensor unit (114), and the first sensor unit (111) to the fourth sensor unit (114) are positioned at different locations on the field, such as near the goal, the central area, and the side, to capture the game from various angles. The first sensor unit (111) to the fourth sensor unit (114) collect video data (201) in real time, and the collected video data (201) can be processed integrally within the edge AI device (100) and utilized for preprocessing tasks such as object tracking and behavior analysis. The structure of FIG. 18 is significant in that it allows the edge AI device (100) itself to collect video data (201) in real time and process it integrally within the edge AI device (100) without the need for a separate node AI device (500′).

[0396] According to one embodiment of the present invention, a head AI device (500) or an edge AI device (100) may store voice relay data (19a-1) generated based on the generated primary behavior analysis result data (3-2), secondary behavior analysis result data (4-2), and text data (205), as well as the primary behavior analysis result data (3-2) and secondary behavior analysis result data (4-2), in a server device (400). The data stored at this time may include the form of identification information of an object, analysis results regarding location and movement patterns, purpose-based behavior estimation data of the object, text and voice files for voice relay, etc.

[0397] The voice relay data (19a-1), primary behavior analysis result data (3-2), and secondary behavior analysis result data (4-2) stored in this manner can be utilized in various subsequent analysis services provided on a cloud basis in the future. For example, the data collected and accumulated through this system can be meaningfully applied to team power analysis that quantifies the strategic movements and power level of a specific team, player stat analysis that evaluates the behavioral patterns and key contributions of individual players, and referee evaluation systems based on factors such as judgment consistency and game flow judgment.

[0398] The voice relay data (19a-1), primary behavior analysis result data (3-2), and secondary behavior analysis result data (4-2) stored in this manner are not only information for real-time relay, but can also be used as a learning basis for an AI analysis system that becomes more advanced over time. Therefore, the edge AI infrastructure of the present invention can be expanded beyond its function as a relay device into an advanced data platform infrastructure for the entire soccer ecosystem.

[0399] According to one embodiment of the present invention, voice relay data (19a-1), primary behavior analysis result data (3-2), and secondary behavior analysis result data (4-2) generated through the edge AI device (100) and head AI device (500) are stored in a server device (400) and can be used as a data source for a team power analysis system to quantitatively evaluate the strategy and power level of a specific team in the future.

[0400] According to one embodiment of the present invention, primary behavior analysis result data (3-2) includes location information, movement direction and speed of individual players or objects, and basic behavioral units such as breakthrough attempts, defensive pressure, and positional changes before passing based on the movement. Based on this, the server device (400) and the edge AI device (100) can extract the basic tactical reaction patterns of a specific player and aggregate them by team to analyze the type and frequency of tactical operations. For example, the server device (400) and the edge AI device (100) can quantitatively determine whether a specific team frequently attempts build-up from the back or mainly uses quick counterattacks after defense.

[0401] According to one embodiment of the present invention, the secondary behavior analysis result data (4-2) is data that expands the primary behavior analysis result data (3-2) into a complex objective-based behavior sequence by combining it with information on the reaction of opposing objects, such as the time continuity or opposing team players or the ball. This is useful for evaluating the execution of detailed strategies, such as tactical selection in specific situations, attempts at forward pressing, or counterattack interception strategies, and can be used as an indicator of the team's decision-making speed, transition responsiveness, and strategic cohesion. In addition, by extracting specific repeating sequences, consistency of game operation and strategy success rate can be analyzed.

[0402] According to one embodiment of the present invention, voice relay data (19a-1) is data including text and voice generated based on analysis results, and serves as situational description and annotation data at the time of the broadcast. Since voice relay data (19a-1) is information that organizes meta-information about a player's key actions in natural language along the time axis, it provides the context of each action's occurrence and an explanatory interpretation at the time. Voice relay data (19a-1) can provide auxiliary information for an analyst or an AI-based statistical system to backtrack the tactical meaning of the corresponding action or to infer the coaching staff's intentions in a specific situation. Additionally, through voice data, non-quantitative elements such as emotional emphasis or urgency at the time of the actual broadcast can also be reflected in the analysis.

[0403] According to one embodiment of the present invention, during the voice relay process, primary behavior analysis result data (3-2), secondary behavior analysis result data (4-2), voice relay data (19a-1), and text data (205) can be stored in a cloud storage by a server device (400), and such a storage structure provides long-term and multifaceted potential for utilization beyond simple real-time relay. First, since the primary behavior analysis result data (3-2), secondary behavior analysis result data (4-2), voice relay data (19a-1), and text data (205) include the linkage between individual behavior units at a specific point in time and the overall flow of the game, they can be reused for team power analysis that comprehensively analyzes a specific team's tactical patterns, strategic response types, and coordination among players even after the game ends. In particular, it is possible to quantify team power based on quantitative data such as repeated tactical execution, action success rate, and movement density, and this can also be used as an input variable for a game outcome prediction algorithm.

[0404] Furthermore, each player's behavior logs and related text / voice data can be used to evaluate individual players' tendencies, tactical adaptability, changes in contribution at specific positions, pressing intensity, and initiative movements as time accumulates, which can be utilized as highly reliable statistical data in the player transfer market.

[0405] Furthermore, behavioral data and audio commentary records stored along with logs of ruling-related situations during a match can be utilized to extract various evaluation metrics, such as referee consistency, reaction speed, and positional accuracy. Consequently, this data also holds the potential to serve as the basis for official formulas to be provided to referee evaluation systems.

[0406] As such, the present invention has great technical flexibility and industrial applicability in that it accumulates data collected from a real-time infrastructure for voice relay in the cloud through a server device (400), thereby enabling expansion into an analysis-centered service based on the accumulated data over time. In particular, since the depth and reliability of the accumulated data are more important than real-time performance for such subsequent analysis services, a centralized analysis structure based on a cloud-based server device is suitable.

[0407] The edge AI device (100) of the present invention goes beyond the role of a simple video and audio relay device and functions as a core preprocessing node for real-time data processing, collection and refinement of multi-sensing information, and AI analysis. In particular, the edge AI device (100) performs the role of an outpost for a complete AI data pipeline that immediately performs a primary analysis on-site of raw data collected from multiple sensor units within the stadium, converts it into text and voice data, and then connects with a server device (400) to perform cloud-based long-term analysis.

[0408] According to one embodiment of the present invention, the edge AI device (100) is a core infrastructure that enables AI-based services throughout the entire soccer ecosystem, such as team power analysis, player evaluation, referee decision evaluation, and building a trust base for the transfer market, in addition to the primary purpose of providing real-time broadcasting, and serves as a starting point for generating quantified analysis indicators for data-based decision-making.

[0409] Therefore, in the patent strategy of the present invention, rather than limiting the edge AI device (100) to a simple "relay device," it is important to position it as an "AI infrastructure device," "behavioral data collection device," "real-time analysis node," etc., thereby securing a broad scope of rights that encompasses various services and the entire scalable ecosystem. This serves as a strategic line of defense that protects the entire data-based sustainable AI sports platform technology without being dependent on a single service such as voice relay.

[0410] The object-based voice relay system (1) according to the present invention not only analyzes major events and player motion data occurring during a match in real time and relays them as voice, but also includes the possibility of expanding linkage with existing referee decision videos (VAR videos).

[0411] In some venues, the issue is being raised that the quality of existing VAR footage is insufficient, and accordingly, a method to link VAR footage data is being discussed. The edge AI device (100) and server device (400) of the present invention process high-resolution multi-angle footage data obtained from sensor units (111 to 114) placed at multiple locations in the stadium in real time, and based on the data, quantitative analysis of a player's position, speed, contact situation, and behavior is possible. Here, the integrated footage sequence (202) may also include VAR footage data corresponding to a specific event segment that occurred during the match, and can be utilized for evaluating the appropriateness of referee decisions and for assisting in decision analysis.

[0412] That is, the integrated video sequence (202) refers to a video sequence based on the overall flow of the game, in which individual video data collected in real time from multiple sensor units (111 to 114) or multiple node AI devices (500') within the stadium is aligned and aligned along the time axis. The integrated video sequence (202) includes full-area data centered on the position and movement of players, the ball, and referee objects, and serves as the basis for generating AI processing-based data such as voice relay or behavior analysis according to the present invention. On the other hand, VAR (Video Assistant Referee) video data is a short segment of video selected for the purpose of reviewing specific judgment situations such as fouls or offsides, and is video data dedicated to assisting referee judgments to which high-speed playback, various viewpoint videos, and slow-motion processing are applied. Therefore, VAR video data can be viewed as derivative content extracted and processed from a portion of the integrated video sequence, and while the integrated video sequence (202) and VAR video data can be linked with each other, their uses and scopes are clearly distinguished.

[0413] According to one embodiment of the present invention, VAR video data generated by an edge AI device (100) or a head AI device (500) is currently stored in a server device (400) primarily consisting of structured behavioral analysis data and voice relay data, but may be stored including actual match video data as needed in the future. In particular, the stored data can be utilized for various subsequent analyses beyond simple broadcasting purposes, such as team tactical analysis, player stat evaluation, and referee decision reliability analysis.

[0414] Specifically, since chronological movement information of each object, such as players and referees, is stored in the form of video data or coordinate-based data, numerical indicators such as velocity and acceleration based on coordinate changes are automatically calculated. These indicators can be utilized to estimate whether mobility declines over time, that is, to determine the point at which physical stamina is depleted. For instance, if a specific player exhibits a pattern of repeatedly decreasing acceleration from the middle of the first half onwards, that segment can be automatically tagged as a stamina decline segment, which can then be used as a standard for constructing individual player stamina profiles and adjusting team tactics.

[0415] According to one embodiment of the present invention, even in the case of a referee, the accuracy of judgment and physical condition can be simultaneously estimated based on data such as location, movement path, and reaction time at the time of a specific ruling, and thus can be utilized as quantitative evidence for future referee evaluation systems. Such analysis enables highly reliable, data-driven judgments rather than relying solely on subjective evaluations, and can contribute to the evaluation of player value in the future transfer market and ensuring fairness in referee assignments.

[0416] These analysis results can be used to complement the video-based qualitative judgment provided by the VAR system or as a basis for auxiliary quantitative judgment when reviewing incorrect decisions. For example, the primary and secondary behavioral analysis result data (3-2, 4-2) of the present invention can quantify acceleration, velocity changes, and movement paths before and after contact in specific collision situations, thereby contributing to increasing the accuracy and reliability of VAR reviews. In addition, such data is stored in the cloud via a server device (400) and can be extended later to verify the fairness of referee decisions, establish referee evaluation criteria at the federation level, and serve as a supplementary tool for post-reviews.

[0417] Therefore, given that the present invention complements the limitations of existing VAR systems and also offers the potential to serve as a data-based quantitative assistance system for ensuring fairness in referee decisions, securing the scope of rights for the data generation and linkage structure for assisting referee decisions can also be considered in terms of patent strategy.

[0418] FIG. 19a is a block diagram illustrating the overall processing structure and data flow of an object-based voice relay system according to one embodiment of the present invention.

[0419] As described above, the object-based voice relay system (1) includes a plurality of node AI devices (500'). The plurality of node AI devices (500') each film a match to be broadcast, such as a soccer match, in real time from different viewpoints and angles, and each node AI device (500') transmits the video data (201) acquired in real time to the head AI device (500). At this time, each node AI device (500') operates as an edge device specialized in video acquisition and transmission, and has a structure that minimizes processing burden by considering real-time performance and network efficiency.

[0420] Additionally, the object-based voice relay system (1) can be implemented not only by receiving video data from a plurality of node AI devices (500′), but also by receiving video data (201) from a plurality of first sensor units (111) to fourth sensor units (114) that are included in a single edge AI device (100). In this case, the plurality of first sensor units (111) to fourth sensor units (114) are installed at different locations within the stadium to collect real-time video from various viewpoints and angles, and the collected video data (201) is directly integrated and processed within the edge AI device (100) and transmitted to the head AI device (500).

[0421] The method of receiving image data (201) from each of the multiple first sensor units (111) to fourth sensor units (114) enables single-device-based operation in terms of installation and management of the object-based voice relay system (1), and can be effectively applied even in environments where network infrastructure is limited. Furthermore, by having the edge AI device (100) consistently manage the time information provided by each sensor unit, it also provides the advantage of improving object tracking accuracy and the reliability of behavior analysis.

[0422] According to one embodiment of the present invention, various behavioral analysis data and voice broadcast data generated for voice broadcasting are not limited to temporary use but can be stored as basic data for subsequent context-based analysis. For example, information such as a specific player's behavioral patterns in a specific situation, timing of behavioral transitions, changes in speed, and positional relationships can be accumulated even after the match ends and utilized for analyzing behavioral patterns and tactical preferences by player and team.

[0423] Based on data accumulated in this manner, the server device (400) or edge AI device (100) can reflect behavioral context information predicted from past data in real-time broadcasting when similar game situations occur in the future. Furthermore, based on the analysis result that Player A has a habit of repeatedly taking the action of "side breakthrough followed by a cross" in specific situations, more sophisticated intention estimation and priority determination can be performed when such action is detected in the future. To this end, the server device (400) can configure a data feedback loop that can improve the accuracy of real-time analysis during a game by storing the data generated after voice broadcasting processing in a predefined format and subsequently providing it back to the edge AI device (100) or head AI device (500).

[0424] According to one embodiment of the present invention, a head AI device (500) can perform an image synthesis step (S210) based on image data (201) received from a plurality of node AI devices. Specifically, the head AI device (500) can align the received plurality of image streams based on time axis and viewpoint information and generate a single integrated image sequence that integrates views from multiple angles.

[0425] According to one embodiment of the present invention, the image synthesis step (S210) is a process of aligning multiple image data (201) received in real time from multiple node AI devices (500') based on time axis and time point information, and aligning frames of multiple time points corresponding to the same time period to generate an object-centered integrated image sequence. For example, it can be assumed that multiple node AI devices (500') placed at different locations within the stadium are each capturing a specific object, player number 11, from different viewing angles. For example, if the first node AI device (500') captures the player's front (front view), the second node AI device (500') captures the rear (back number), and the third node AI device (500') captures the side (running side view), each node AI device (500') generates an independent image frame for that time point.

[0426] At this time, the head AI device (500) receives video data of each time point as described above through the communication unit (520), and sorts and matches the frames based on timestamp information, shooting location and direction metadata, and object identification information included in each of these video frames. Then, the head AI device (500) analyzes multiple frames taken at the same time point, merges the front, rear, and side video data of player number 11 on an object basis, and creates a consistent viewpoint-based integrated video frame by removing or sorting background and unnecessary duplicate data. That is, the head AI device (500) can refine unstructured video obtained at multiple times points through the video synthesis step (S210) to create an object-centered integrated sequence video.

[0427] FIG. 19b is a diagram illustrating an integrated video sequence and object-specific continuous tracking data of an object-based voice relay system according to an embodiment of the present invention.

[0428] According to one embodiment of the present invention, the object detection and tracking step (S220) is a step of detecting specific objects, such as players, balls, and referees, from an integrated video sequence (202) generated through an image synthesis step (S210), and managing them by assigning a unique identifier (ID) to consistently track the path of movement of the objects over time.

[0429] First, the head AI device (500) detects candidate objects of interest, such as people and balls, within the integrated video through the processor unit (530). At this time, since it includes multi-angle viewpoint information based on the integrated video sequence (202), which is omnidirectional video data, the resolution and reliability regarding the shape, size, and movement of the object are significantly improved compared to the existing single-viewpoint-based detection method.

[0430] At this time, the detected object (203-1) is registered with a unique identifier (ID), which is a unique identification ID (203-2), and tracking is performed to maintain consistency of IDs for the same object that appears continuously along the time axis. For example, when player number 11 is detected in the integrated video, the unique identification ID (203-2) ID#11 is assigned to the player, and subsequently, based on change information such as position, size, speed, and direction of movement in the video frames, it is determined that the detected object (203-1) corresponds to the same object and is continuously tracked with the unique identification ID (203-2) ID#11.

[0431] At this time, since collisions or overlaps between objects may occur, various correction techniques such as 3D position estimation using multiple viewpoints, skeleton analysis, and color and pattern-based auxiliary recognition may be used together.

[0432] In particular, in sports game environments where rapid interaction between the ball and the player is frequent, the reliability of result interpretation depends on the frame-by-frame detailed position estimation and tracking accuracy, so the head AI device (500) can be configured to solve this continuity problem through an object tracking algorithm based on high-speed parallel computation.

[0433] According to one embodiment of the present invention, the object primary behavior analysis step (S230) is a step of quantitatively analyzing the short-term behavior pattern of each detected object (203-1) based on object-specific continuous tracking data (204) including information such as location, speed, and direction obtained through the preceding object detection and tracking step (S220). The primary behavior analysis of the detected object (203-1) is mainly performed on a short time interval of a few seconds, such as 510 frames or 0.51 seconds, and is centered on the amount of change between the current state and the previous state of the detected object (203-1).

[0434] For example, if player number 11 shows a movement of rapidly entering the center from the left side at a specific point in time, the head AI device (500) can assign a behavior tag of “accelerated breakthrough from left side to center” based on changes in position, direction of movement, and increase or decrease in speed within that time interval.

[0435] The primary behavior analysis input data (3-1) used at this time may include spatial coordinate and movement data (3-1-1) including information such as time series coordinates (x, y) of each object, distance traveled, speed, and acceleration; relative position and interrelationship data (3-1-2) including information such as distance between a specific object and adjacent objects, direction of entry, and whether they intersect; object pattern data (3-1-3) including information such as player type characteristics such as past behavior pattern history, pass frequency, and dribbling tendency; and context data (3-1-4) including information such as game time, zone information, and previous event.

[0436] According to one embodiment of the present invention, a head AI device (500) combines primary behavior analysis input data (3-1) to extract unit behavior clues such as stopping, running, changing direction, ball possession, and collision avoidance performed by each detection object (203-1), and generates qualitative and quantitative analysis results for the behavior at that time.

[0437] The results of such primary behavior analysis are stored in the form of recording a sequence of behavioral changes over time for each detected object (203-1), and this is used as base data for secondary behavior analysis and tactical interpretation performed by the server device (400) thereafter.

[0438] The object-based voice relay system (1) of the present invention can be classified into two representative embodiments depending on the conditions of the relay environment and the purpose of use. Specifically, the first embodiment (C1) corresponds to field-centered real-time relay (based on on-device processing), and the second embodiment (C2) corresponds to server device (400)-based online relay.

[0439] First, the first embodiment (C1) is a method in which a head AI device (500) receives video data (201) from a plurality of node AI devices (500'), performs video synthesis (S210), object detection and tracking (S220), and primary object behavior analysis (S230), and then generates voice relay data (19a-1) internally and transmits it directly to a user terminal (300). In this case, data transmission to a server device (400) is omitted, and since the entire process is performed autonomously at the edge, latency is minimized, and a highly reliable relay method capable of operating even in offline environments can be implemented. This is suitable for environments where network infrastructure is limited or immediate responsiveness is required, such as inside a stadium or at a sports venue.

[0440] On the other hand, the second embodiment (C2) transmits primary behavioral analysis result data (3-2) performed by the head AI device (500) to the server device (400) in a manner that includes online relay or cloud-based high-dimensional analysis, and the server device (400) performs secondary behavioral analysis, text and voice conversion processing based on this, and then generates voice relay data (19a-1) and transmits it to the user terminal (300) via the voice relay server (600). This case is suitable for online service environments where relay is provided to multiple users simultaneously or where complex sentence generation and content optimization are required. Since the server has high computational power and is capable of fusion with external data, it has the advantage of enabling more sophisticated interpretation and the provision of diversified relay content.

[0441] As such, the present invention provides both an edge AI terminal-based real-time field processing method (first embodiment) and a server-based online multi-user processing method (second embodiment), thereby possessing structural advantages that allow for flexible response to various operational scenarios. The first embodiment will be described in detail below.

[0442] According to one embodiment of the present invention, the first behavior analysis result data transmission step (S240) includes the process of transmitting the first behavior analysis result data (3-2) generated by the head AI device (500) through the object first behavior analysis step (S230) to the server device (400). Specifically, the head AI device (500) extracts short-term behavior information for each object, such as a player or a ball, based on video data (201) received from a plurality of node AI devices (500'), and composes the first behavior analysis result data (3-2), which is structured analysis data including an object identifier (ID), time information, behavior tag, spatial coordinates, speed and direction information, relative relationship, etc. At this time, the first behavior analysis result data (3-2) for each detected object (203-1) is organized into a data sequence arranged in chronological order and may include the time and duration of the behavior occurrence, the type of behavior, and context information.

[0443] According to the present invention, the server device (400) performs a high-dimensional content creation operation based on primary behavior analysis result data (3-2) received from the head AI device (500).

[0444] Step (S21) of generating text data based on primary behavior analysis result data

[0445] According to one embodiment of the present invention, the step (S21) of generating text data based on primary behavior analysis result data is a step of converting primary behavior analysis result data (3-2) input from a head AI device (500) into descriptive text in the form of natural language that can be delivered to a user. The text data generation step (S21) is performed in a high-performance computing environment of a server device (400), and various Natural Language Processing (NLP) techniques and language generation models are utilized.

[0446] According to one embodiment of the present invention, a server device (400) first analyzes object identification information, location coordinates, movement direction and speed, whether there is interaction, specific event detection information, etc., included in the primary behavior analysis result data (3-2), and performs a preprocessing process for the specific meaning and contextual interpretation of the behavior. Subsequently, it converts the behavior into a human-readable sentence structure by utilizing a deep learning-based natural language generation model, such as a behavior template-based narrative method or a Transformer-type text generation model, to express the behavior of each object.

[0447] For example, if the first behavior analysis result data (3-2) includes analysis information such as “Object A (ID#11, attacking midfielder) steals the ball from Object B (ID#5, defender), breaks forward 7.2m, and is accelerating toward the goal,” the first behavior analysis result data (3-2) can be generated as explanatory text such as “Player Kim Hyun-woo steals the ball from the defender and starts a solo breakthrough. He is advancing toward the goal at high speed.”

[0448] The text data (205) generated in this way is used as input for the subsequent Text-to-Speech (TTS) processing, and additional natural language processing algorithms for improving content quality, such as not only simple action descriptions but also sentence length adjustment, emotion emphasis, and diversification of expressions according to tactical importance, can be applied in parallel.

[0449] Text data and voice data matching step (S22)

[0450] According to one embodiment of the present invention, the text data and voice data matching step (S22) is a process for converting explanatory text data (205) generated in the step (S21) of generating text data based on primary behavior analysis result data into actual audible voice content, and is a step of configuring voice data (206) using text-to-speech (TTS) technology or a pre-acquired voice sample library.

[0451] According to one embodiment of the present invention, a server device (400) first segments text data (205) into sentence units or semantic units, and then includes meta-information to provide appropriate speech style, intonation, speed, and emotion information for each segment. Subsequently, the text data (205) to which meta-information has been provided can be converted into speech using a TTS-based matching method. Specifically, the server device (400) inputs the text data (205) into a deep learning-based TTS engine such as Tacotron, FastSpeech, or VITS to synthesize speech data (206) in real time. The TTS-based matching method has the advantage of enabling flexible generation for various situations and sentences, and does not require prior voice recording.

[0452] According to one embodiment of the present invention, a server device (400) can match voice data (206) to text data (205) by utilizing a pre-built voice sample-based matching method. Specifically, the server device (400) can generate voice relay data (19a-1) by combining voice fragments most suitable for the text data (205) by referring to a pre-built sentence-unit or word-unit voice sample database. In this case, the actual voice of the speaker can be preserved, and refined voice expression equivalent to broadcast quality is possible. In particular, in situations such as sports broadcasting, emotional emphasis such as “It is a dangerous situation!” or “It is an opportunity!”, speed control such as rapid reading in urgent scenes, and the application of an intonation model that matches the tone of the broadcast become important factors, so the selection of a voice expression method that considers the content context can be carried out in parallel with simple voice generation.

[0453] Secondary behavior analysis result data generation step (S22-1)

[0454] According to one embodiment of the present invention, the second behavior analysis result data generation step (S22-1) is a step of generating higher-dimensional cognitive interpretation information by additionally analyzing the interaction between detection objects (203-1), contextual information on the flow of the game, and tactical structure, based on the first behavior analysis result data (3-2) received from the head AI device (500).

[0455] According to one embodiment of the present invention, a server device (400) can interpret complex movements between multiple detection objects (203-1), such as between players or between a player and the ball, by comprehensively considering temporal continuity and spatial relationships for input primary behavioral analysis data, and can derive secondary behavioral analysis result data (4-2). Specifically, the server device (400) can generate tactical situation classification data (4-2-1) that classifies which tactical context the current situation corresponds to, such as counterattack development, side switching, breaking through a compact defense, or avoiding pressure. The tactical situation classification data (4-2-1) can be generated based on various tactical indicators, such as the location of the field, the placement of players, the direction of movement, and the trajectory of the ball.

[0456] According to one embodiment of the present invention, a server device (400) can generate behavioral purpose / intention estimation data (4-2-2) based on input primary behavioral analysis result data (3-2). Specifically, the behavioral purpose / intention estimation data (4-2-2) may be data that estimates what purpose a specific player's action is based on, such as advancing with a dribble, securing space for a pass, luring a defender, or preparing to attempt a shot. The server device (400) can generate the behavioral purpose / intention estimation data (4-2-2) by considering the direction of gaze, changes in speed, surrounding pressure information, etc.

[0457] According to one embodiment of the present invention, a server device (400) can generate behavior importance / priority determination data (4-2-3) based on input primary behavior analysis result data (3-2). Specifically, the behavior importance / priority determination data (4-2-3) may be data that quantifies the importance of the analyzed behavior in the flow of the game, for example, whether it is a threatening behavior with a high probability of scoring, and determines the priority to be included in the explanation content. At this time, importance may be determined by complex factors such as location, opponent defense density, and game time.

[0458] According to one embodiment of the present invention, a server device (400) can generate content extraction element data (4-2-4) based on input primary behavior analysis result data (3-2). Specifically, the content extraction element data (4-2-4) may be data for extracting descriptive content components, such as keywords, emphasis points, and exclamation expressions, for generating actual broadcast content. For example, the content extraction element data (4-2-4) can enrich descriptive expressions such as “lightning-fast steal,” “movement that created an opportunity,” and “decisive dribble that broke through the defense.” That is, the server device (400) of the present invention can perform an advanced recognition function that interprets the strategic intentions of multiple objects and the resulting meaningful changes in the game. Accordingly, the server device (400) can improve the quality and immersion of the final voice broadcast data (19a-1).

[0459] Voice relay data generation and transmission step (S23)

[0460] According to one embodiment of the present invention, the voice relay data generation and transmission step (S23) is a process of transmitting the voice relay data (19a-1) finally generated by the server device (400) to a voice relay server (600) rather than directly transmitting it to a user terminal (300), and corresponds to the second embodiment of the present invention (i.e., an online relay scenario). Specifically, the voice relay data (19a-1) generated by the server device (400) is composed of, for example, voice descriptions of game scenes, commentary, highlight descriptions, etc., and is in a state that takes the form of relay content to be provided to the user in real time. However, in this embodiment, in order to effectively handle real-time multiple transmission and load distribution to various user terminals (300), the server device (400) adopts a structure in which it does not directly transmit the voice relay data but transmits it to a dedicated voice relay server (600).

[0461] Meanwhile, in the second embodiment of the present invention, the configuration of transmitting voice relay data (19a-1) generated by the server device (400) to the user terminal (300) via the voice relay server (600) has been described primarily, but it is not limited thereto. In some embodiments, a configuration in which the server device (400) transmits the voice relay data (19a-1) directly to the user terminal (300) without passing through the voice relay server (600) is also possible. For example, in relays limited to a specific user group, in an environment with a simple network infrastructure, or in an independent service structure that does not require a separate voice relay server, the server device (400) can provide real-time relay by directly transmitting the voice relay data (19a-1) to the user terminal (300). Therefore, it is clarified that all configurations in which the server device (400) transmits the voice relay data (19a-1) to the user terminal (300) directly or indirectly are included within the spirit and scope of the present invention.

[0462] According to one embodiment of the present invention, the voice relay server (600) transmits the received voice relay data (19a-1) to the user terminal (300) in real time, and can additionally perform network delay correction, user custom settings (language, tempo, etc.), cache-based transmission, etc. if necessary. Therefore, the voice relay data generation and transmission step (S23) can serve as an intermediate hub within the online relay system in that it is not merely a simple data transmission step, but serves as a final relay point to ensure real-time distribution of voice content and user experience quality.

[0463] FIG. 20 is a block diagram showing the detailed configuration of an object primary behavior analysis step (S230) according to an embodiment of the present invention. Specifically, FIG. 20 illustrates a primary behavior analysis process performed by a head AI device (500) based on an integrated video sequence (202) received from a plurality of node AI devices (500').

[0464] Spatial Coordinates and Transformation Data (3-1-1)

[0465] According to one embodiment of the present invention, primary behavior analysis input data (3-1) is input to the head AI device (500), and the primary behavior analysis input data (3-1) includes spatial coordinates and movement data (3-1-1). Specifically, the spatial coordinates and movement data (3-1-1) includes time-based information related to the position and movement of specific objects such as players and balls. The spatial coordinates and movement data (3-1-1) is generated by integrating and processing multi-angle image data acquired in real-time by a plurality of node AI devices (500') in the head AI device (500).

[0466] According to one embodiment of the present invention, the head AI device (500) can represent each object with unique coordinate values ​​(x, y, z) at each point in time and calculate velocity and direction of movement (vector) from the temporal change of the corresponding coordinates. For example, in the case of player number 11, dynamic movement characteristics such as acceleration, deceleration, sharp turn, and stopping can be quantitatively analyzed through position changes between consecutive frames. In addition, spatial coordinates and movement data (3-1-1) can be interpreted in conjunction with area information where the detected object (203-1) is located, such as the central area or near the penalty box, as well as the absolute coordinate system of the field, and the movement trajectory aligned along the time axis can form base data for subsequent context analysis or strategic position judgment.

[0467] Relative position and cross-relationship data (3-1-2)

[0468] According to one embodiment of the present invention, relative position and interrelationship data (3-1-2) is input data used to identify positional and operational relationships between multiple objects in the object primary behavior analysis step (S230) of the present invention. Specifically, the relative position and interrelationship data (3-1-2) includes the mutual distance, arrangement structure, relative direction of movement, and whether there is an intersection of multiple objects, such as players and a ball, at the same point in time or at adjacent points in time on the time axis. For example, if attacking midfielder A and defender B are located within a certain distance and the relative speed and approach angle between the two objects exceed a certain standard, it can be interpreted as a "standoff situation" or a "ball contest situation." Such interpretation is impossible with simple position data alone, but is possible through a fusion analysis of various parameters such as the rate of change in distance between objects, the amount of change in angle, and the duration of interaction.

[0469] Additionally, the head AI device (500) can consider the significance of tactical placement when analyzing the interrelationships between objects. For example, the head AI device (500) can consider patterns such as whether defenders are arranged at specific intervals or whether multiple defenders are focusing on marking one object during the process of pressing the defensive line while the attacker possesses the ball as relative position and interrelationship data (3-1-2).

[0470] Object Pattern Data (3-1-3)

[0471] According to one embodiment of the present invention, object pattern data (3-1-3) is input data used in the object primary behavior analysis step (S230) of the present invention, and is information that quantifies and expresses 'characteristic behavior patterns,' such as behavior habits over time, repetitive movements, and unique reaction tendencies of individual objects. The object pattern data (3-1-3) is not limited to short-term location and velocity information, but is calculated by analyzing the accumulated movement history and the behavioral patterns at the current time together.

[0472] For example, if a player with jersey number 11 has a tendency to change direction momentarily just before receiving the ball, or a tendency to penetrate the left space at a speed greater than a certain level during an attack, and this is repeatedly confirmed through past match data, the head AI device (500) can reflect this information in the object pattern data (3-1-3) as the player's unique movement pattern.

[0473] Additionally, the head AI device (500) can classify specific dribbling styles, such as speed acceleration after an inside cut used by a specific player to beat an opposing defender, common direction of vision movement after stealing the ball, and pass selection tendencies as object pattern data (3-1-3). Such patterns can be usefully utilized by the head AI device (500) to predict the behavior of the object in the real-time data flow, or to determine whether the behavior is within a normal range of movement or is an abnormal (exceptional) behavior.

[0474] Contextual data (3-1-4)

[0475] According to one embodiment of the present invention, context data (3-1-4) is input data for considering the temporal, spatial, and tactical background context in which an action occurred, rather than interpreting the action of an object in the object's first action analysis step (S230) of the present invention based solely on its location or movement. The context data (3-1-4) includes various external conditions such as game time, detailed location area within the field, whether there is an attack / defense transition, ball possession status, scoring situation, tactical pressure based on remaining time, and whether it occurs immediately after a major event such as a corner kick.

[0476] For example, even for the same 'side penetration' action, the priority of its tactical significance and interpretation can vary completely depending on whether it occurred in the 5th minute of the first half or in added time just before the end of the second half, whether it took place during a transition from one's own half to the opponent's half, or whether the team had possession or was recovering defensively. Such differences cannot be distinguished based solely on the movement of objects; accurate analysis is only possible by considering the 'context' of the overall flow of the game and the tactical development situation.

[0477] In addition, field location information helps determine which spatial area the current object is located in—such as the center, side, penalty box, or near the halfway line—and what tactical implications that area has, for example, whether it is a danger zone, a build-up area, or an area where second balls are likely to occur.

[0478] In this way, the head AI device (500) generates an integrated video sequence (201) by aligning and integrating video streams received from multiple node AI devices (500′) based on time axis and field of view information, detects a detected object (203-1) in real time from the generated integrated video sequence (201), and identifies the behavior flow of the object by assigning an object identifier (ID) and tracking it in a time series.

[0479] Subsequently, the head AI device (500) generates primary behavioral analysis input data (3-1) containing information such as the location, movement, interaction, repetitive patterns, and the game context at the time of occurrence of the tracked object, and outputs primary behavioral analysis result data (3-2) based on this. The primary behavioral analysis result data (3-2) includes results that identify and describe meaningful short-term behaviors at the behavioral unit level of the object, such as, for example, acceleration / deceleration of a specific player, confrontation with a defender, change of direction, escaping pressure, and attempts to steal the ball.

[0480] The head AI device (500) of the present invention may utilize various deep learning-based video analysis models to perform object detection, tracking, behavior recognition, and pattern analysis. For example, the head AI device (500) may apply models such as YOLOv8, Faster R-CNN, and DETR for object detection, thereby enabling real-time detection of objects such as players and balls with high precision within an integrated video sequence. The detected objects are subsequently tracked continuously over time through multiple object tracking algorithms such as DeepSORT, ByteTrack, and TrackFormer, and a unique identifier (ID) is assigned to each object to track the continuous flow of movement.

[0481] According to one embodiment of the present invention, to analyze the position change and temporal movement patterns of an object, the head AI device (500) may utilize an ST-GCN (Spatial-Temporal Graph Convolutional Network) or Transformer-based behavior recognition model, thereby classifying or predicting short-term actions performed by the object based on time-series data. Additionally, the head AI device (500) can perform sophisticated behavior analysis considering the correlation between consecutive actions by applying a CNN-LSTM hybrid structure or a 3D CNN model to enhance spatiotemporal pattern analysis.

[0482] FIG. 21 is a block diagram schematically illustrating the process of the secondary behavior analysis result data generation step (S22-1) according to an embodiment of the present invention.

[0483] As described, the server device (400) receives primary behavior analysis result data (3-2) transmitted from the head AI device (500) as input and generates secondary behavior analysis result data (4-2) based on it.

[0484] According to one embodiment of the present invention, the server device (400) performs a comprehensive judgment by considering complex factors such as the interaction between multiple objects, the game context, and the tactical flow, going beyond the short-term analysis results of the operation of a single object. The secondary behavior analysis result data (4-2) generated through this analysis may be composed of sub-data such as, for example, tactical situation classification data (4-2-1), behavior purpose / intention estimation data (4-2-2), behavior importance / priority judgment data (4-2-3), and content extraction element data (4-2-4).

[0485] Tactical Situation Classification Data (4-2-1)

[0486] According to one embodiment of the present invention, tactical situation classification data (4-2-1) is information generated during a secondary behavioral analysis process performed by the server device (400) of the present invention, and includes the results of classifying interactions and scenarios between objects occurring during a match from a tactical perspective. This is intended to go beyond simply describing the movements or actions of individual objects (players) and to express, as structured information, which tactical phase the current scene corresponds to. For example, in a scene where an attacking midfielder quickly breaks forward immediately after stealing the ball from an opposing defender, the situation corresponds to a 'transition from defense to attack,' and tactical tags such as 'central space penetration' or '1-on-1 breakthrough' may be assigned together. Specifically, the tactical situation classification data (4-2-1) may be composed of detailed items such as whether there is an attack / defense transition, location-based tactical tags such as lateral pressing and box penetration, tactical unit action sequences such as pass → penetration → shooting, and types of interactions between multiple objects such as double-team pressing and space-creating movements.

[0487] Estimated Behavioral Purpose / Intention Data (4-2-2)

[0488] According to one embodiment of the present invention, the behavioral purpose / intention estimation data (4-2-2) is high-dimensional interpretation information generated during a secondary behavioral analysis process performed by the server device (400) of the present invention, and is structured data that estimates the tactical or strategic intention inherent behind the object, rather than the simple external movement of the object. The behavioral purpose / intention estimation data (4-2-2) is generated by comprehensively analyzing various quantitative elements, such as the object's location, direction of movement, speed change, distance from surrounding objects, and interaction history, which are input from the primary behavioral analysis result data (3-2), in a manner that estimates what purpose the object acted with at a specific point in time.

[0489] For example, if an attacking midfielder rapidly breaks forward while keeping their gaze fixed on a teammate on the left flank, luring a defender, and making subtle adjustments to their direction, such behavior may not be a simple dribble but rather embodies a tactical intention, such as "creating space by inducing a pass" or "luring for a flank penetration." While this intention is not evident in the movement itself, it can be precisely estimated based on contextual information, such as changes in viewpoint, the placement of surrounding objects, timing, and differences in speed.

[0490] The behavioral purpose / intention estimation data (4-2-2) according to the present invention may include direct purposes of current actions such as passing, shooting, securing space, and containment, tactical purposes such as luring the defense, avoiding pressure, and inducing a collapse of positions, and time or location-based intentions such as delaying time in the late stages of the second half and attacking side spaces.

[0491] According to one embodiment of the present invention, a server device (400) can generate behavioral purpose / intention estimation data (4-2-2) that can estimate what tactical purpose or intention a specific player has based on real-time motion data of the player. At this time, the server device (400) can determine whether the behavior is a simple position change or includes an intention such as pressing, penetration, or preparation for a pass by analyzing the movement direction, speed change, and distance adjustment with the opponent of players who have a significant influence based on the distance from the ball or the player possessing the ball. Such analysis is not a one-time position-based judgment, but is performed precisely by reflecting continuous changes over time, such as acceleration and distance changes in 10-frame units.

[0492] Additionally, the server device (400) can quantitatively evaluate the impact of the action on scoring probability, that is, the threat level or tactical importance within the game, based on the results of the above-mentioned action analysis. For example, actions such as forward movement near the penalty box, attempts to break through on the flanks, and actions to widen the gap between opposing defenders have high importance, and the server device (400) quantifies this and reflects it in the action importance / priority judgment data (4-2-3). Since this judgment is based on a dynamic judgment structure that reflects the current game context and real-time development situation rather than a predefined weighting model, it is possible to generate consistent and contextual data.

[0493] Behavior Importance / Priority Judgment Data (4-2-3)

[0494] According to one embodiment of the present invention, the behavior importance / priority judgment data (4-2-3) is a component of the secondary behavior analysis result data (4-2) generated by the server device (400) of the present invention based on the primary behavior analysis result data (3-2), and is high-dimensional analysis information that evaluates the tactical and strategic importance of a specific object's behavior that occurred during the game and its priority as broadcast content, and outputs it in a numerical or classified form.

[0495] According to one embodiment of the present invention, the action importance / priority judgment data (4-2-3) is determined by comprehensively considering not only whether a certain action has occurred, but also the influence of the action on the overall flow of the game, the density of interaction with other objects (players), the ripple effect on the transition between attack and defense, and the urgency regarding broadcast timing. For example, even if it is the same dribble, a short ball carrying in the defensive zone may be evaluated as having low importance, but a dribble that beats a defender and breaks through space during forward pressing is given high priority because it has high strategic value and is also highly visible as broadcast content.

[0496] The behavior importance / priority judgment data (4-2-3) according to the present invention may include tactical impact data indicating the level of effect the behavior has had on team strategy, such as offensive development, defensive collapse, and line breaking; behavior specificity data such as unpredictable creative behavior and whether high-difficulty skills are used; urgency and development speed data such as whether there is a decisive timing affecting the speed of the game; and broadcast content priority data evaluating which behavior among simultaneous events should be delivered to the user first.

[0497] Content extraction element data (4-2-4)

[0498] According to one embodiment of the present invention, the content extraction element data (4-2-4) is one of the final output data generated by the server device (400) in the second behavior analysis step, and is structured data in which information elements to be selected and utilized as voice broadcast content in the game situation at that time are extracted. The content extraction element data (4-2-4) does not simply list the analysis results, but rather defines the 'components of the broadcast message' to be delivered to the user, and performs the role of filtering and refining information so that it can be reflected in the broadcast flow.

[0499] According to one embodiment of the present invention, a server device (400) comprehensively analyzes primary behavior analysis result data (3-2) and tactical situation classification data (4-2-1), behavior purpose / intention estimation data (4-2-2), and behavior importance / priority judgment data (4-2-3) derived therefrom, and selects content that is worth informing the user at that time. For example, the content extraction element data (4-2-4) may include key object data, such as a player who is the center of the broadcast at that time. Specifically, if there is a player with jersey number 11, Kim Hyun-woo, who is the center of the broadcast at that time, the content extraction element data (4-2-4) may refer to the data of player Kim Hyun-woo with jersey number 11.

[0500] According to one embodiment of the present invention, the content extraction element data (4-2-4) may include action summary data such as key actions or tactics. For example, if a specific player is breaking forward after stealing the ball, the content extraction element data (4-2-4) may represent action summary data of the forward breakthrough. Additionally, the content extraction element data (4-2-4) may include context-based situation data such as when the opposing defensive line has collapsed. Furthermore, the content extraction element data (4-2-4) may represent location / zone data indicating whether the action occurred in a specific area of ​​the field, such as "near the center line" or "near the left penalty box." Additionally, the content extraction element data (4-2-4) may represent semantic emphasis elements, such as emotional or emphasis-targeted expressions like "lightning breakthrough" or "1-on-1 situation with the goal." Furthermore, the content extraction element data (4-2-4) may represent predictability data indicating the possibility of future development, such as "high likelihood of leading to a shot."

[0501] FIG. 22a is a drawing that visually represents an actual situation occurring in a soccer stadium, serving as a specific scene example to explain the real-time object behavior recognition and relay processing flow according to an embodiment of the present invention. Specifically, it depicts a tense scene in which attacking midfielder number 11 steals the ball from the opposing team's defender number 4 by dribbling, and schematically illustrates how such a real-time game situation is processed by the system of the present invention.

[0502] As described above, multiple node AI devices (500') are installed around the stadium to capture the scene from various angles, and each node AI device (500') acquires video data (201) in real time from different viewpoints, such as the front side view, rear view, left or right side view of player number 11. Subsequently, each node AI device (500') transmits the acquired video data (201) to the head AI device (500) via wireless or wired communication.

[0503] As illustrated in FIG. 22a, a plurality of node AI devices (500′) each capture the same scene of a broadcasted match from multiple angles at different points in time and transmit the captured video data (201) to the head AI device (500). For example, the first sensor unit (111) located on the left side captures the side view of player H11 and the acceleration motion when starting to dribble, the second sensor unit (112) located on the front acquires the frontal confrontation situation between player H11 and defender A4 in high resolution, and the third sensor unit (113) located on the rear acquires video that clearly captures player H11's jersey number, direction of movement, and changes in posture immediately after stealing the ball. Through such multi-angle shooting, the head AI device (500) receives a plurality of video data (201) in which features for each point in time are preserved, and then precisely extracts information such as the position, speed, and direction of objects (players and balls) to construct spatial coordinates and movement data (3-1-1).

[0504] FIG. 22b is a table-formatted drawing summarizing the specific configuration and example values ​​of spatial coordinates and movement data (3-1-1) utilized in the object primary behavior analysis step (S230) performed by the head AI device (500) of the present invention, based on the game scene (player number 11's ball stealing and breakthrough situation) described in FIG. 22a.

[0505] As described, based on the integrated video sequence (202) acquired in real time by the head AI device (500), dynamic elements such as the position, velocity, and direction of objects (players and balls) can be quantified and interpreted. For example, the position coordinates of player H11, an attacking midfielder, are recorded as (35.4, 22.1), and the position coordinates of defender A4 are recorded as (36.2, 22.5), indicating that a close situation between the two players has been captured. Additionally, it can be determined that player H11 possesses the ball by the fact that the ball's position coordinates are identical to those of H11. Furthermore, the instantaneous velocity (8.5 m / s) and acceleration (+1.1 m / s²) of player H11 indicate that he is rapidly breaking through, and the movement direction vector (θ = 65°) and rate of change of direction (Δθ / Δt = 0.5 rad / s) indicate that a sudden change of direction is detected while advancing toward the goal.

[0506] FIG. 22c is a table showing a specific configuration example of relative position and interrelationship data (3-1-2) used in the object primary behavior analysis step (S230).

[0507] As described, the relative position and interrelationship data (3-1-2) quantitatively expresses the physical positional relationship and interaction patterns between objects by including information such as the change in distance between player H11 and opposing defender A4, approach speed, and relative approach angle. For example, the distance between player H11 and player A4 decreased rapidly to 0.5m immediately after the ball was stolen and then increased again to 2.5m, indicating that a separation situation was formed immediately after the ball was stolen. Additionally, player A4 approached from the side at an angle of 120° relative to the reference point, and the approach speed was measured to be gradually decreasing to 2.3m / s, predicting the possibility of a failure in defensive response. Furthermore, the distance between player H11 and the rear defender is estimated to be about 15m, which can be used as a tactical indicator to determine whether the defense is organized. Meanwhile, since the coordinates and speeds of the ball and player H11 are measured identically, it can be clearly determined that player H11 is the possessor of the ball.

[0508] In order for relative position and interrelationship data (3-1-2) such as Fig. 22c to be extracted, it can be presumed that the integrated video sequence (202) contained the following scenario. Specifically, in the central area, an attacking midfielder with jersey number H11 is rapidly approaching the ball, and an opposing defender with jersey number A4 is rapidly approaching from the side to stop H11. At this point, multiple node AI devices (500′) each capture the movements of H11 and A4 from different viewing angles, and these videos are integrated in the head AI device (500) to form a single continuous sequence.

[0509] At this time, the integrated video sequence (202) shows the change in distance between H11 and A4 in detail over time. Before the ball was stolen, there was a distance of about 1.2m, but at the moment of the steal, it narrowed to 0.5m, and then, as H11 advanced quickly, the distance widened again to 2.5m. Additionally, A4 enters from the side at a direction of about 120° from the reference coordinate system, and the approach speed is captured gradually decreasing from 2.3m / s. This implies the possibility of failure in defensive response actions.

[0510] In addition, the integrated video sequence (202) shows that the distance between player H11 and the rear defender is maintained at approximately 15m, thereby providing information on the arrangement and spacing of the overall defensive line. The ball is attached to H11's foot, and the speed and direction of movement match H11's, clearly indicating that H11 has possession of the ball.

[0511] FIG. 22d is a diagram summarizing the process of identifying the unique behavior style and repetitive movement patterns of a specific object, player H11, through specific items and example values ​​of the object pattern data (3-1-3) utilized in the present invention.

[0512] According to one embodiment of the present invention, the situation in which object pattern data (3-1-3) is extracted in the integrated video sequence (202) can be assumed as follows. It is assumed that in the second half of the match, a scene was captured in which the home team's attacking midfielder H11 attempts to break through quickly toward the forward space immediately after stealing the ball from the opposing defender (A4) near the left side half-line. At this time, it is assumed that H11 intended to neutralize the opposing defensive line through a momentary increase in speed, and that his acceleration was actually observed increasing from +0.6 to +1.1 m / s² in the video. This means that a behavioral pattern related to agile speed changes, rather than linear movement, was captured.

[0513] According to one embodiment of the present invention, it is assumed that player H11 performs rapid left-right turning movements to outrun opposing defenders, such as attempting 1.3 changes of direction per second during a breakthrough, and that these turning movements are accurately analyzed through frame-by-frame tracking data within the integrated video.

[0514] This may be an important signal that can be interpreted not as a simple movement, but as a 'deceptive movement' or 'defense evasion technique'. Additionally, it is assumed that in the integrated video sequence (202), player H11 was captured performing three consecutive dribbles in the left-right-left direction. This is a touch pattern frequently repeated during individual breakthroughs, which allows the head AI device (500) of the present invention to recognize it as a specific unique behavioral pattern by comparing it with the dribbling pattern performed by the same player in similar scenes in the past. Furthermore, regarding the overall play path, it is assumed that in the integrated video sequence (202), player H11 repeated the typical path of "dribbling from the left flank and penetrating into the center," which he frequently demonstrated in past matches, in this scene as well.

[0515] As described, object pattern data (3-1-3) corresponds to information extracted by the head AI device (500) based on the continuous temporal movement of an object from the integrated video sequence (202). For example, in the case of player H11, a pattern of continuous acceleration change increasing from +0.6 to +1.1 m / s² was detected, which can be interpreted as a speed increase pattern commonly seen when attempting to break through an opponent or entering an attack. Additionally, the frequency of direction change was measured at 1.3 times / second, which indicates a rapid change of direction movement in the face of a defense, and can be interpreted as an intention to evade defense through deceptive movements.

[0516] As described, the touch pattern in which the player performed three consecutive dribbles (left-right-left) in this scene can be identified as a unique behavior repeatedly used during individual breakthroughs or 1-on-1 responses, and this pattern can be compared and analyzed with the same player's past behavioral records. Finally, in the repeated path comparison item, the path “left flank dribble -> central penetration” has been observed identically three times in the past; this serves as data reflecting the player's typical behavioral tendencies and is utilized as a basis for behavioral prediction and risk assessment.

[0517] FIG. 22e is a diagram showing specific items and example analysis contents of context data (3-1-4) utilized by the head AI device (500) of the present invention in performing the object primary behavior analysis step (S230).

[0518] In the situation of FIG. 22a, it is assumed that the content included in the integrated video sequence (202) captured the flow of the game scene and surrounding circumstances from various angles. First, it would have contained a scene where the away team's defender A4 (Park Jae-hoon) was controlling the ball and attempting a build-up in the left flank area near the half-line, while the home team's attacking midfielder H11 (Kim Hyun-woo) quickly applied pressure and stole the ball. It is assumed that during this process, A4 momentarily lost the ball or showed an unstable touch, and H11 detected this and actively put his body in to win the ball, and this scene is included in the integrated video sequence (202).

[0519] At this time, the scene corresponds to the 34th minute of the second half in terms of match time, which can be extracted from timecode information within the integrated video sequence (202). The 34th minute of the second half is given the context that it is a point where physical strength may decline after the middle of the second half and can be a turning point in the match. Additionally, immediately after stealing the ball, H11 attempts a breakthrough forward from the same left flank to induce an attack transition, and at this time, movements in which the entire team's positions quickly change would have been captured. For example, forward attackers preparing to penetrate or supporting movements in the midfield may have been captured.

[0520] In addition, regarding field location information, it can be seen that the scene occurred on the left side near the half-line, and this is clearly reflected in the integrated sequence based on the location-based shooting time and video metadata of the node AI devices (500'). Overall, the integrated video sequence (202) captured not only the simple capture scene but also the flow of play immediately before and after the capture, the tactical movements of surrounding players, specific location information of the field, and the time and team strategy situation together, and it is assumed that the head AI device (500) of the present invention was able to accurately extract context-based contextual data (3-1-4) from this.

[0521] According to one embodiment of the present invention, contextual data (3-1-4), unlike simple location or movement information, includes contextual elements such as the overall flow of the game, timing, and team strategy, thereby enabling a high-level understanding of the situational meaning of the scene. For example, in the object ID item, information is specified such that H11 is Kim Hyun-woo (attacking midfielder, home team) and A4 is Park Jae-hoon (center back, away team), allowing for different interpretations of behavior depending on the position and role of each object. In the ball possession item, the result is specified as “determined to be in H11’s possession immediately after interception,” which means that the scene corresponds to the starting point of an attack transition and is used as a basis for determining the direction and flow of subsequent behavior analysis.

[0522] As described, the time information item lists the point in time as "34 minutes into the second half," which can be used to assess the urgency of situations such as potential player fatigue or the decisive moments in the final stages of the match. Additionally, the field position information includes a specific location such as "left flank, near the halfway line," which contributes to understanding the spatial context, such as attempts at flank breakthroughs or build-up play from the back. Furthermore, the team position information item lists strategic flow details such as "Home team transitioning to attack," enabling the identification of how individual actions are connected to the overall team strategy.

[0523] FIG. 23a is a diagram illustrating primary behavior analysis result data (3-2) output by a head AI device (500) using primary behavior analysis input data (3-1) as input according to an embodiment of the present invention.

[0524] Object identification data (3-2-1)

[0525] According to one embodiment of the present invention, the head AI device (500) can extract object identification data (3-2-1) based on the previously input primary behavior analysis input data (3-1). Specifically, the head AI device (500) can identify real-time location, movement speed, and direction information of each object through spatial coordinates and movement data (3-1-1), and thereby separate identifiable unit objects. In addition, the head AI device (500) can perform role and situation judgment for individual objects by considering relational information such as distance from surrounding objects, approach direction, and ball ownership status together through relative location and interrelationship data (3-1-2).

[0526] Based on the analysis results, the head AI device (500) assigns a unique ID to each detected object (203-1) and, for example, can identify a player with jersey number H11 as "Kim Hyun-woo (attacking midfielder, belonging to the home team)." At this time, the head AI device (500) can utilize a previously learned player database, team placement information, position distribution map, etc., to distinguish whether the object type is a "player," a "ball," or a "referee," and can determine the exact position through the object's affiliated team and location information. In addition, the head AI device (500) can analyze whether the object possesses the ball to generate a comprehensive identification result that includes the current state of the object, such as possessing the ball or not possessing it.

[0527] Behavior state data (3-2-2)

[0528] According to one embodiment of the present invention, a head AI device (500) can derive behavior state data (3-2-2) indicating the instantaneous position and movement state of an object by analyzing various numerical-based information collected from primary behavior analysis input data (3-1). Specifically, the head AI device (500) can identify the field coordinates where the object is located at a specific point in time based on spatial coordinates and movement data (3-1-1), and obtain real-time location data such as (35.4, 22.1). In addition, by analyzing velocity and acceleration information such as 8.5 m / s and +1.1 m / s² measured at the same point in time, it can determine whether the object is in a simple stationary state, moving, or accelerating with an intention to break through.

[0529] According to one embodiment of the present invention, the head AI device (500) can specifically infer the movement path of a detected object (203-1) by considering an absolute angle such as 65° or a tactical reference point such as the direction of the goalpost, and can also evaluate whether the movement is continued or the frequency of changing direction based on repetitive movement data. In particular, by comprehensively analyzing such speed, direction, and position information, movement state labels such as “stop,” “movement,” or “attempt to break through” are finally assigned to the object, and, for example, a result can be derived in which player H11 is classified as currently being “breakthrough.”

[0530] Relationship data between objects (3-2-3)

[0531] According to one embodiment of the present invention, the head AI device (500) can generate relationship data (3-2-3) between objects by identifying the relative positions and operational relationships between objects based on the input primary behavior analysis input data (3-1). Specifically, the head AI device (500) analyzes the ID and location information of the opposing objects to determine which surrounding objects a specific object is interacting with. For example, if opposing defender A4 (Park Jae-hoon) located near player H11 (Kim Hyun-woo) is detected, the spatial and operational relationship between the opposing object and the object currently being analyzed is identified. At this time, real-time distance information between the two objects is analyzed together, and for example, if the distance to A4 is measured as 2.5m, it can be used to predict defensive pressure intensity or risk level. In addition, if the approach speed of player A4 is confirmed to be 2.3 m / s, this can be interpreted as having an intention to actively intervene in defense, going beyond simply being close in distance.

[0532] Behavioral Context Summary Data (3-2-4)

[0533] According to one embodiment of the present invention, a head AI device (500) can generate behavioral context summary data (3-2-4) that concisely summarizes the overall situation of the scene by comprehensively considering various analysis input data. For example, in the analysis time item, by specifying a time point in the game such as “34 minutes into the second half,” it is possible to interpret the situation, such as physical exhaustion or a turning point in the match. Such time information can be utilized as an important element not only for the meaning of individual actions but also for team tactical analysis.

[0534] In addition, the area information item indicating the location on the field includes a spatial classification called “left center area,” through which the head AI device (500) can determine whether the point where the action occurred is a transition section from the left side to the center or a rear area. The head AI device (500) indicates whether there is a change in ball possession with a flag called “disposal occurred,” allowing it to immediately identify whether the scene was the starting point of an attack transition or a defensive failure. Additionally, the head AI device (500) assigns a code-based classification result such as “S_ID_047 = Breakthrough_Solo” in the situation identifier item, which can be used as a useful identification criterion in various application processing such as automatic content classification, highlight generation, and game analysis report writing in the future.

[0535] FIG. 23b is a diagram showing an example in JSON format in which primary behavior analysis result data (3-2) generated by a head AI device (500) according to one embodiment of the present invention is output in a structured form.

[0536] Figure 23b shows a structure in which various quantitative and qualitative information is integrated based on the analysis results for object H11. For example, 'object_id' specifies the object to be analyzed as H11, and the 'team' and 'position' items indicate that the object is an attacking midfielder belonging to the home team. Additionally, 'is_ball_owner' indicates the current ball possession status as true, so it can be seen that H11 is in possession of the ball at the time of analysis.

[0537] As described, items describing behavioral states include 'location', 'speed', 'acceleration', 'direction', and 'movement_state', respectively presenting operational states such as real-time coordinates, speed, acceleration, direction of movement, and breakthrough status in numerical or label form. In addition, regarding relationship information with opposing objects, 'opponent_id' indicates that it is A4, and information on the distance (2.5m) and approach speed (2.3m / s) to the opposing object is structured into the 'distance_to_opponent' and 'opponent_approach_speed' items.

[0538] In addition, items reflecting the context of the game include 'field_zone' (left center area), 'ball_event' (occupation occurred), 'time_stamp' (34 minutes into the second half), and 'situation_id' (S_ID_047), which serve as useful elements for determining the strategic flow, timing, and occurrence of events in the scene.

[0539] FIG. 24a is a diagram illustrating an example of the configuration of tactical situation classification data (4-2-1) generated by a server device (400) based on primary behavior analysis result data (3-2) received from a head AI device (500) in the secondary behavior analysis result data generation step (S22-1) according to an embodiment of the present invention. Specifically, the tactical situation classification data (4-2-1) is a result for determining which tactical context the current behavior of an object corresponds to, and is analyzed including the play flow and location-based information at a given point in time.

[0540] According to one embodiment of the present invention, a server device (400) can generate tactical situation classification data (4-2-1) for determining the tactical significance of an object based on primary behavior analysis result data (3-2) received from a head AI device (500). Specifically, the server device (400) comprehensively analyzes the object's current movement, ball possession status, field position, and time information included in the behavior state data (3-2-2) and behavior context summary data (3-2-4) to classify which type of tactical situation the action corresponds to. For example, if an object is breaking through and increasing acceleration while in close proximity to an opposing defender, this can be determined as a “solo breakthrough situation.”

[0541] Additionally, the server device (400) can determine whether a play transition occurs by identifying whether the analysis point is a section of rapid transition from defense to attack. To this end, the server device (400) utilizes contact event flags and ball possession change information to identify which flow the scene belongs to, such as an attack transition, a sustained attack, or a set play. Furthermore, the server device (400) enables location-based interpretation of tactical situations by comparing the current position coordinates of an object with field zone information and assigning directional tags such as a central area, a side, or a forward pressing area.

[0542] FIG. 24b is a diagram illustrating an example of the configuration of behavioral purpose / intention estimation data (4-2-2) generated by a server device (400) based on primary behavioral analysis result data (3-2) received from a head AI device (500) in a secondary behavioral analysis result data generation step (S22-1) according to an embodiment of the present invention.

[0543] According to one embodiment of the present invention, a server device (400) can generate behavioral purpose / intention estimation data (4-2-2) that predicts the tactical purpose and future intention of an object based on the movement and relationship information of the object included in the primary behavioral analysis result data (3-2). To this end, the server device (400) analyzes behavioral state data (3-2-2), such as the object's position coordinates, velocity and acceleration, and direction of movement, to determine whether the object is likely to occupy a specific space or perform a specific technique as the next action. For example, if an object holding a ball maintains a certain distance from surrounding defenders while accelerating in a certain direction, the server device (400) interprets the situation as an additional dribble or shooting preparation action and can reflect that possibility in the 'predicted action' item.

[0544] Additionally, the server device (400) considers the approach speed, position zone, and analysis time of surrounding defenders identified in the relationship data between objects (3-2-3) and the behavioral context summary data (3-2-4) to infer whether the current action is a simple response or a tactical execution based on a strategic intention. Through this, the server device (400) determines whether the object aims for a solo breakthrough or follows a strategy of opening space to induce the team's pattern play, and reflects this in the 'Strategy Type' item. At this time, by referring to past play types or team / player characteristic data together, the intention can be distinguished more precisely even for the same action.

[0545] FIG. 24c is a diagram illustrating an example of the configuration of behavior importance / priority judgment data (4-2-3) generated by a server device (400) based on the first behavior analysis result data (3-2) received from a head AI device (500) in the second behavior analysis result data generation step (S22-1) according to an embodiment of the present invention.

[0546] According to one embodiment of the present invention, a server device (400) can generate action importance / priority judgment data (4-2-3) that evaluates the importance of each action within the game and determines its priority by comprehensively analyzing primary action analysis result data (3-2), tactical situation classification data (4-2-1) derived therefrom, and action purpose / intention estimation data (4-2-2). To this end, the server device (400) receives multidimensional data such as the viewpoint, zone, transition status, relationship with the opponent, and predicted action of the scene as input, quantifies the relative importance that the current action has within the flow of the entire game, and outputs it as an “importance score” item. For example, a breakthrough attempt in the central pressing zone during the latter part of the game, such as the 34th minute of the second half, has a significant impact on the overall flow, so a high importance score such as 0.92 may be assigned.

[0547] Additionally, the server device (400) can output an “event flag” item by determining whether the action is worthy of inclusion in a highlight video or contributes substantially to the possibility of scoring. This can be directly utilized for content creation or for selecting key scenes. Furthermore, the server device (400) analyzes the impact of an individual object's action on the team's overall strategy and refl...

Claims

1. In an edge AI device for object tracking and collecting spatial dwell information per object, A sensor unit for sensing objects in a designated area; A communication unit that communicates with other edge AI devices; Memory section; and Includes a processor unit; and The above processor unit The sensor unit analyzes the object sensed and outputs object analysis information, and stores the movement path information and dwell time information of the analyzed object in the memory unit. Configured to generate object-specific spatial dwell information based on the object analysis information output above, the stored movement path information, and the stored dwell time information, Edge AI device for object tracking and collection of spatial dwell information per object.

2. In Paragraph 1, The above processor unit The above sensor unit is configured to analyze the sensed object in real time and output object analysis information. Edge AI device for object tracking and collection of spatial dwell information per object.

3. In Paragraph 1, The above object analysis information Information output based on at least one of the size information of the object, the shape information of the object, or the color information of the object, Edge AI device for object tracking and collection of spatial dwell information per object.

4. In Paragraph 1, The above object analysis information Information comprising at least one of age information of the object, gender information of the object, or race information of the object, Edge AI device for object tracking and collection of spatial dwell information per object.

5. In Paragraph 1, The above processor unit is, When the object moves and the sensor unit can no longer sense the object in the designated area, at least one of the outputted object analysis information, the stored movement path information and the stored dwell time information, and the created object-specific spatial dwell information is configured to be transmitted to the other edge AI device through the communication unit. Edge AI device for object tracking and collection of spatial dwell information per object.

6. An edge AI device for object tracking and collecting spatial facility stay information of said object, A sensor unit for sensing objects in a designated area; A communication unit that communicates with other edge AI devices, other terminals, or external servers; A memory unit storing unique information required for the identification of the above object; and Includes a processor unit; and The above processor unit The sensor unit matches the unique information stored in the memory unit to the object sensed by the sensor unit, and stores the movement path information and dwell time information of the matched object in the memory unit. Configured to create spatial dwell information of the sensed object based on unique information required for the identification of the matched object, the stored movement path information, and the stored dwell time information. Edge AI for object tracking and collection of spatial facility stay information of the said object.

7. In Paragraph 6, The above processor unit If the unique information stored in the memory unit does not match the object sensed by the sensor unit, Configured to transmit information related to outsider access to the other terminal, the external server, or the other edge AI device through the communication unit, Edge AI device for object tracking and collection of spatial facility stay information of said object.

8. In Paragraph 7, The above information regarding external access is At least one of the movement path information of the object sensed by the sensor unit, the dwell time information of the object sensed by the sensor unit, or the external shape information of the object sensed by the sensor unit, Edge AI device for object tracking and collection of spatial facility stay information of said object.

9. In Paragraph 6, The above processor unit is, If the above-mentioned matched object moves and the sensor unit can no longer sense the above-mentioned matched object in the above-mentioned designated area, Configured to transmit at least one of the unique information required for identification of the matched object, the stored movement path information, the stored dwell time information, and the created spatial dwell information to the other edge AI device through the communication unit. Edge AI device for object tracking and collection of spatial facility stay information of said object.

10. In Paragraph 6, The above sensor unit Sense the face of the object in the designated area above, and The above processor unit is, The sensor unit is configured to match the unique information stored in the memory unit with the sensed object based on the face of the object sensed. Edge AI device for object tracking and collection of spatial facility stay information of said object.

11. In a vehicle management system for identifying and tracking vehicles on board a vessel, An edge AI device positioned at least one of the dock entrance, the ship boarding point, the ship disembarking point, and the dock exit; Server device; and Includes user terminal; The above edge AI device is, It is configured to generate vehicle identification information for the above vehicle and transmit the generated vehicle identification information to the server device, and The above server device is, Configured to provide user guidance information or administrator guidance information to the user terminal based on the vehicle identification information received from the edge AI device, Vehicle management system for identifying and tracking vehicles on board a ship.

12. In Paragraph 11, The above vehicle identification information is, at least one of the following: vehicle number information for the vehicle, vehicle type information for the vehicle, total vehicle length information for the vehicle, entry time information for the vehicle, reservation status information for the vehicle, and passenger count information for the vessel. Vehicle management system for identifying and tracking vehicles on board a ship.

13. In Paragraph 11, The above user guide information is, at least one of information on whether the vehicle is boarded, information on the remaining waiting time expected until the vehicle can actually board the vessel, information on the time until the vessel arrives, and information on the real-time loading status on the vessel. Vehicle management system for identifying and tracking vehicles on board a ship.

14. In Paragraph 11, The above administrator guidance information is, at least one of the following: information on the reservation and reception status of the above-mentioned vehicle, information on vehicles waiting on-site at the above-mentioned pier, information on the range of vehicles permitted to board the above-mentioned vessel, information on the loading rate of the above-mentioned vessel, information on the remaining space of the above-mentioned vessel, information on the list of vehicles eligible for priority boarding for the above-mentioned vessel, information on the arrival time of the above-mentioned vessel, and information on the departure time of the above-mentioned vessel. Vehicle management system for identifying and tracking vehicles on board a ship.

15. In Paragraph 11, The above edge AI device is, It includes a plurality of node AI devices and a head AI device that communicates with the node AI devices, and The above-mentioned node AI device is, Generates sensing data of the above vehicle, and The above head AI device is, Configured to receive the generated sensing data and generate the vehicle identification information, Vehicle management system for identifying and tracking vehicles on board a ship.

16. An edge AI device located in a stadium that performs object-based voice relay, Memory section; One or more sensor units for sensing the above object; and Includes a processor unit; and The above one or more sensor units are, Each is positioned at a different location within the aforementioned stadium, and each acquires one or more video data, The above processor unit is: Configured to analyze the above video data to generate primary behavioral analysis result data based on primary behavioral analysis input data of the object, generate text data for the object based on the generated primary behavioral analysis result data, and generate voice relay data based on the generated text data. Edge AI device located in the stadium that performs object-based voice relay.

17. In Paragraph 16, The primary behavioral analysis input data of the above object is, A method comprising at least one of spatial coordinate data of the object, movement data of the object, relative position data of the object, interrelationship data of the object within the stadium, pattern data of the object, and context data of the object. Edge AI device located in the stadium that performs object-based voice relay.

18. In Paragraph 16, The above processor unit is, Based on the primary behavioral analysis result data generated above, it is configured to generate secondary behavioral analysis result data for the object, and The above secondary behavioral analysis result data is, At least one of the tactical situation classification data of the object, the behavioral purpose and intention estimation data of the object, the behavioral importance and priority judgment data of the object, and the content extraction element data of the object Edge AI device located in the stadium that performs object-based voice relay.

19. In Paragraph 16, The above processor unit is, The text data is configured to be generated to follow a sentence structure corresponding to the action of the object, and Configured so that the text data generated above matches the template of a pre-secured voice sample library stored in the memory unit, Edge AI device located in the stadium that performs object-based voice relay.

20. In Paragraph 16, The voice relay data generated above is, including at least one meta-information among intonation information, speed information, and emotion expression information of the voice corresponding to the above text data, Edge AI device located in the stadium that performs object-based voice relay.

Images