Ai platform for providing field-customized and personalized artificial intelligence, method for determining field situation, and edge ai-based system for collecting and analyzing space stay information

Abstract

The present invention relates to an edge AI device for tracking an object and collecting space stay information for each object, the edge AI device comprising: a sensor unit for sensing an object in a designated area; a communication unit for communicating with another edge AI device; a memory unit; and a processor unit. The processor unit is configured to: analyze the object sensed by the sensor unit to output object analysis information; store movement path information and stay time information about the analyzed object in the memory unit; and create the space stay information for each object on the basis of the output object analysis information, the stored movement path information, and the stored stay time information.

Description

An AI platform that provides on-site, personalized AI, a method for assessing on-site situations, and an edge AI-based system that collects and analyzes spatial stay information.

[0001] The present invention relates to an AI platform that provides field-tailored and personalized artificial intelligence, a method for performing field situation assessment, and an edge AI-based system that collects and analyzes spatial stay information.

[0002] In recent years, rapid advancements in artificial intelligence (AI) technology have brought about revolutionary changes in various fields. In particular, advancements in machine learning and deep learning have made AI a crucial tool in various industries, including healthcare, finance, manufacturing, safety management, and transportation. Systems that utilize this AI technology to assess and respond to on-site situations in real time are also attracting attention.

[0003] Traditional on-site situation assessment systems collect field data through various sensors installed on-site, then transmit this data to a central server or cloud, where an AI model analyzes and derives results. For example, in a manufacturing plant, if sensors monitor machine temperature, vibration, and noise, the sensor data is quickly transmitted to a central server, where an AI model analyzes it to detect machine abnormalities early, enabling immediate action. Thus, existing systems operate efficiently when sensors accurately collect field data, rapidly transmit the data to a central server or cloud, and the server's AI model rapidly analyzes the data and derives results. This allows for near-real-time assessment of field situations and appropriate responses.

[0004] However, existing systems suffer from issues such as delays in data transmission, noise in sensor data, resulting in reduced accuracy, and delays in analysis on the central server. Furthermore, field data transmitted over networks poses security threats, and privacy concerns, particularly when collecting data containing personal information, can arise. Therefore, there is a growing need to implement a system that enables faster and more efficient assessment of field situations while addressing security and privacy concerns.

[0005] To address the challenges of existing systems, the development of AI capable of processing and analyzing data on-site is necessary. In other words, AI needs to be able to analyze and make decisions in real time, on-site, without relying on a central server or cloud. On-device AI, or edge AI, refers to AI that operates within the device itself, without a network. By processing data directly on-site, edge AI minimizes data transmission delays and enables accurate, real-time decisions. Furthermore, by processing field data and making decisions within the device without a network, it can address security and privacy concerns that can arise in existing systems.

[0006] To implement an edge AI-based on-site situation assessment system, the AI ​​model must be sufficiently lightweight and optimized to operate within the device. Furthermore, a stable local network for communication between edge devices is required.

[0007] The present invention has been conceived in response to the above-described background technology, and its purpose is to analyze field data and perform field situation judgment based on edge AI without network connection.

[0008] In addition, the present invention discloses a method for performing field situation judgment by transmitting and receiving field situation analysis information generated by each edge AI through a network between edge AIs.

[0009] The present invention has been conceived in response to the above-described background technology, and the purpose of the present invention is to optimize an artificial intelligence model to suit an individual's needs while downloading and using an artificial intelligence model through an artificial intelligence platform.

[0010] Additionally, the present invention aims to prevent personal information leakage by enabling users to use artificial intelligence models on edge devices without a network connection.

[0011] In addition, the present invention aims to increase usability by allowing users to share their experiences of using and optimizing artificial intelligence models through an artificial intelligence platform, so that other users can also experience them.

[0012] The present invention can provide a field facility control infrastructure that can collect information on-site and determine appropriate actions directly on-site by utilizing an edge AI system and an integrated control server.

[0013] The present invention can provide a field facility control method that can collect information on-site and determine appropriate actions directly on-site by utilizing an edge AI system and an integrated control server.

[0014] The present invention aims to resolve the performance degradation caused by existing centralized data processing for analyzing the movement path of a specific person and the time of stay in a specific area in a large complex facility, thereby processing large-scale data in real time.

[0015] The present invention aims to resolve the performance degradation caused by existing centralized data processing for analyzing the movement path of a specific person within a store and the time of stay in a specific area, thereby processing large-scale data in real time.

[0016] The present invention seeks to protect the personal information of objects when tracking them, as existing systems process data in a way that identifies the identity of the objects, which may raise legal issues when collecting data in public places.

[0017] The present invention provides a driver assistance system using an AR device, comprising: an electronic device constituting a plurality of sensor device infrastructures communicating with each other via a communication protocol; and an AR device; wherein the electronic device constituting the plurality of sensor device infrastructures comprises: a sensor device for capturing an image of a road; a processor for directly processing an image processing-based artificial intelligence model and performing a situation judgment operation regarding the road using the image of the road captured by the sensor device as an input of the image processing-based artificial intelligence model; and a memory; wherein the AR device comprises: an AR device processor unit for receiving data regarding the situation judgment via the communication protocol; and a display unit for outputting the received data.

[0018] The present invention provides a driver assistance system using an AR device, wherein the AR device is in a form that can be worn by a driver.

[0019] The present invention provides a driver assistance system using an AR device, wherein the display unit outputs data related to the road situation as an augmented reality image.

[0020] The present invention provides a driver assistance system using an AR device, wherein the road situation is at least one of a traffic accident on the road, traffic congestion on the road, construction on the road, breakdown of facilities on the road, regulation on the road, illegal parking on the road, and violation of traffic rules on the road.

[0021] The present invention provides a driver assistance system using an AR device, wherein the display unit outputs data related to the location of a pedestrian or object in a blind spot of a moving vehicle as an augmented reality image.

[0022] The present invention provides a driver assistance system using an AR device, wherein the display unit outputs data related to road lanes or road surface guide signs as an augmented reality image.

[0023] The present invention provides a driver assistance system using an AR device, in which the display unit outputs data related to securing a safe distance from another vehicle located below a critical distance during driving as an augmented reality image.

[0024] The present invention provides a driver assistance system using an AR device, wherein the display unit outputs data related to the shortest parking space between the driver's destination and the driver's location as an augmented reality image.

[0025] The present invention provides a driver assistance system using an AR device, comprising: an electronic device constituting a plurality of sensor device infrastructures communicating with each other via a communication protocol; and an AR device; wherein the electronic device constituting the plurality of sensor device infrastructures comprises: a sensor device for capturing an image of a road; a processor; and a memory; wherein the AR device comprises: a display unit; and an AR device processor unit directly processing an image processing-based artificial intelligence model, receiving an image of the road captured by the sensor device via the communication protocol, and performing a situation judgment operation regarding the road using the received image of the road as an input of the image processing-based artificial intelligence model; and a display unit outputting data regarding the performed situation judgment.

[0026] The present invention provides a driver assistance system using an AR device, comprising: an electronic device constituting a plurality of sensor device infrastructures communicating with each other via a communication protocol; an external terminal; and an AR device; wherein the electronic device constituting the plurality of sensor device infrastructures comprises: a sensor device for capturing an image of a road; a processor; and a memory; wherein the external terminal comprises: an external terminal processor for processing a general-purpose artificial intelligence model, receiving an image of the road captured by the sensor device via the communication protocol, and performing a situation judgment operation regarding the road using the received image of the road as an input of the general-purpose artificial intelligence model; and wherein the AR device comprises: an AR device processor unit for receiving data regarding the situation judgment performed via the communication protocol; and a display unit for outputting the received data.

[0027] The present invention relates to a driver assistance system using an AR device, comprising: an electronic device constituting an infrastructure of a plurality of sensor devices communicating with each other via a communication protocol; an external terminal; and an AR device; wherein the electronic device constituting the infrastructure of the plurality of sensor devices comprises: a sensor device for capturing an image of a road; a processor directly processing an image processing-based artificial intelligence model and performing part or all of a situation judgment operation regarding the road using the image of the road captured by the sensor device as an input of the image processing-based artificial intelligence model; and a memory; wherein the external terminal comprises: an external terminal processor processing a general-purpose artificial intelligence model and communicating at least one or more of the image of the road captured by the sensor device and data regarding situation judgment partially or wholly performed by the processor via the communication protocol, and performing part or all of a situation judgment operation regarding the road using at least one or more of the received image of the road and data regarding situation judgment partially or wholly performed by the processor as inputs of the general-purpose artificial intelligence model; wherein the AR device comprises: an AR device processor unit receiving data regarding situation judgment partially or wholly performed from the processor and the external terminal processor; A driver assistance system using an AR device is provided, including a display unit for outputting the received data.

[0028] The present invention relates to a driver assistance system using an AR device, comprising: an electronic device constituting an infrastructure of a plurality of sensor devices communicating with each other via a communication protocol; an external terminal; and an AR device; wherein the electronic device constituting the infrastructure of the plurality of sensor devices comprises: a sensor device for capturing an image of a road; a processor; and a memory; wherein the external terminal comprises: an external terminal processor for processing a general-purpose artificial intelligence model, communicating an image of the road captured by the sensor device via the communication protocol, and performing a situation judgment operation on part or all of the road using the received image of the road as an input to the general-purpose artificial intelligence model; and wherein the AR device comprises: an AR device processor unit for directly processing an image processing-based artificial intelligence model, communicating at least one or more of the image of the road captured by the sensor device and data regarding situation judgment partially or fully performed by the external terminal processor via the communication protocol, and performing a situation judgment operation on part or all of the road using at least one or more of the image of the road captured by the sensor device and data regarding situation judgment partially or fully performed by the processor as inputs to the image processing-based artificial intelligence model; A driver assistance system using an AR device is provided, including a display unit that outputs data regarding the situation judgment performed above.

[0029] The present invention relates to a driver assistance system using an AR device, comprising: an electronic device constituting a plurality of sensor device infrastructures communicating with each other via a communication protocol; an external terminal; and an AR device; wherein the electronic device constituting the plurality of sensor device infrastructures comprises: a sensor device for capturing an image of a road; a processor for directly processing an image processing-based artificial intelligence model, and performing a situation judgment operation for part or all of the road using data regarding the image of the road captured by the sensor device as input to the image processing-based artificial intelligence model; and memory; and the external terminal includes an external terminal processor that processes a general-purpose artificial intelligence model, communicates at least one or more of the image of the road captured by the sensor device, and data regarding situation judgment partially or wholly performed by the processor through the communication protocol, and performs a situation judgment operation for part or all of the road by using the received image of the road, and data regarding situation judgment partially or wholly performed by the processor as inputs to the general-purpose artificial intelligence model; and the AR device includes an AR device processor unit that directly processes an image processing-based artificial intelligence model, and communicates at least one or more of the image of the road captured by the sensor device, data regarding situation judgment partially or wholly performed by the external terminal processor, and data regarding situation judgment partially or wholly performed by the processor as inputs to the image processing-based artificial intelligence model, and performs a situation judgment operation for part or all of the road by using at least one or more of the received image of the road, data regarding situation judgment partially or wholly performed by the external terminal processor, and data regarding situation judgment partially or wholly performed by the processor as inputs to the image processing-based artificial intelligence model; and a display unit that outputs data regarding the situation judgment performed above;Provides a driver assistance system using an AR device, including;

[0030] According to one embodiment of the present invention, an electronic device for performing a situation judgment operation in a field includes: a sensor unit; a memory unit for storing an image processing-based artificial intelligence model; and a processor unit; wherein the sensor unit senses first field sensing information, wherein the first field sensing information includes at least one of first field acoustic information, first field text information, and first field image information; and the processor unit generates first field situation analysis information by using the first field sensing information sensed by the sensor unit as an input of the image processing-based artificial intelligence model, and the processor unit can perform a situation judgment operation based on the generated first field situation analysis information.

[0031] In addition, according to one embodiment of the present invention, the sensor unit senses second field sensing information, wherein the second field sensing information includes at least one of second field sound information, second field text information, and second field image information, and the processor unit generates second field situation analysis information using the second field sensing information as an input of the stored image processing-based artificial intelligence model, and performs verification on the first field situation analysis information based on the second field situation analysis information.

[0032] Additionally, according to one embodiment of the present invention, the processor unit can perform the situation judgment operation based on the generated first field situation analysis information and the performed verification result.

[0033] Additionally, according to one embodiment of the present invention, the verification may be performed by the processor unit based on a comparison of the first field situation analysis information and the second field situation analysis information.

[0034] In addition, according to one embodiment of the present invention, the electronic device further includes a network unit; the network unit receives third field situation analysis information generated by the other electronic device from another electronic device, and the processor unit can perform the situation judgment operation based on at least one of the generated first field situation analysis information, the performed verification result, and the received third field situation analysis information.

[0035] Additionally, according to one embodiment of the present invention, the electronic device further includes a network unit, and the network unit can transmit the generated first field situation analysis information to a server.

[0036] In addition, according to one embodiment of the present invention, the network unit may receive update information of the artificial intelligence model generated by the server based on the first field situation analysis information from the server, and the processor unit may update the artificial intelligence model based on the update information.

[0037] According to one embodiment of the present invention, an electronic device for performing a situation judgment operation in a field includes: a sensor unit; a memory unit storing an image processing-based artificial intelligence model; a processor unit; and a network unit; wherein the sensor unit senses first field sensing information, wherein the first field sensing information includes at least one of first field acoustic information, first field text information, and first field image information; and the processor unit generates first field situation analysis information by using the first field sensing information sensed by the sensor unit as an input of the stored image processing-based artificial intelligence model; and the processor unit generates a control signal for performing the situation judgment operation based on the generated first field situation analysis information; and the network unit can transmit the control signal to another device.

[0038] In addition, according to one embodiment of the present invention, the sensor unit senses second field sensing information, wherein the second field sensing information includes at least one of second field sound information, second field text information, and second field image information, and the processor unit generates second field situation analysis information by using the second field sensing information as an input of the stored image processing-based artificial intelligence model, and performs verification on the first field situation analysis information based on the second field situation analysis information, and the control signal may be generated based on the generated first field situation analysis information and the performed verification result.

[0039] Additionally, according to one embodiment of the present invention, the verification may be performed by the processor unit based on a comparison of the first field situation analysis information and the second field situation analysis information.

[0040] In addition, according to one embodiment of the present invention, the network unit may receive third field situation analysis information generated by another electronic device from another electronic device, and the processor unit may generate a control signal for performing the situation judgment operation based on at least one of the generated first field situation analysis information, the performed verification result, and the received third field situation analysis information.

[0041] Additionally, according to one embodiment of the present invention, the network unit can transmit the generated first field situation analysis information to a server.

[0042] In one embodiment of the present invention, an electronic device is disclosed for receiving field sensing information from a sensor device and performing a situation judgment operation in the field.

[0043] According to one embodiment of the present invention, an electronic device for receiving field sensing information from a sensor device and performing a situation judgment operation in the field includes a memory unit; a processor unit; and a network unit; wherein the network unit receives field sensing information including at least one of field sound information, field text information, and field image information from the sensor device, the memory unit stores an image processing-based artificial intelligence model, the processor unit generates field situation analysis information by using the field sensing information received from the network unit as an input of the stored image processing-based artificial intelligence model, the processor unit generates a control signal for performing the situation judgment operation based on the generated field situation analysis information, and the network unit can transmit the control signal to another device.

[0044] In addition, according to one embodiment of the present invention, the network unit receives second field sensing information, wherein the second field sensing information includes at least one of second field sound information, second field text information, and second field image information, from the sensor device, and the processor unit generates second field situation analysis information using the second field sensing information as an input of the stored image processing-based artificial intelligence model, and performs verification on the first field situation analysis information based on the second field situation analysis information, and the control signal may be generated based on the generated first field situation analysis information and the performed verification result.

[0045] Additionally, according to one embodiment of the present invention, the verification may be performed by the processor unit based on a comparison of the first field situation analysis information and the second field situation analysis information.

[0046] In addition, according to one embodiment of the present invention, the network unit may receive third field situation analysis information generated by another electronic device from another electronic device, and the processor unit may generate a control signal for performing the situation judgment operation based on at least one of the generated first field situation analysis information, the performed verification result, and the received third field situation analysis information.

[0047] Additionally, according to one embodiment of the present invention, the network unit can transmit the generated first field situation analysis information to a server.

[0048] In addition, according to one embodiment of the present invention, the network unit may receive update information of the artificial intelligence model generated by the server based on the field situation analysis information from the server, and the processor unit may update the artificial intelligence model based on the update information.

[0049] In addition, according to one embodiment of the present invention, the artificial intelligence model is a CNN-based model that performs multi-modal learning based on field acoustic information, field text information, and field image information, and can be learned using field sensing information, where the field sensing information includes at least one of field acoustic information, field text information, and field image information, as learning data.

[0050] Additionally, according to one embodiment of the present invention, the artificial intelligence model can be learned based on the received third field situation analysis information.

[0051] In one embodiment of the present invention, a method for performing situational judgment in the field is disclosed using an edge AI device including a sensor unit, a memory unit storing an image processing-based artificial intelligence model, and a processor unit.

[0052] According to one embodiment of the present invention, a method for performing situation judgment in a field using an edge AI device may include: sensing first field sensing information including at least one of first field sound information, first field text information, and first field image information; generating first field situation analysis information by using the sensed first field sensing information as an input to the image processing-based artificial intelligence model; and generating a control signal for performing a situation judgment operation based on the generated first field situation analysis information.

[0053] In addition, according to one embodiment of the present invention, the method further includes: sensing second field sensing information, wherein the second field sensing information includes at least one of second field sound information, second field text information, and second field image information; generating second field situation analysis information using the second field sensing information as an input of the stored image processing-based artificial intelligence model; and performing verification on the first field situation analysis information based on the second field situation analysis information; wherein the control signal may be generated based on the generated first field situation analysis information and the performed verification result.

[0054] Additionally, according to one embodiment of the present invention, the verification may be performed by the processor unit based on a comparison of the first field situation analysis information and the second field situation analysis information.

[0055] In addition, according to one embodiment of the present invention, the edge AI device may further include a network unit; and a step of receiving third field situation analysis information generated by the other edge AI device from another edge AI device; and a step of generating a control signal for performing the situation judgment operation based on at least one of the first field situation analysis information generated, the verification result performed, and the third field situation analysis information received.

[0056] In addition, according to one embodiment of the present invention, the edge AI device may further include a network unit, and may further include a step of transmitting the generated first field situation analysis information to a server.

[0057] In addition, according to one embodiment of the present invention, the method may further include a step of receiving, from the server, update information of the artificial intelligence model generated by the server based on the first field situation analysis information; and a step of updating the artificial intelligence model based on the update information.

[0058] In addition, according to one embodiment of the present invention, the artificial intelligence model is a CNN-based model that performs multi-modal learning based on field acoustic information, field text information, and field image information, and can be learned using field sensing information, where the field sensing information includes at least one of field acoustic information, field text information, and field image information, as learning data.

[0059] Additionally, according to one embodiment of the present invention, the artificial intelligence model can be learned based on the received third field situation analysis information.

[0060] The present invention provides a field facility control infrastructure utilizing edge AI, comprising: at least one edge AI system configured to generate facility control information; and an integrated control server configured to receive the facility control information from the at least one edge AI system and control the field facility based on the facility control information.

[0061] In addition, the at least one edge AI system is configured to receive location information from a user terminal of at least one user, identify personal information transmitted by the user terminal from which the location information was received to the integrated control server from the integrated control server, and sense facility environment information related to the at least one user, and the facility control information may be information generated by the at least one edge AI system based on at least one of the location information, the personal information, and the facility environment information.

[0062] In addition, the facility control information may be configured such that the integrated control server transmits user terminal control information to the user terminal, and the user terminal includes an output module, receives the user terminal control information from the integrated control server, and outputs the user terminal control information through the output module.

[0063] In addition, the edge AI system may include at least one node device configured to receive the location information from the user terminal and sense the facility environment information; a head device configured to receive the location information and the facility environment information from the at least one node device, identify personal information transmitted to the integrated control server by the user terminal from which the location information was received, and generate the facility control information by using at least one of the location information, the personal information, and the facility environment information as input to an artificial intelligence model.

[0064] Additionally, the at least one node device may be configured to obtain the location information by at least one of a UWB-based tracking method and a Wi-Fi waveform pattern analysis method.

[0065] Additionally, the at least one node device may include a sensor unit and be configured to acquire the facility environment information through the sensor unit.

[0066] In addition, the system may further include a digital twin system configured to receive the facility control information from the integrated control server, create a digital twin model for the field facility based on the facility control information, and display the same.

[0067] The present invention provides a field facility control method utilizing at least one edge AI system and an integrated control server, comprising: a step in which the edge AI system receives location information from at least one user terminal; a step in which the edge AI system identifies, from the integrated control server, personal information transmitted to the integrated control server by the user terminal from which the location information was received; a step in which the edge AI system senses facility environment information related to the at least one user; a step in which the edge AI system generates facility control information based on at least one of the location information, the personal information, and the facility environment information; and a step in which the edge AI system transmits the facility control information to the integrated control server.

[0068] Additionally, the above facility control information may be configured so that the integrated control server controls the field facility.

[0069] In addition, the facility control information may be configured such that the integrated control server transmits user terminal control information to the user terminal, and the user terminal includes an output module, receives the user terminal control information from the integrated control server, and outputs the user terminal control information through the output module.

[0070] The edge AI system of the present invention may include at least one node device configured to receive the location information from the user terminal and sense the facility environment information; a head device configured to receive the location information and the facility environment information from the at least one node device, identify personal information transmitted to the integrated control server by the user terminal from which the location information was received, and generate the facility control information by using at least one of the location information, the personal information, and the facility environment information as an input to an artificial intelligence model.

[0071] Additionally, the at least one node device may be configured to obtain the location information by at least one of a UWB-based tracking method and a Wi-Fi waveform pattern analysis method.

[0072] Additionally, the at least one node device may include a sensor unit and be configured to acquire the facility environment information through the sensor unit.

[0073] In addition, the integrated control server may be configured to transmit the facility control information to a digital twin system, and the digital twin system may be configured to receive the facility control information from the integrated control server, create a digital twin model for the field facility based on the facility control information, and display the same.

[0074] The present invention provides a field facility control method utilizing at least one edge AI system and an integrated control server, comprising: a step in which the integrated control server receives personal information from at least one user terminal; a step in which the integrated control server stores the received personal information; a step in which the integrated control server receives facility control information from at least one edge AI system; and a step in which the integrated control server controls the field facility based on the received facility control information.

[0075] In addition, the facility control information may be configured such that the integrated control server transmits user terminal control information to the user terminal, and the user terminal includes an output module, receives the user terminal control information from the integrated control server, and outputs the user terminal control information through the output module.

[0076] In addition, the at least one edge AI system is configured to receive location information from a user terminal of at least one user, identify personal information transmitted by the user terminal from which the location information was received to the integrated control server from the integrated control server, and sense facility environment information related to the at least one user, and the facility control information may be information generated by the at least one edge AI system based on at least one of the location information, the personal information, and the facility environment information.

[0077] In addition, the edge AI system may include at least one node device configured to receive the location information from the user terminal and sense the facility environment information; a head device configured to receive the location information and the facility environment information from the at least one node device, identify personal information transmitted to the integrated control server by the user terminal from which the location information was received, and generate the facility control information by using at least one of the location information, the personal information, and the facility environment information as input to an artificial intelligence model.

[0078] Additionally, the at least one node device may be configured to obtain the location information by at least one of a UWB-based tracking method and a Wi-Fi waveform pattern analysis method.

[0079] Additionally, the at least one node device may include a sensor unit and be configured to acquire the facility environment information through the sensor unit.

[0080] In addition, the integrated control server may further include a step of transmitting the facility control information to a digital twin system; and the digital twin system may be configured to receive the facility control information from the integrated control server, create a digital twin model for the field facility based on the facility control information, and display the same.

[0081] In order to solve the problem of the present invention, an edge AI-based vehicle parking management system is provided, comprising: at least one electronic device; wherein the at least one electronic device comprises: a sensor unit for obtaining first image information including the vehicle and / or the parking lot; a network unit capable of receiving second image information including the vehicle and / or the parking lot from at least one other electronic device; a memory unit in which an edge AI model is installed; and a processor unit configured to process parking-related information using the first image information and / or the second image information as inputs of the edge AI model; wherein the memory unit is configured to store at least one of the first image information, the second image information, and the parking-related information, and the network unit is capable of transmitting the first image information and / or the parking-related information to at least one other electronic device.

[0082] According to one embodiment of the present invention, the edge AI model may include an artificial intelligence model based on image processing.

[0083] According to one embodiment of the present invention, the parking-related information may include at least one of real-time location information of the vehicle, target parking space information of the vehicle, identifier information of the vehicle, structure information of the parking lot, entry time information of the vehicle, exit time information of the vehicle, and parking time information of the vehicle.

[0084] According to one embodiment of the present invention, the target parking space information of the vehicle may be information about the parking space closest to the vehicle or a reserved parking space.

[0085] According to one embodiment of the present invention, the processor unit is configured to generate route information of the vehicle based on the parking-related information, and the route information of the vehicle may be information regarding route guidance from the vehicle to the target parking space of the vehicle.

[0086] According to one embodiment of the present invention, the processor unit may be configured to generate parking fee information of the vehicle based on the parking-related information.

[0087] According to one embodiment of the present invention, the at least one electronic device further includes an output unit, and the output unit can output at least one of the first image information, the second image information, the parking-related information, the route information, and the parking fee information.

[0088] In order to solve the problem of the present invention, an edge AI-based vehicle parking management method is provided, comprising: an image information acquisition step of acquiring image information including the vehicle and / or the parking lot; a vehicle recognition step of recognizing the vehicle based on the acquired image information; a parking space recognition step of recognizing a parking space included in the parking lot based on the acquired image information; and a parking determination step of determining whether the recognized vehicle is parked in the recognized parking space; wherein the vehicle recognition step, the parking space recognition step, and the parking determination step are performed using the acquired image information as input to an edge AI model.

[0089] According to one embodiment of the present invention, if the recognized vehicle is parked in the recognized parking space, the method may further include a parking information storage step of storing an identifier of the vehicle and / or an identifier of the parking space in a memory unit.

[0090] According to one embodiment of the present invention, the edge AI model may include an artificial intelligence model based on image processing.

[0091] In order to solve the problem of the present invention, an edge AI-based personal mobility device parking management device is provided, comprising: a first sensor unit for obtaining wide-area image information including the personal mobility device and / or a parking lot; a memory unit in which an edge AI model is installed; a processor unit for determining a parking intention of the personal mobility device using the obtained wide-area image information as an input of the edge AI model; and a second sensor unit for obtaining focused image information including the personal mobility device when there is a parking intention of the personal mobility device; wherein the processor unit is configured to recognize an identifier of the personal mobility device using the obtained focused image information as an input of the edge AI model.

[0092] According to one embodiment of the present invention, the memory unit may be configured to store at least one of the wide-area image information, the focused image information, and the identifier of the personal mobility device.

[0093] According to one embodiment of the present invention, when there is an intention to park the personal transportation means, the processor unit may be configured to process parking-related information using the wide-area image information and / or the focused image information as inputs to the edge AI model.

[0094] According to one embodiment of the present invention, the edge AI model may include an artificial intelligence model based on image processing.

[0095] According to one embodiment of the present invention, the parking-related information may include at least one of target parking space information of the personal transportation means, entry time information of the personal transportation means, exit time information of the personal transportation means, and parking time information of the personal transportation means.

[0096] According to one embodiment of the present invention, the target parking space information of the personal transportation means may be information about the parking space closest to the personal transportation means.

[0097] According to one embodiment of the present invention, the processor unit may be configured to generate parking fee information of the personal transportation means based on the parking-related information.

[0098] According to one embodiment of the present invention, the edge AI-based personal mobility device parking management device further includes an output unit, and the output unit can output at least one of the wide-area image information, the focused image information, the identifier of the personal mobility device, the parking-related information, and the parking fee information.

[0099] In order to solve the problem of the present invention, a method for managing parking of a personal mobility device based on edge AI is provided, comprising: a wide-area image information acquisition step of acquiring wide-area image information including the personal mobility device and / or a parking lot; a parking intention determination step of determining a parking intention of the personal mobility device by using the acquired wide-area image information as an input of an edge AI model; a focused image information acquisition step of acquiring focused image information including the personal mobility device when there is a parking intention of the personal mobility device; and a personal mobility device identifier recognition step of recognizing an identifier of the personal mobility device by using the acquired focused image information as an input of the edge AI model.

[0100] According to one embodiment of the present invention, the method may further include a storage step of storing at least one of the wide-area image information, the focused image information, and the identifier of the personal mobility device in a memory unit.

[0101] According to one embodiment of the present invention, if there is an intention to park the personal transportation device, the method may further include a parking-related information acquisition step of acquiring parking-related information by using the wide-area image information and / or the focused image information as inputs to the edge AI model.

[0102] According to one embodiment of the present invention, the edge AI model may include an artificial intelligence model based on image processing.

[0103] According to one embodiment of the present invention, the parking-related information may include at least one of target parking space information of the personal transportation means, entry time information of the personal transportation means, exit time information of the personal transportation means, and parking time information of the personal transportation means.

[0104] According to one embodiment of the present invention, the target parking space information of the personal transportation means may be information about the parking space closest to the personal transportation means.

[0105] According to one embodiment of the present invention, the edge AI-based personal mobility device parking management method may further include a parking fee information generation step of generating parking fee information for the personal mobility device based on the parking-related information.

[0106] According to one embodiment of the present invention, the method may further include an output step of outputting at least one of the wide-area image information, the focused image information, the identifier of the personal transportation means, the parking-related information, and the parking fee information.

[0107] According to one embodiment of the present invention, a server device for providing personalized artificial intelligence includes a memory unit; a processor unit; and a communication unit; wherein the memory unit stores at least one artificial intelligence model, the processor unit generates an artificial intelligence platform storing the at least one artificial intelligence model, and the communication unit can transmit a first artificial intelligence model stored in the artificial intelligence platform to an edge device.

[0108] Additionally, according to one embodiment of the present invention, the communication unit can receive, from the edge device, a second artificial intelligence model learned based on the first-first personalized information transmitted by the first artificial intelligence model to the edge device.

[0109] Additionally, according to one embodiment of the present invention, the communication unit can transmit the second artificial intelligence model to another edge device.

[0110] Additionally, according to one embodiment of the present invention, the communication unit can receive a third artificial intelligence model additionally learned based on the second personalized information from the other edge device.

[0111] Additionally, according to one embodiment of the present invention, the first artificial intelligence model may be configured to perform multi-modal learning based on field data.

[0112] Additionally, according to one embodiment of the present invention, the second artificial intelligence model or the third artificial intelligence model may be configured to perform multi-modal learning based on personalized information.

[0113] In another embodiment of the present invention, an edge device for providing personalized artificial intelligence is disclosed.

[0114] According to one embodiment of the present invention, an edge device for providing personalized artificial intelligence includes a memory unit; a processor unit; a communication unit; and an input unit; wherein the communication unit receives a first artificial intelligence model from an artificial intelligence platform storing at least one artificial intelligence model, and the memory unit can store the received first artificial intelligence model.

[0115] In addition, according to one embodiment of the present invention, the input unit may receive first-first personalized information in an environment in which the first artificial intelligence model operates, the processor unit may generate a second artificial intelligence model learned based on the stored first artificial intelligence model and the input first-first personalized information, and the communication unit may transmit the generated second artificial intelligence model to the artificial intelligence platform.

[0116] In addition, according to one embodiment of the present invention, the communication unit receives the second artificial intelligence model received from another edge device from the artificial intelligence platform, and the second artificial intelligence model may be learned based on the first artificial intelligence model received by the other terminal device from the artificial intelligence platform based on the first-second personalized information.

[0117] In addition, according to one embodiment of the present invention, the input unit may receive second personalized information in an environment in which the second artificial intelligence model operates, the processor unit may generate a third artificial intelligence model by additionally training the second artificial intelligence model based on the second personalized information, and the communication unit may transmit the generated third artificial intelligence model to the artificial intelligence platform.

[0118] Additionally, according to one embodiment of the present invention, the artificial intelligence platform may be configured to modularize the at least one artificial intelligence model based on the purpose of use.

[0119] Additionally, according to one embodiment of the present invention, the first artificial intelligence model may be configured to perform multi-modal learning based on field data.

[0120] Additionally, according to one embodiment of the present invention, the second artificial intelligence model or the third artificial intelligence model may be configured to perform multi-modal learning based on personalized information.

[0121] In another embodiment of the present invention, a system for providing personalized artificial intelligence based on an artificial intelligence platform is disclosed.

[0122] According to one embodiment of the present invention, a system for providing personalized artificial intelligence based on an artificial intelligence platform includes: a computing device; a server device; and an edge device; wherein the computing device learns at least one artificial intelligence model based on field data and transmits the learned artificial intelligence model to the server device; the server device receives the at least one artificial intelligence model from the computing device and generates an artificial intelligence platform that stores the at least one artificial intelligence model; and the edge device can receive a first artificial intelligence model stored in the artificial intelligence platform from the server device.

[0123] In addition, according to one embodiment of the present invention, the edge device may receive first-first personalized information in a field where the first artificial intelligence model is used, generate a second artificial intelligence model learned based on the first-first personalized information inputted, and transmit the second artificial intelligence model to the server device.

[0124] Additionally, according to one embodiment of the present invention, the server device can transmit the second artificial intelligence model to another edge device, and the other edge device can receive the second artificial intelligence model from the server device.

[0125] In addition, according to one embodiment of the present invention, the other edge device may receive second personalized information in an environment in which the second artificial intelligence model operates, generate a third artificial intelligence model based on the input second personalized information, and transmit the generated third artificial intelligence model to the server device.

[0126] Additionally, according to one embodiment of the present invention, the artificial intelligence platform may be configured to modularize the at least one artificial intelligence model based on the purpose of use.

[0127] Additionally, according to one embodiment of the present invention, the first artificial intelligence model may be configured to perform multi-modal learning based on field data.

[0128] Additionally, according to one embodiment of the present invention, the second artificial intelligence model or the third artificial intelligence model may be configured to perform multi-modal learning based on personalized information.

[0129] In order to solve the problem of the present invention, a field facility control infrastructure utilizing edge AI is provided, comprising at least one edge AI system; wherein the at least one edge AI system comprises at least one node device configured to receive location information in real time from at least one user terminal; and a head device configured to receive the location information from the node device and track the location of the at least one user terminal in real time based on the location information.

[0130] According to one embodiment of the present invention, there is provided an integrated control server configured to receive personal information from at least one user terminal and store the personal information;

[0131] In addition, the head device may be configured to identify the personal information transmitted by the at least one user terminal from which the location information is received to the integrated control server from the integrated control server.

[0132] According to one embodiment of the present invention, the head device may be configured to generate space-specific residence time information within the field facility for the at least one user terminal based on the location information.

[0133] According to one embodiment of the present invention, the head device may be configured to generate facility control information by using at least one of the location information, the personal information, and the space-specific residence time information as inputs to an artificial intelligence model, and transmit the facility control information to the integrated control server.

[0134] According to one embodiment of the present invention, the facility control information may be configured such that the integrated control server transmits user terminal control information to at least one user terminal.

[0135] According to one embodiment of the present invention, the facility control information may be configured so that the integrated control server controls the field facility.

[0136] In order to solve the problem of the present invention, a field facility control infrastructure utilizing edge AI is provided, including: an integrated control server configured to receive personal information from at least one first user terminal and store the personal information; and at least one edge AI system configured to receive the personal information from the integrated control server, generate facility control information using the personal information as an input to an artificial intelligence model, and transmit the facility control information to the integrated control server; wherein the facility control information is configured to transmit user terminal control information from the integrated control server to at least one second user terminal.

[0137] According to one embodiment of the present invention, the at least one edge AI system comprises at least one node device configured to receive location information in real time from the at least one first user terminal; and a head device configured to receive the location information from the node device and track the location of the at least one first user terminal in real time based on the location information; wherein the head device may be configured to generate space-specific residence time information within the field facility for the at least one first user terminal based on the location information.

[0138] According to one embodiment of the present invention, the head device is configured to generate the facility control information by using at least one of the personal information, the location information, and the space-specific residence time information as inputs to the artificial intelligence model, and to transmit the facility control information to the integrated control server, and the facility control information may be configured to allow the integrated control server to transmit the user terminal control information to the at least one second user terminal.

[0139] In order to solve the problem of the present invention, a field facility control infrastructure utilizing edge AI is provided, including: an integrated control server configured to receive personal information from at least one user terminal and store the personal information; and at least one edge AI system configured to receive the personal information from the integrated control server and generate user analysis information on users using the field facility by using the personal information as input to an artificial intelligence model.

[0140] According to one embodiment of the present invention, the at least one edge AI system comprises at least one node device configured to receive location information in real time from the at least one user terminal; and a head device configured to receive the location information from the node device and track the location of the at least one user terminal in real time based on the location information; wherein the head device may be configured to generate the user analysis information by using at least one of the location information and personal information as inputs to an artificial intelligence model.

[0141] According to one embodiment of the present invention, the head device may be configured to generate space-specific residence time information for the at least one user terminal within the field facility based on the location information, and generate the user analysis information by using at least one of the location information, the personal information, and the space-specific residence time information as inputs to the artificial intelligence model.

[0142] In order to solve the problem of the present invention, a field facility control method utilizing at least one edge AI system is provided, comprising: a step in which at least one node device included in the at least one edge AI system receives location information in real time from at least one user terminal; a step in which a head device included in the at least one edge AI system receives the location information from the node device; and a step in which the head device tracks the location of the at least one user terminal in real time based on the location information.

[0143] According to one embodiment of the present invention, the head device is configured to identify personal information transmitted from the at least one user terminal from which the location information has been received to the integrated control server, and the integrated control server may be configured to receive the personal information from the at least one user terminal and store the personal information.

[0144] According to one embodiment of the present invention, the head device may be configured to generate space-specific residence time information within the field facility for the at least one user terminal based on the location information.

[0145] According to one embodiment of the present invention, the head device may be configured to generate facility control information by using at least one of the location information, the personal information, and the space-specific residence time information as inputs to an artificial intelligence model, and transmit the facility control information to the integrated control server.

[0146] According to one embodiment of the present invention, the facility control information may be configured such that the integrated control server transmits user terminal control information to at least one user terminal.

[0147] According to one embodiment of the present invention, the facility control information may be configured so that the integrated control server controls the field facility.

[0148] In order to solve the problem of the present invention, a field facility control method utilizing at least one edge AI system is provided, comprising: a step in which an integrated control server receives personal information from at least one first user terminal and stores the personal information; a step in which the at least one edge AI system receives the personal information from the integrated control server; a step in which the at least one edge AI system generates facility control information using the personal information as an input to an artificial intelligence model; a step in which the integrated control server receives the facility control information from the at least one edge AI system; a step in which the integrated control server generates user terminal control information based on the facility control information; and a step in which the integrated control server transmits the user terminal control information to at least one second user terminal.

[0149] According to one embodiment of the present invention, the at least one edge AI system comprises at least one node device configured to receive location information in real time from the at least one first user terminal; and a head device configured to receive the location information from the node device and track the location of the at least one first user terminal in real time based on the location information; wherein the head device may be configured to generate space-specific residence time information within the field facility for the at least one first user terminal based on the location information.

[0150] According to one embodiment of the present invention, the head device is configured to generate the facility control information by using at least one of the personal information, the location information, and the space-specific residence time information as inputs to the artificial intelligence model, and to transmit the facility control information to the integrated control server, and the facility control information may be configured to allow the integrated control server to transmit the user terminal control information to the at least one second user terminal.

[0151] In order to solve the problem of the present invention, a method for controlling a field facility using at least one edge AI system is provided, comprising: a step in which an integrated control server receives personal information from at least one user terminal and stores the personal information; a step in which the at least one edge AI system receives the personal information from the integrated control server; and a step in which the at least one edge AI system generates user analysis information on a user using the field facility using the personal information as an input to an artificial intelligence model.

[0152] According to one embodiment of the present invention, the at least one edge AI system comprises at least one node device configured to receive location information in real time from the at least one user terminal; and a head device configured to receive the location information from the at least one node device and track the location of the at least one user terminal in real time based on the location information; wherein the head device may be configured to generate the user analysis information by using at least one of the location information and personal information as inputs to an artificial intelligence model.

[0153] According to one embodiment of the present invention, the head device may be configured to generate space-specific residence time information for the at least one user terminal within the field facility based on the location information, and generate the user analysis information by using at least one of the location information, the personal information, and the space-specific residence time information as inputs to the artificial intelligence model.

[0154] The present invention relates to an edge AI device for object tracking and object-specific spatial residence information collection, comprising: a sensor unit for sensing an object in a designated area; a communication unit for communicating with another edge AI device; a memory unit; and a processor unit; wherein the processor unit analyzes an object sensed by the sensor unit to output object analysis information, stores movement path information and residence time information of the analyzed object in the memory unit, and creates object-specific spatial residence information based on the output object analysis information, the stored movement path information, and the stored residence time information.

[0155] Additionally, the processor unit is configured to analyze an object sensed by the sensor unit in real time and output object analysis information.

[0156] Additionally, the object analysis information is information output based on at least one of size information of the object, appearance information of the object, or color information of the object.

[0157] Additionally, the object analysis information is information composed of at least one of age information of the object, gender information of the object, or race information of the object.

[0158] In addition, the processor unit is configured to transmit at least one of the output object analysis information, the stored movement path information, the stored residence time information, and the created object-specific spatial residence information to the other edge AI device through the communication unit when the object moves and the sensor unit can no longer sense the object in the designated area.

[0159] The present invention relates to an object tracking and spatial information collection system, wherein the system comprises a first edge AI device and a second edge AI device, wherein the first edge AI device comprises: a sensor unit for sensing an object in a designated area; a communication unit for communicating with the second edge AI device; a memory unit; and a processor unit, wherein the processor unit analyzes an object sensed by the sensor unit to output object analysis information, stores movement path information and residence time information of the analyzed object in the memory unit, creates spatial residence information for each object based on the output object analysis information, the stored movement path information, and the stored residence time information, and transmits at least one piece of information from among the output object analysis information, the stored movement path information, the stored residence time information, and the created spatial information to the second edge AI device through the communication unit.

[0160] Additionally, the processor unit is configured to analyze an object sensed by the sensor unit in real time and output object analysis information.

[0161] Additionally, the object analysis information is information output based on at least one of size information of the object, appearance information of the object, or color information of the object.

[0162] Additionally, the object analysis information is information composed of at least one of age information of the object, gender information of the object, or race information of the object.

[0163] In addition, the processor unit is configured to transmit at least one of the generated object analysis information, the stored movement path information, the stored residence time information, and the generated spatial information to the second edge AI device through the communication unit when the object moves and the sensor unit can no longer sense the object in the designated area.

[0164] The present invention relates to an edge AI device for object tracking and collecting object-specific spatial residence information, comprising: at least one sensor unit for sensing an object in a designated area; a communication unit for communicating with a central server; a memory unit; and a processor unit; wherein the processor unit analyzes an object sensed by the at least one sensor unit to output object analysis information, stores movement path information and residence time information of the analyzed object in the memory unit, creates object-specific spatial residence information based on the output object analysis information, the stored movement path information, and the stored residence time information, and transmits the created object-specific spatial residence information to the central server via the communication unit.

[0165] The present invention relates to a system for object tracking and object-specific spatial residence information collection, wherein the system includes an edge AI device, wherein the edge AI device includes at least one sensor unit for sensing an object in a designated area; a communication unit for communicating with a central server; a memory unit; and a processor unit, wherein the processor unit analyzes an object sensed by the at least one sensor unit to output object analysis information, stores movement path information and residence time information of the analyzed object in the memory unit, creates object-specific spatial residence information based on the output object analysis information, the stored movement path information, and the stored residence time information, and transmits at least one piece of information among the output object analysis information, the stored movement path information, the stored residence time information, and the created spatial information to the central server through the communication unit.

[0166] The present invention relates to a system for object tracking and object-specific spatial residence information collection, wherein the system comprises a first edge AI device and a second edge AI device, wherein the first edge AI device comprises: at least one sensor unit for sensing an object in a designated area; a communication unit for communicating with the second edge AI device and a central server; a memory unit; and a processor unit, wherein the communication unit receives ID information about an object that has entered a space in which the first edge AI device is located from the second edge AI device, the processor unit analyzes an object sensed by the at least one sensor unit to output object analysis information, confirms whether the output object analysis information matches the received ID information, stores movement path information and residence time information of the analyzed object in the memory unit, creates object-specific spatial residence information based on the output object analysis information, the stored movement path information, and the stored residence time information, and transmits at least one or more pieces of information from among the output object analysis information, the stored movement path information, the stored residence time information, and the created spatial information to the central server through the communication unit.

[0167] In addition, the received ID information is at least one of object analysis information for the object output by the second edge AI device, movement path information of the object stored in the second edge AI device, residence time information of the object stored in the second edge AI device, or object-specific spatial residence information of the object created by the second edge AI device.

[0168] The present invention relates to an edge AI device for object tracking and object-specific spatial residence information collection, comprising: a sensor unit for sensing an object in a designated area; a communication unit for communicating with another edge AI device or another terminal; a memory unit for storing unique information necessary for identification of the object; and a processor unit; wherein the processor unit is configured to match the unique information stored in the memory unit with an object sensed by the sensor unit, store movement path information and residence time information of the matched object in the memory unit, and create spatial residence information of the sensed object based on the unique information necessary for identification of the matched object, the stored movement path information, and the stored residence time information.

[0169] In addition, the processor unit is configured to transmit external access-related information to the other terminal or the external server through the communication unit when the unique information stored in the memory unit does not match the object sensed by the sensor unit.

[0170] In addition, the processor unit is configured to transmit, to the other edge AI device through the communication unit, at least one of the unique information necessary for identifying the matched object, the stored movement path information, the stored residence time information, and the spatial residence information of the created object, when the matched object moves and the sensor unit can no longer sense the matched object in the designated area.

[0171] In addition, the sensor unit senses the face of an object in the designated area, and the processor unit is configured to match the unique information stored in the memory unit with the sensed object based on the face of the object sensed by the sensor unit.

[0172] In addition, the sensor unit senses the iris of an object in the designated area, and the processor unit is configured to match the unique information stored in the memory unit with the sensed object based on the iris of the object sensed by the sensor unit.

[0173] In addition, the memory unit stores unique information necessary for identifying the object and access restriction information related to the unique information, and the processor unit is configured to determine whether the stored movement path information and residence time information are included in the stored access restriction information.

[0174] Additionally, the processor unit is configured to transmit an access restriction notification signal to the other terminal when it is determined that the stored movement path information and residence time information are included in the stored access restriction information.

[0175] The present invention relates to a system for object tracking and collecting information on the object's spatial facility stay, wherein the system comprises a first edge AI device and a second edge AI device, wherein the first edge AI device comprises: a sensor unit for sensing an object in a designated area; a communication unit for communicating with the second edge AI device or the spatial facility; a memory unit for storing unique information necessary for identifying the object; and a processor unit, wherein the processor unit matches the unique information stored in the memory unit with an object sensed by the sensor unit, stores movement path information and stay time information of the matched object in the memory unit, creates spatial stay information of the sensed object based on the unique information necessary for identifying the matched object, the stored movement path information, and the stored stay time information, and transmits at least one or more pieces of information from among the unique information necessary for identifying the matched object, the stored movement path information, the stored stay time information, and the created spatial stay information to the second edge AI device via the communication unit.

[0176] In addition, the processor unit is configured to transmit external access-related information to the other terminal or the external server through the communication unit when the unique information stored in the memory unit does not match the object sensed by the sensor unit.

[0177] In addition, the processor unit is configured to transmit, to the other edge AI device through the communication unit, at least one of the unique information necessary for identifying the matched object, the stored movement path information, the stored residence time information, and the spatial residence information of the created object, when the matched object moves and the sensor unit can no longer sense the matched object in the designated area.

[0178] In addition, the sensor unit senses the face of an object in the designated area, and the processor unit is configured to match the unique information stored in the memory unit with the sensed object based on the face of the object sensed by the sensor unit.

[0179] In addition, the sensor unit senses the iris of an object in the designated area, and the processor unit is configured to match the unique information stored in the memory unit with the sensed object based on the iris of the object sensed by the sensor unit.

[0180] In addition, the memory unit stores unique information necessary for identifying the object and access restriction information related to the unique information, and the processor unit is configured to determine whether the stored movement path information and residence time information are included in the stored access restriction information.

[0181] Additionally, the processor unit is configured to transmit an access restriction notification signal to the other terminal when it is determined that the stored movement path information and residence time information are included in the stored access restriction information.

[0182] The present invention relates to an edge AI device for object tracking and object-specific spatial residence information collection, comprising: a sensor unit for sensing an object in a designated area; a communication unit for communicating with another edge AI device or another terminal; a memory unit for storing unique information necessary for identifying the object and access restriction information related to the unique information; and a processor unit; wherein the processor unit is configured to match the unique information stored in the memory unit with an object sensed by the sensor unit, store movement path information and residence time information of the matched object in the memory unit, determine whether the stored movement path information and residence time information are included in the stored access restriction information, and transmit an access restriction notification signal to the other terminal when it is determined that the stored movement path information and residence time information are included in the stored access restriction information.

[0183] In addition, the processor unit is configured to transmit, to the other edge AI device through the communication unit, at least one of the unique information necessary for identifying the matched object, the stored movement path information, the stored residence time information, and the spatial residence information of the created object, when the matched object moves and the sensor unit can no longer sense the matched object in the designated area.

[0184] In addition, the sensor unit senses the face of an object in the designated area, and the processor unit is configured to match the unique information stored in the memory unit with the sensed object based on the face of the object sensed by the sensor unit.

[0185] In addition, the sensor unit senses the iris of an object in the designated area, and the processor unit is configured to match the unique information stored in the memory unit with the sensed object based on the iris of the object sensed by the sensor unit.

[0186] In addition, in the object tracking and object-specific space residence information collection system, the system includes a first edge AI device and a second edge AI device, and the first edge AI device includes: a sensor unit for sensing an object in a designated area; a communication unit for communicating with the second edge AI device; a memory unit for storing unique information necessary for identifying the object and access restriction information related to the unique information; and a processor unit; wherein the processor unit matches the unique information stored in the memory unit with an object sensed by the sensor unit, stores movement path information and residence time information of the matched object in the memory unit, determines whether the stored movement path information and residence time information are included in the stored access restriction information, and, when it is determined that the stored movement path information and residence time information are included in the stored access restriction information, transmits an access restriction notification signal to the other terminal, and transmits at least one or more pieces of information from among the unique information necessary for identifying the matched object, the stored movement path information, the stored residence time information, and the created space residence information to the second edge AI device through the communication unit.

[0187] In addition, the processor unit is configured to transmit, to the other edge AI device through the communication unit, at least one of the unique information necessary for identifying the matched object, the stored movement path information, the stored residence time information, and the spatial residence information of the created object, when the matched object moves and the sensor unit can no longer sense the matched object in the designated area.

[0188] In addition, the sensor unit senses the face of an object in the designated area, and the processor unit is configured to match the unique information stored in the memory unit with the sensed object based on the face of the object sensed by the sensor unit.

[0189] In addition, the sensor unit senses the iris of an object in the designated area, and the processor unit is configured to match the unique information stored in the memory unit with the sensed object based on the iris of the object sensed by the sensor unit.

[0190] According to the present invention, a system can be provided in which an edge AI device is installed in a security-related location such as a research lab to identify an object and then continuously track the identified object.

[0191] According to the present invention, distributed processing using an edge AI device enables local real-time data processing, thereby reducing delay time and saving network bandwidth.

[0192] According to the present invention, there is an effect of providing a detailed pattern analysis of a specific object by accumulating information on the movement path of a specific object and the time of stay in a specific area.

[0193] The on-site facility control infrastructure utilizing the edge AI system according to the present invention can obtain location information, space-specific residence time information, and human information on on-site facility users through the edge AI system and integrated control server, and can immediately control the on-site facility based on this.

[0194] According to the present invention, the AI ​​platform can increase the accessibility of AI technology by providing a variety of AI models based on intended use. Furthermore, after downloading an AI model from the AI ​​platform, users can use the downloaded AI model on edge devices without a network connection, thereby addressing security and privacy concerns.

[0195] According to the present invention, an AI model optimized for individual needs can be created by additionally training the AI ​​model based on personalized information in the environment in which the model operates. Accordingly, users can share their personalized AI model on the AI ​​platform, allowing other users to experience it as well, thereby enhancing the accessibility and fairness of AI technology.

[0196] According to the present invention, by utilizing edge AI to provide on-site, customized parking management services suitable for various types of parking lots, it can contribute to alleviating parking shortages in urban areas and reducing parking search times. Furthermore, it can contribute to increasing the convenience of parking lot operations.

[0197] Additionally, parking reservation and parking fee payment services can provide drivers with a convenient parking experience.

[0198] According to the present invention, by utilizing edge AI to provide on-site, customized parking management services suitable for various types of parking lots, it can contribute to solving parking shortages in urban areas and reducing parking search times. Furthermore, it can contribute to increasing the convenience of parking lot operations. Furthermore, by supporting parking reservations, parking guidance, and parking fee payment services, it can provide drivers with a convenient parking experience.

[0199] The present invention utilizes edge AI to obtain location information, personal information, and facility environment information about users of on-site facilities, thereby enabling immediate control of on-site facilities.

[0200] According to the present invention, edge AI devices can analyze and assess on-site situations without a network connection to a central server or cloud, contributing to a quick and accurate response to on-site situations in real time. Furthermore, the present invention addresses security and privacy concerns by enabling edge AI devices to immediately process on-site data without a network connection to a central server or cloud. Furthermore, the present invention enables improved decision-making by transmitting and receiving on-site data and on-site situation analysis information from each edge AI device over a network.

[0201] According to the present invention, by utilizing edge AI to provide on-site, customized parking management services suitable for various types of parking lots, it can contribute to alleviating parking shortages in urban areas and reducing parking search times. Furthermore, it can contribute to increasing the convenience of parking lot operations.

[0202] Additionally, parking reservation and parking fee payment services can provide drivers with a convenient parking experience.

[0203] Figure 1a is a block diagram illustrating a driver assistance system using edge AI according to the present invention.

[0204] FIG. 1b is a diagram for explaining a communication network between an external terminal (100), an AR device (20), and an electronic device (10) constituting an infrastructure of multiple sensor devices installed on a road according to one embodiment of the present invention.

[0205] Figure 1c is a block diagram for explaining an external terminal (100) according to one embodiment of the present invention.

[0206] FIG. 2 is a drawing for explaining an embodiment in which an AR device (20) receives data from an electronic device (10) that constitutes a plurality of sensor device infrastructures installed on a road according to the present invention.

[0207] FIG. 3 is a drawing for explaining an embodiment in which, when there is a pedestrian located in a blind spot (31) of a vehicle (30) of a driver wearing an AR device according to one embodiment of the present invention, an AR device (20) receives data from an electronic device (10) constituting a plurality of sensor device infrastructures indicating that a pedestrian is located in a blind spot (31).

[0208] FIG. 4a is a drawing for explaining an embodiment in which, when a contact accident occurs between a vehicle (30-5, 30-6) of a driver wearing an AR device according to one embodiment of the present invention, an external terminal (100) recognizes this and transmits information about the occurrence of a contact accident to an AR device (20) of a vehicle (30-1, 30-2, 30-3, 30-4) of a driver wearing another AR device.

[0209] FIG. 4b is a drawing showing an augmented reality image projected in the form of a 2D map in the field of view of a vehicle driver when a contact accident occurs between a vehicle (30-5, 30-6) of a driver wearing an AR device according to one embodiment of the present invention, the AR device determines that there has been a contact accident and communicates this to an external terminal (100), and the external terminal (100) recognizes this and transmits information about whether a contact accident has occurred to an AR device (20) of a vehicle (30-1, 30-2, 30-3, 30-4) of a driver wearing another AR device.

[0210] FIG. 4c is a drawing showing an augmented reality image (21-1) indicating the point of occurrence of the contact accident projected into the field of vision of a vehicle driver when, in the event of a contact accident between a vehicle (30-5, 30-6) of a driver wearing an AR device according to one embodiment of the present invention, the AR device determines that there has been a contact accident and communicates this to an external terminal (100), and the external terminal (100) recognizes this and transmits information about whether or not a contact accident has occurred to an AR device (20) of a vehicle (30-1, 30-2, 30-3, 30-4) of a driver wearing another AR device.

[0211] FIG. 5a is a drawing for explaining an embodiment of transmitting vehicle usage method information of a vehicle (30) of a driver wearing an AR device to a vehicle driver from an external terminal (100) according to one embodiment of the present invention.

[0212] FIG. 5b is a drawing showing an augmented reality image (21-1) projected into the field of vision of a vehicle driver when information on how to use a vehicle (30) of a driver wearing an AR device is transmitted to the vehicle driver from an external terminal (100) according to one embodiment of the present invention.

[0213] FIG. 6 is a drawing showing an augmented reality image (21-1) that provides distance information from a preceding vehicle among one embodiment of an augmented reality image (21-1) displayed on a display unit (21) of an AR device (20) according to the present invention.

[0214] FIG. 7 is a drawing showing an augmented reality image (21-1) that highlights a lane on a rainy day among one example of an augmented reality image (21-1) displayed on a display unit (21) of an AR device (20) according to the present invention.

[0215] FIG. 8 is a drawing for explaining an embodiment of guiding a vehicle driver to an optimal parking space considering the destination of the vehicle driver through communication with an external terminal (100) and electronic devices (10-1, 10-2) constituting a plurality of sensor device infrastructures when the vehicle (30) of the driver wearing the AR device according to the present invention enters a parking lot.

[0216] FIG. 9 is a drawing showing an augmented reality image that appears to guide the driver to an optimal parking space considering the driver's destination through communication with an external terminal and an electronic device constituting a plurality of sensor device infrastructures when the driver's vehicle wearing the AR device according to the present invention enters a parking lot.

[0217] FIG. 10 is a drawing showing an augmented reality image that provides a path from a parking space in a building to a destination when a vehicle of a driver wearing an AR device according to the present invention enters a parking lot, and the optimal parking space considering the destination of the driver is guided through communication with an electronic device constituting an external terminal and a plurality of sensor device infrastructure.

[0218] Figure 11 is a diagram showing a field facility control infrastructure utilizing edge AI according to the present invention.

[0219] FIG. 12A is a diagram illustrating an edge AI infrastructure comprising at least one edge AI system according to the present invention.

[0220] FIG. 12b is a diagram illustrating an edge AI system comprising a head device and at least one node device according to the present invention.

[0221] FIG. 12c is a diagram showing a node device configured to receive location information from a user terminal according to the present invention and sense facility environment information from a field facility.

[0222] FIG. 12d is a drawing showing a head device including a network unit, a memory unit, and a processor unit according to the present invention.

[0223] FIG. 12e is a drawing showing a head device configured to transmit facility control information according to the present invention to an integrated control server.

[0224] FIG. 12f is a schematic diagram illustrating one or more network functions for generating facility control information according to the present invention.

[0225] FIG. 13A is a diagram showing an integrated control server configured to receive personal information from a user terminal according to the present invention and to allow an edge AI system to identify the personal information through the integrated control server.

[0226] FIG. 13b is a diagram showing an integrated control server configured to control field facilities based on facility control information according to the present invention.

[0227] FIG. 14 is a diagram showing a digital twin system that can receive facility control information from an integrated control server according to the present invention and create a digital twin for a field facility based on the facility control information.

[0228] FIG. 15A is a diagram illustrating a field facility control method utilizing at least one edge AI system and an integrated control server according to one embodiment of the present invention.

[0229] Figure 15b is a drawing for explaining in detail the field facility control method illustrated in Figure 15a.

[0230] FIGS. 16A to 16F are diagrams illustrating one embodiment of a field facility control infrastructure utilizing edge AI according to the present invention.

[0231] FIGS. 17a to 17e are drawings illustrating another embodiment of a field facility control infrastructure utilizing edge AI according to the present invention.

[0232] FIGS. 18a to 18f are drawings illustrating another embodiment of a field facility control infrastructure utilizing edge AI according to the present invention.

[0233] FIGS. 19a to 19f are drawings illustrating another embodiment of a field facility control infrastructure utilizing edge AI according to the present invention.

[0234] FIGS. 20A to 20E are drawings illustrating another embodiment of a field facility control infrastructure utilizing edge AI according to the present invention.

[0235] FIGS. 21A to 21K are drawings illustrating another embodiment of a field facility control infrastructure utilizing edge AI according to the present invention.

[0236] Figure 22 is a block diagram showing the configuration of an edge AI-based vehicle parking management system according to the present invention.

[0237] Figure 23 is a block diagram showing another configuration of an edge AI-based vehicle parking management system according to the present invention.

[0238] Figure 24 is an exemplary diagram illustrating the operation of an edge AI-based vehicle parking management system according to the present invention.

[0239] Figure 25 is an exemplary diagram for explaining the operation of an edge AI-based vehicle parking management system according to the present invention and a special parking area.

[0240] Figure 26 is an exemplary diagram for explaining the operation and role of the output unit of the edge AI-based vehicle parking management system according to the present invention.

[0241] Figure 27 (a) is a drawing showing the front part of a vehicle according to the present invention, and Figure 27 (b) is a drawing showing the rear part of a vehicle according to the present invention.

[0242] Figure 28 is a conceptual diagram illustrating information transmission and reception between at least one electronic device according to the present invention.

[0243] Figure 29 is a drawing showing path information according to the present invention.

[0244] Figure 30 is a drawing showing parking fee information according to the present invention.

[0245] Figure 31 is a flowchart illustrating a vehicle parking management method based on edge AI according to the present invention.

[0246] Figure 32 is a block diagram showing the configuration of an edge AI-based personal mobility device parking management device according to the present invention.

[0247] Figure 33 is a block diagram showing another configuration of an edge AI-based personal mobility device parking management device according to the present invention.

[0248] Figure 34 is a diagram briefly illustrating the operation of an edge AI-based personal mobility device parking management device according to the present invention.

[0249] Figure 35 is an exemplary diagram illustrating the operation of an edge AI-based personal mobility device parking management device according to the present invention.

[0250] Figure 36 is an exemplary diagram for explaining the operation and role of the output unit of an edge AI-based personal mobility device parking management device according to the present invention.

[0251] FIG. 37 is a drawing for explaining how a processor unit according to the present invention determines the parking intention of a personal mobility device.

[0252] Figure 38 is a drawing for explaining how a processor unit according to the present invention recognizes an identifier of a personal mobility device.

[0253] Figure 39 is a drawing showing parking fee information according to the present invention.

[0254] Figure 40 is a flowchart illustrating a personal mobility device parking management method based on edge AI according to the present invention.

[0255] Figure 41 is a conceptual diagram illustrating the configuration of an edge AI-based field situation judgment system according to one embodiment of the present invention.

[0256] FIG. 42a is a block diagram illustrating an edge AI device according to one embodiment of the present invention.

[0257] FIG. 42b is a block diagram illustrating an edge AI device according to one embodiment of the present invention.

[0258] FIG. 42c is a block diagram illustrating an edge AI device according to one embodiment of the present invention.

[0259] Figure 43 is a schematic diagram showing one or more network functions that constitute an artificial intelligence model for performing field situation judgment according to one embodiment of the present invention.

[0260] Figure 44a is a flowchart for explaining a multimodal learning method of an artificial intelligence model according to one embodiment of the present invention.

[0261] Figure 44b is a flowchart for explaining a multimodal learning method of an artificial intelligence model according to one embodiment of the present invention.

[0262] Figure 44c is a flowchart for explaining a multimodal learning method of an artificial intelligence model according to one embodiment of the present invention.

[0263] Figure 44d is a flowchart for explaining a multimodal learning method of an artificial intelligence model according to one embodiment of the present invention.

[0264] Figure 45 is a flowchart illustrating the creation and learning of an edge AI model according to one embodiment of the present invention.

[0265] FIG. 46a is a diagram illustrating field acoustic information on a highway according to one embodiment of the present invention.

[0266] FIG. 46b is a diagram illustrating field image information on a highway according to one embodiment of the present invention.

[0267] Figure 47 is a flowchart for explaining an edge AI-based field situation judgment method according to one embodiment of the present invention.

[0268] Figure 48 is a flowchart for explaining an edge AI-based field situation judgment method according to one embodiment of the present invention.

[0269] FIG. 49a is a conceptual diagram illustrating an interaction method between edge AI devices according to one embodiment of the present invention.

[0270] Figure 49b is a flowchart illustrating a method for judging field situations based on interaction between edge AI devices according to one embodiment of the present invention.

[0271] Figure 50 is a flowchart illustrating a method for performing an update of an edge AI model according to one embodiment of the present invention.

[0272] FIG. 51 is a diagram illustrating an edge AI system for responding to a dangerous situation according to one embodiment of the present invention.

[0273] Figure 52 is a conceptual diagram illustrating the configuration of a system for providing personalized artificial intelligence according to one embodiment of the present invention.

[0274] FIG. 53 is a block diagram illustrating a computing device (100-6), a server (200-6), and an edge device according to one embodiment of the present invention.

[0275] Figure 54 is a schematic diagram showing one or more network functions that constitute an artificial intelligence model according to one embodiment of the present invention.

[0276] Figure 55 is a flowchart illustrating a method for providing personalized artificial intelligence according to one embodiment of the present invention.

[0277] Figure 56a is a diagram for explaining the creation of an artificial intelligence platform according to one embodiment of the present invention.

[0278] FIG. 56b is a diagram illustrating the creation of a personalized artificial intelligence model according to one embodiment of the present invention.

[0279] Figure 56c is a diagram for explaining an update of an artificial intelligence platform according to one embodiment of the present invention.

[0280] FIG. 56d is a diagram illustrating the use and additional learning of personalized artificial intelligence according to one embodiment of the present invention.

[0281] FIG. 57 is a diagram illustrating a personalized artificial intelligence system for indoor safety management in a home according to one embodiment of the present invention.

[0282] FIG. 58 is a diagram illustrating a personalized artificial intelligence system for outdoor safety management at home according to one embodiment of the present invention.

[0283] FIG. 59 is a diagram illustrating a personalized artificial intelligence system for allergy management according to one embodiment of the present invention.

[0284] Figure 60 is a diagram showing a field facility control infrastructure utilizing edge AI according to the present invention.

[0285] FIG. 61a is a diagram illustrating that an edge AI infrastructure according to the present invention includes at least one edge AI system.

[0286] FIG. 61b is a diagram illustrating that an edge AI system according to the present invention includes a head device and at least one node device.

[0287] FIG. 61c is a drawing for explaining how a node device according to the present invention receives location information from a user terminal.

[0288] FIG. 61d is a drawing for explaining that a head device according to the present invention includes a network section, a memory section, and a processor section.

[0289] FIG. 61e is a diagram for explaining that a head device according to the present invention receives location information from at least one node device, identifies personal information through an integrated control server, and generates facility control information based on at least one of the location information and personal information and transmits the same to the integrated control server.

[0290] Figure 62a is a diagram illustrating an integrated control server according to the present invention receiving personal information from a user terminal, and an edge AI system identifying the personal information through the integrated control server.

[0291] FIG. 62b is a diagram for explaining that an integrated control server according to the present invention receives facility control information from an edge AI system, transmits user terminal control information to a user terminal based on the facility control information, and controls a field facility based on the facility control information.

[0292] FIG. 63a is a drawing for explaining that a head device according to the present invention tracks a user's location in real time.

[0293] FIG. 63b is a drawing for explaining that a head device according to the present invention generates spatial residence time information based on location information.

[0294] FIG. 63c is a drawing for explaining that a head device according to the present invention generates facility control information based on at least one of location information, personal information, and space-specific residence time information, and transmits the same to an integrated control server.

[0295] FIGS. 64a to 64f are drawings for explaining how the field facility control infrastructure according to the present invention controls the entrance / exit doors of an exhibition hall and / or user terminals of users using the exhibition hall.

[0296] FIGS. 656a to 656b are drawings illustrating how the field facility control infrastructure according to the present invention controls a user terminal to display a user interface that displays information about a booth of interest.

[0297] FIG. 66 is a drawing illustrating a field facility control infrastructure according to the present invention controlling a user terminal to display a user interface that displays information about a related booth.

[0298] FIGS. 67a and 67b are diagrams illustrating how the field facility control infrastructure according to the present invention controls a user terminal to display a user interface that displays a list of users of interest.

[0299] FIG. 68 is a drawing illustrating a field facility control infrastructure according to the present invention controlling a user terminal to display a user interface that displays a visitor list.

[0300] FIG. 69 is a drawing illustrating a field facility control infrastructure according to the present invention controlling a user terminal to display a user interface that displays a list of the next viewing booths for the visitor.

[0301] FIGS. 70 to 72 are drawings for explaining embodiments of a field facility control method utilizing at least one edge AI system according to the present invention.

[0302] FIG. 73 is a diagram illustrating an embodiment of communication between an edge AI device and a plurality of edge AI devices according to one embodiment of the present invention.

[0303] Figure 74 is a drawing showing an embodiment for tracking an object by an edge AI device within a large complex facility according to the present invention.

[0304] FIG. 75 is a diagram illustrating an example of tracking the movement path of an object within a large complex facility according to one embodiment of the present invention.

[0305] Figure 76 is a drawing showing an example of tracking the movement path of an object in a store according to one embodiment of the present invention.

[0306] FIG. 77 is a diagram illustrating an embodiment of communication between an edge AI device and a plurality of edge AI devices according to one embodiment of the present invention.

[0307] FIG. 78 is a diagram illustrating communication and object tracking between multiple edge AI devices according to one embodiment of the present invention.

[0308] FIG. 79 is a drawing for explaining an embodiment in which the processor unit according to the present invention cannot match an object sensed by the sensor unit with unique information stored in the memory unit.

[0309] FIG. 80 is a drawing for explaining an embodiment in which an access authorization authentication device according to the present invention communicates with an edge AI device.

[0310] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. In describing the embodiments, descriptions of technical details that are well-known in the technical field to which the present invention pertains and are not directly related to the present invention will be omitted. This is to avoid obscuring the essence of the present invention by omitting unnecessary explanations and to convey it more clearly.

[0311] For the same reason, some components in the attached drawings are exaggerated, omitted, or schematically depicted. Furthermore, the dimensions of each component do not entirely reflect its actual size. Identical or corresponding components in each drawing are assigned the same reference numbers.

[0312] The advantages and features of the present invention, and the methods for achieving them, will become clearer with reference to the embodiments described in detail below together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided only to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims. Like reference numerals designate like elements throughout the specification.

[0313] Figure 1a is a block diagram illustrating a driver assistance system using edge AI according to the present invention.

[0314] Overall composition

[0315] The entire configuration of the electronic device (10) that constitutes the infrastructure of multiple sensor devices

[0316] As illustrated in FIG. 1a, a driver assistance system using edge AI according to the present invention may include an electronic device (10) that constitutes a plurality of sensor device infrastructures installed on a road, an AR device (20) that provides AR worn by a vehicle driver, and an external terminal (100).

[0317] In this specification, the driver assistance system using edge AI is described as an AR device (20) that provides AR worn by a vehicle driver. However, it includes not only an AR device (20) that provides AR worn by a vehicle driver, but also all embodiments in which drivers of transportation such as airplanes, helicopters, bicycles, and motorcycles wear AR devices, and is not limited thereto.

[0318] Sensor device (13)

[0319] According to one embodiment of the present invention, a driver assistance system using edge AI may include an electronic device (10) constituting a plurality of sensor device infrastructures installed on a road. The electronic device (10) constituting a plurality of sensor device infrastructures installed on a road may include a processor (11), a memory (12), and a sensor device (13). Specifically, the sensor device (13) may be a camera, and is a device capable of capturing still images and moving images, and may include one or more image sensors, a lens, an image signal processor (ISP), or a flash (e.g., an LED or a xenon lamp, etc.).

[0320] According to one embodiment of the present invention, the sensor device (13) can be installed on a smart pole and capture various traffic images occurring on the road. For example, in the case where a vehicle (30) of a driver wearing an AR device, which is one of the embodiments described below, enters a blind spot (31) where the driver's field of vision is blocked and cannot see, the sensor device (13) can sense that a pedestrian is located in the blind spot (31), and a camera, which is one embodiment of the sensor device (13), can capture the pedestrian located in the blind spot (31). In another example, in the case where a traffic congestion occurs due to an accident or the like on the road on which the vehicle (30) of a driver wearing an AR device, which is one of the embodiments described below, is located during the route from the driver's destination to the road on which the driver's destination is located, the sensor device (13) can sense the point where the accident occurred on the road on which the driver's destination is located, and a camera, which is one embodiment of the sensor device (13), can capture the point where the accident occurred.

[0321] Processor (11)

[0322] According to one embodiment of the present invention, a driver assistance system using edge AI may include an electronic device (10) that constitutes a plurality of sensor device infrastructures installed on a road, and a processor (11). Specifically, the processor (11) may execute a program and control the electronic device (10) that constitutes a plurality of sensor device infrastructures installed on a road. The code of the program executed by the processor (11) may be stored in a memory (12). In addition, the processor (11) may be connected to an external device through an input / output device and exchange data. The external device may be an electronic device that constitutes a plurality of sensor device infrastructures installed on another road, an external terminal (100) such as a server, or an AR device (20) that provides AR worn by a vehicle driver.

[0323] According to one embodiment of the present invention, a processor (11) included in an electronic device (10) constituting a plurality of sensor device infrastructures installed on a road can directly process an image processing-based artificial intelligence model, and can also independently process an image processing-based artificial intelligence model. Various models, such as a Recurrent Neural Network (RNN), a Deep Neural Network (DNN), and a Dynamic Recurrent Neural Network (DRNN), can be utilized as artificial intelligence network models for such learning. In this way, the processor (11) can process and analyze data on the device itself without sending the data to a central data center or the cloud. Therefore, the electronic device (10) constituting a plurality of sensor device infrastructures installed on a road can be highly efficient in data processing and be very useful in situations requiring real-time responses.

[0324] Memory (12)

[0325] According to one embodiment of the present invention, a driver assistance system using edge AI may include an electronic device (10) that constitutes a plurality of sensor device infrastructures installed on a road, and a memory (12). Specifically, when a sensor device (13) mounted on an electronic device (10) that constitutes a plurality of sensor device infrastructures installed on a road captures an image of the road, the memory (12) may store the image data and store intermediate results or temporary data generated while the processor (11) processes the data.

[0326] According to one embodiment of the present invention, the memory (12) can be used to record information related to detected events, accidents, traffic flow, etc., and the processor (11) can use the memory (12) to remember and learn patterns of accidents occurring on the road.

[0327] According to one embodiment of the present invention, the memory (12) may be a volatile memory or a non-volatile memory, and may be referred to as a 'database', a 'storage', or the like.

[0328] AR device

[0329] As illustrated in FIG. 1a, an AR device (20) capable of communicating with an electronic device (10) constituting the aforementioned plurality of sensor device infrastructures may include a display unit (21), an AR device camera unit (22), and an AR device processor unit (23).

[0330] According to one embodiment of the present invention, the AR device (20) may be in a form that the driver can wear on the head, forehead, ear, etc., but is not limited thereto.

[0331] In this specification, the driver assistance system using edge AI is described as an AR device (20) that provides AR worn by a vehicle driver. However, it includes not only an AR device (20) that provides AR worn by a vehicle driver, but also all embodiments in which drivers of transportation such as airplanes, helicopters, bicycles, and motorcycles wear AR devices, and is not limited thereto.

[0332] According to one embodiment of the present invention, an electronic device (10) constituting a plurality of sensor device infrastructures can communicate with an AR device (20) through a communication protocol.

[0333] According to one embodiment of the present invention, the AR device (20) may include, but is not limited to, smart glasses that can be worn by a driver, a head-up display (HUD), an AR safety warning system, a smart helmet, a smart cap, a smart wearable device, an in-vehicle console display with AR function, etc.

[0334] The attached drawings of this specification describe smart glasses as one embodiment of AR devices (20), but are not limited thereto, and all devices that can communicate with an external device to provide information to a vehicle driver using AR will be included.

[0335] According to one embodiment of the present invention, an AR device (20) worn by a vehicle driver may be equipped with an augmented reality function. This serves to visually convey information to the driver by projecting necessary information while driving. Furthermore, the present invention is not limited thereto. While the accompanying drawings of the present invention illustrate the provision of augmented reality, it is also possible to provide images of virtual reality, in addition to augmented reality, to project necessary information while driving and visually convey information to the driver.

[0336] According to one embodiment of the present invention, the processor unit (23) of the AR device (20) worn by the vehicle driver can recognize the voice of the vehicle driver, so that the AR device (20) and the vehicle driver can communicate with each other by voice.

[0337] Display section (21)

[0338] According to the present invention, the display unit (21) of the AR device (20) can project information into the driver's field of vision, allowing the driver to check necessary information while keeping their eyes on the road. Specifically, the display unit (21) can project real-time road maps and navigation information into the driver's field of vision. Information on road conditions, vehicle locations, surrounding environments, etc. can be received through communication with the electronic device (10) constituting the multiple sensor device infrastructure based on data collected and processed by the electronic device (10) constituting the multiple sensor device infrastructure, and provided to the driver's field of vision.

[0339] According to one embodiment of the present invention, the display unit (21) can output data related to road conditions as an augmented reality image, but it is also possible to output it as a virtual reality image as well as an augmented reality image.

[0340] According to one embodiment of the present invention, data related to road conditions output by the display unit (21) may include road conditions such as road traffic accidents, road congestion, road construction, road facility failure, road regulations, illegal parking on the road, and violation of road traffic rules.

[0341] According to one embodiment of the present invention, the display unit (21) can receive and communicate with the electronic device (10) constituting the plurality of sensor device infrastructures based on data collected and processed by the electronic device (10) constituting the plurality of sensor device infrastructures, and project real-time traffic information on the road, distance warnings from the vehicle ahead, whether a signal has been violated, traffic signal status, information on nearby stores or tourist attractions, major buildings, etc., and monitoring of the driver's status using the facial recognition technology of the AR device (20), etc., into the driver's field of vision. Accordingly, the driver can detect the current road conditions or the driver's fatigue or drowsiness status, etc. from the display unit (21), and the AR device (20) can not only provide information visually, but can also provide information audibly by enabling voice output, etc.

[0342] According to one embodiment of the present invention, a processor (11) included in an electronic device (10) constituting a plurality of sensor device infrastructures installed on a road directly processes an image processing-based artificial intelligence model, inputs an image of a road captured by a sensor device (13) into the image processing-based artificial intelligence model, and performs a situation judgment operation regarding the road. At this time, the AR device processor unit (23) of the AR device (20) can receive data regarding the situation judgment performed by the processor (11) through a communication protocol, and the display unit (21) of the AR device (20) can output the received data, thereby providing information regarding the road situation to the driver.

[0343] As illustrated, the display unit (21) can output data related to securing a safe distance from other vehicles located below a critical distance while driving as an augmented reality image.

[0344] As illustrated, the display unit (21) can output data related to the shortest distance parking space between the driver's destination and the parking location as an augmented reality image.

[0345] AR device camera section (22)

[0346] According to one embodiment of the present invention, the AR device camera unit (22) of the AR device (20) has a wide field of view and can effectively capture the surrounding environment of the vehicle while the driver is driving. Since the AR device processor unit (23) of the AR device (20), which will be described later, is equipped with artificial intelligence based on image processing, it is also possible for the AR device processor unit (23) to process the road conditions captured by the AR device camera unit (22).

[0347] According to one embodiment of the present invention, the AR device (20) can receive a manual, such as a method for starting the engine of the corresponding vehicle, from an external terminal (100), and accordingly, the AR device processor unit (23) can identify the face of the vehicle driver through the AR device camera unit (22) and check whether it matches the face of the vehicle driver registered in advance. In addition, the AR device processor unit (23) can detect the gesture of the vehicle driver through the AR device camera unit (22) and interpret a specific movement as engine start control. In addition, the AR device processor unit (23) can detect the driver's gaze by utilizing the driver's eye tracking technology through the AR device camera unit (22), and can recognize whether the driver is in a state of drowsiness or accumulated fatigue by detecting a specific gaze or a specific pattern.

[0348] AR device processor (23)

[0349] According to one embodiment of the present invention, the AR device (20) may include an AR device processor unit (23), and the AR device processor unit (23) may communicate with an external terminal (100), an electronic device (10) constituting a plurality of sensor device infrastructures, and a driver's vehicle (30). The AR device processor unit (23) may directly process an image processing-based artificial intelligence model, and may also independently process an image processing-based artificial intelligence model. Various models, such as a Recurrent Neural Network (RNN), a Deep Neural Network (DNN), and a Dynamic Recurrent Neural Network (DRNN), may be utilized as artificial intelligence network models for such learning. In this way, the AR device processor unit (23) may process and analyze data within the device itself without sending the data to a central data center or the cloud. Therefore, the electronic device (10) constituting a plurality of sensor device infrastructures installed on the road may be highly efficient in data processing and may be very useful in situations requiring real-time responses.

[0350] According to one embodiment of the present invention, a processor (11) included in an electronic device (10) constituting a plurality of sensor device infrastructures installed on a road directly processes an image processing-based artificial intelligence model, inputs an image of a road captured by a sensor device (13) into the image processing-based artificial intelligence model, and performs a situation judgment operation regarding the road. At this time, the AR device processor unit (23) of the AR device (20) can receive data regarding the situation judgment performed by the processor (11) through a communication protocol, and the display unit (21) of the AR device (20) can output the received data, thereby providing information regarding the road situation to the driver.

[0351] According to another embodiment of the present invention, the AR device processor unit (23) of the AR device (20) can directly process an artificial intelligence model based on image processing. At this time, the AR device processor unit (23) can receive an image of the road captured by the sensor device (13) through a communication protocol, and the AR device processor unit (23) can input the received image of the road into the artificial intelligence model based on image processing to perform a situation judgment operation regarding the road. At this time, the display unit (21) of the AR device (20) can output data regarding the situation judgment performed by the AR device processor unit (23) to provide the driver with information regarding the road situation.

[0352] An embodiment of a specific AR device processor unit (23) will be described below.

[0353] External terminal (100)

[0354] Figure 1c is a block diagram for explaining an external terminal (100) according to one embodiment of the present invention.

[0355] As illustrated in FIG. 1C, the external terminal (100) may include an external terminal processor (110), an external terminal memory (120), and a communication module (170). The external terminal (100) may be an external server or a cloud server. The external server may be a digital device equipped with a processor, memory, and computing power, such as a laptop computer, notebook computer, desktop computer, web pad, or mobile phone. The external server may be a web server that processes services. The types of servers described above are merely examples, and the present invention is not limited thereto.

[0356] According to one embodiment of the present invention, an electronic device (10) constituting a plurality of sensor device infrastructures can communicate with an AR device (20) via a communication protocol. In addition, an electronic device (10) constituting a plurality of sensor device infrastructures can communicate with an external terminal (100) via a communication protocol. In addition, an AR device (20) and an external terminal (100) can communicate with each other via a communication protocol. That is, an electronic device (10), an AR device (20), and an external terminal (100) constituting a plurality of sensor device infrastructures can communicate with each other via a communication protocol.

[0357] According to one embodiment of the present invention, the external terminal (100) may be a server providing a cloud computing service. More specifically, the external terminal (100) may be a server providing a cloud computing service, a type of Internet-based computing that processes information using another computer connected to the Internet rather than the user's computer. The cloud computing service may be a service that stores data on the Internet and allows users to access it anytime and anywhere via an Internet connection without having to install necessary data or programs on their own computers. Furthermore, data stored on the Internet can be easily shared and transferred with simple operations and clicks. Furthermore, the cloud computing service may be a service that not only stores data on an Internet server, but also allows users to perform desired tasks using the functions of web-based application programs without having to install separate programs. Furthermore, the cloud computing service may be a service that allows multiple people to simultaneously share and work on documents. Furthermore, the cloud computing service may be implemented in at least one of the following forms: Infrastructure as a Service (IaaS), Platform as a Service (PaaS), Software as a Service (SaaS), a virtual machine-based cloud server, and a container-based cloud server. In other words, the external terminal (100) of the present invention may be implemented in at least one of the aforementioned cloud computing services. The specific description of the cloud computing service described above is merely an example, and may include any platform that constructs the cloud computing environment of the present invention.

[0358] According to one embodiment of the present invention, the external terminal processor (110) can control the external terminal (100) in general, and the external terminal processor (110) can include an AI processor (115). Specifically, the external terminal processor (110) can process general-purpose artificial intelligence. Specifically, the AI ​​processor (115) can learn a neural network using a program stored in the external terminal memory (120). In particular, the AI ​​processor (115) can learn a neural network for recognizing data related to the operations of an electronic device (10) and an AR device (20) constituting a plurality of sensor device infrastructures installed on a road. Here, the neural network can be designed to simulate the human brain structure (e.g., the neuron structure of a human neural network) on a computer. The neural network can include an input layer, an output layer, and at least one hidden layer. Each layer includes at least one neuron with weights, and the neural network may include synapses connecting neurons. In the neural network, each neuron can output an input signal received through a synapse as a function value of an activation function with respect to the weights and / or biases.

[0359] According to one embodiment of the present invention, a plurality of network modes can exchange data according to their connection relationships, respectively, so as to simulate the synaptic activity of neurons that exchange signals through synapses. Here, the neural network may include a deep learning model developed from a neural network model. In the deep learning model, a plurality of network nodes may be located in different layers and exchange data according to convolutional connection relationships. Examples of the neural network model include various deep learning techniques such as a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network, a restricted Boltzmann machine, a deep belief network, and a deep Q-network, and can be applied in fields such as vision recognition, speech recognition, natural language processing, and speech / signal processing.

[0360] Meanwhile, the external terminal processor (110) performing the function described above may be a general-purpose processor (e.g., CPU), but may be an AI-only processor for artificial intelligence learning (e.g., GPU, TPU).

[0361] According to one embodiment of the present invention, the external terminal memory (120) can store various programs and data required for the operation of the electronic device (10) and AR device (20) constituting the infrastructure of multiple sensor devices installed on the road. The external terminal memory (120) is accessed by the AI ​​processor (115), and data reading / recording / modifying / deleting / updating, etc. can be performed by the AI ​​processor (115). In addition, the external terminal memory (120) can store a neural network model (e.g., a deep learning model) generated through a learning algorithm for data classification / recognition. Furthermore, the external terminal memory (120) can store not only the learning model (121), but also input data, learning data, learning history, etc.

[0362] According to one embodiment of the present invention, the communication module (170) can transmit the AI ​​processing result by the AI ​​processor (115) to an electronic device (10) or AR device (20) that constitutes a plurality of sensor device infrastructures installed on the road.

[0363] According to another embodiment of the present invention, the external terminal processor (110) of the external terminal (100) can directly process an artificial intelligence model based on image processing. At this time, the external terminal processor (110) can receive an image of the road captured by the sensor device (13) through a communication protocol, and the external terminal processor (110) can input the received image of the road into the artificial intelligence model based on image processing to perform a situation judgment operation regarding the road. At this time, the AR device processor unit (23) can receive data regarding the situation judgment performed by the external terminal processor (110) through a communication protocol. The display unit (21) can output the data regarding the received situation judgment to provide the driver with information regarding the road situation.

[0364] Communication between electronic devices that constitute the infrastructure of AR devices, external terminals, or multiple sensor devices installed on the road

[0365] Embodiments of communication between multiple devices

[0366] FIG. 1b is a diagram for explaining a communication network between an external terminal (100), an AR device (20), and an electronic device (10) constituting an infrastructure of multiple sensor devices installed on a road according to one embodiment of the present invention.

[0367] According to one embodiment of the present invention, an electronic device (10) constituting a plurality of sensor device infrastructures can communicate with an AR device (20) via a communication protocol. In addition, an electronic device (10) constituting a plurality of sensor device infrastructures can communicate with an external terminal (100) via a communication protocol. In addition, an AR device (20) and an external terminal (100) can communicate with each other via a communication protocol. That is, an electronic device (10), an AR device (20), and an external terminal (100) constituting a plurality of sensor device infrastructures can communicate with each other via a communication protocol.

[0368] As illustrated in FIG. 1B, an external terminal (100) can communicate with a plurality of AR devices (20), and the plurality of AR devices (20) can communicate with electronic devices (10) constituting a plurality of sensor device infrastructures installed on a plurality of roads, and the electronic devices (10) constituting a plurality of sensor device infrastructures installed on a plurality of roads can communicate with the external terminal (100). Specifically, the electronic devices (10) constituting a plurality of sensor device infrastructures installed on a plurality of roads can capture and analyze image data generated on the roads in real time, and the image processing-based artificial intelligence installed in the processor (11) of the electronic devices (10) constituting a plurality of sensor device infrastructures installed on the roads can process the data and extract important information. The extracted information can indicate local events, traffic conditions, etc., and can transmit such information to the AR device (20) or the external terminal (100) in real time.

[0369] According to one embodiment of the present invention, an AR device (20) can receive information about road conditions from an electronic device (10) constituting a plurality of sensor device infrastructures installed on a road and display the information to a vehicle driver. The AR device (20) can locally process some of the information received from the electronic device (10) constituting a plurality of sensor device infrastructures installed on a road, and can also transmit the information to an external terminal (100) as needed.

[0370] According to one embodiment of the present invention, an external terminal (100) can centrally process data collected from an AR device (20) or an electronic device (10) constituting a plurality of sensor device infrastructures installed on a road, and perform high-level analysis and judgment. The external terminal (100) can integrate information collected from the AR device (20) or the electronic device (10) constituting a plurality of sensor device infrastructures installed on a road to understand a broader range of road conditions, and the external terminal (100) can transmit necessary additional information to the AR device (20) or the electronic device (10) constituting a plurality of sensor device infrastructures installed on a road, and can analyze data collected from the AR device (20) or the electronic device (10) constituting a plurality of sensor device infrastructures installed on a road to generate comprehensive insight into road conditions. This can be transmitted back to the AR device (20), and a vehicle driver can visually or audibly recognize data based on judgments about road conditions, etc.

[0371] communication protocol

[0372] According to one embodiment of the present invention, an external terminal (100) can communicate with a plurality of AR devices (20), and the plurality of AR devices (20) can communicate with electronic devices (10) constituting a plurality of sensor device infrastructures installed on a plurality of roads.

[0373] According to one embodiment of the present invention, an electronic device (10) constituting a plurality of sensor device infrastructures can communicate with an AR device (20) via a communication protocol. In addition, an electronic device (10) constituting a plurality of sensor device infrastructures can communicate with an external terminal (100) via a communication protocol. In addition, an AR device (20) and an external terminal (100) can communicate with each other via a communication protocol. That is, an electronic device (10), an AR device (20), and an external terminal (100) constituting a plurality of sensor device infrastructures can communicate with each other via a communication protocol.

[0374] In this case, various communication protocols can be used. At this time, the selected protocol may vary depending on specific requirements, bandwidth, security level, etc., and the following protocols may be combined and used. For specific examples, lightweight protocols such as MQTT (Message Queuing Telemetry transport) and CoAP (Constrained Application Protocol), which are optimized for the Internet of Things because they require few resources, can be used; HTTP (HyperTexT Protocol) for fast hypertext exchange; HTTPS (HyperTexT Protocol Secure) for providing a secure connection and encrypting data; AMQP (Advanced Message Queuing Protocol) for exchanging and communicating messages; DDS (Data Distribution Service) for exchanging data between real-time systems, etc. can be used; in addition, FTP (File Transfer Protocol), RTSP (Real Time Streaming Protocol), RTP (Real-time Transport Protocol), etc. can be used, but are not limited thereto.

[0375] According to another embodiment of the present invention, an external terminal (100) can communicate with a plurality of AR devices (20), and the plurality of AR devices (20) can communicate with electronic devices (10) constituting a plurality of sensor device infrastructures installed on a plurality of roads. In this case, various communication protocols can be used. At this time, a wireless communication technology having an extremely wide bandwidth, such as UWB (Ultra-Wide Band), can be used, and a wireless communication protocol based on a method of transmitting data using a very wide frequency bandwidth can be used. This enables accurate search in a wide area based on very precise spatial recognition and directionality, but is not limited thereto.

[0376] According to another embodiment of the present invention, the external terminal processor (110) of the external terminal (100) can process a general-purpose artificial intelligence model. At this time, the external terminal processor (110) can receive an image of the road captured by the sensor device (13) through a communication protocol, and the external terminal processor (110) can input the received image of the road into the general-purpose artificial intelligence model to perform a situation judgment operation regarding the road. At this time, the AR device processor unit (23) can receive data regarding the situation judgment performed by the external terminal processor (110) through a communication protocol. The display unit (21) can output the data regarding the received situation judgment to provide the driver with information regarding the road situation.

[0377] According to another embodiment of the present invention, the external terminal processor (110) of the external terminal (100) can process a general-purpose artificial intelligence model. In addition, the AR device processor unit (23) can directly process an artificial intelligence model based on image processing. In addition, the processor (11) can directly process an artificial intelligence model based on image processing. At this time, the processor (11) can perform a situation judgment operation on part or all of the road using the image of the road captured by the sensor device (13) as input to the artificial intelligence model based on image processing. At this time, the external terminal processor (110) can receive at least one or more of the situation judgment on part or all of the road performed by the processor (11) and the image of the road captured by the sensor device (13) through a communication protocol, and can perform a situation judgment operation on part or all of the road using at least one or more of the situation judgment on part or all of the road performed by the received processor (11) and the image of the road captured by the sensor device (13) as input to the general-purpose artificial intelligence model. At this time, the AR device (20) directly processes an artificial intelligence model based on image processing, and receives at least one of a part or all of the situation judgment regarding the road performed by the processor (11), an image of the road captured by the sensor device (13), and a part or all of the situation judgment performed by the external terminal processor (110), and performs a part or all of the situation judgment operation regarding the road based on this.

[0378] According to one embodiment of the present invention, the display unit (21) can output data related to road conditions as an augmented reality image, but it is also possible to output it as a virtual reality image as well as an augmented reality image.

[0379] According to one embodiment of the present invention, data related to road conditions output by the display unit (21) may include road conditions such as road traffic accidents, road congestion, road construction, road facility failure, road regulations, illegal parking on the road, and violation of road traffic rules.

[0380] As illustrated, the display unit (21) can output data related to the location of a pedestrian or object in the blind spot (31) of a moving vehicle as an augmented reality image.

[0381] As shown, the display unit (21) can output data related to road lanes or road surface guide signs as an augmented reality image.

[0382] Example of Fig. 2

[0383] FIG. 2 is a drawing for explaining an embodiment in which an AR device (20) receives data from an electronic device (10) that constitutes a plurality of sensor device infrastructures installed on a road according to the present invention.

[0384] In this specification, the driver assistance system using edge AI is described as an AR device (20) that provides AR worn by a vehicle driver. However, it includes not only an AR device (20) that provides AR worn by a vehicle driver, but also all embodiments in which drivers of transportation such as airplanes, helicopters, bicycles, and motorcycles wear AR devices, and is not limited thereto.

[0385] As illustrated, according to one embodiment of the present invention, a vehicle driver can wear an AR device (20) while driving a vehicle (30) of a driver wearing an AR device. As described above, the attached drawing describes an embodiment in which an AR device (20) in the form of a 'smart glass' is worn, but the AR device (20) is not limited to the form of a 'smart glass'.

[0386] According to one embodiment of the present invention, an electronic device (10) constituting a plurality of sensor device infrastructures can communicate with an AR device (20) via a communication protocol. In addition, an electronic device (10) constituting a plurality of sensor device infrastructures can communicate with an external terminal (100) via a communication protocol. In addition, an AR device (20) and an external terminal (100) can communicate with each other via a communication protocol. That is, an electronic device (10), an AR device (20), and an external terminal (100) constituting a plurality of sensor device infrastructures can communicate with each other via a communication protocol.

[0387] According to another embodiment of the present invention, the external terminal processor (110) of the external terminal (100) can directly process an artificial intelligence model based on image processing. In addition, the processor (11) of the electronic device (10) constituting the infrastructure of multiple sensor devices can also directly process an artificial intelligence model based on image processing. The sensor device (13) captures an image of a road, and the processor (11) directly processes an artificial intelligence model based on image processing to perform part or all of the road situation judgment operations. At this time, the external terminal processor (110) can receive the image of the road captured by the sensor device (13) and the data regarding the road situation judgment performed by the processor (11) through a communication protocol, and the external terminal processor (110) inputs the received image of the road and the data regarding the road situation judgment performed by the processor (11) into the image processing-based artificial intelligence model, thereby performing part or all of the road situation judgment operations. At this time, the AR device processor unit (23) can receive data regarding situation judgment performed by the external terminal processor (110) and data regarding situation judgment performed by the processor (11) via a communication protocol. The display unit (21) can output the received data regarding situation judgment, thereby providing the driver with information regarding road conditions.

[0388] According to another embodiment of the present invention, the external terminal processor (110) of the external terminal (100) can process a general-purpose artificial intelligence model. In addition, the AR device processor unit (23) can directly process an artificial intelligence model based on image processing. At this time, the external terminal processor (110) and the AR device processor unit (23) can receive an image of a road captured by the sensor device (13) through a communication protocol, and the external terminal processor (110) can input the received image of the road into an artificial intelligence model based on image processing, thereby performing part or all of a situation judgment operation regarding the road. At this time, the AR device processor unit (23) can receive data regarding the situation judgment performed by the external terminal processor (110) and the image of the road captured by the sensor device (13) through a communication protocol. The AR device processor unit (23) can input the received data regarding the situation judgment performed by the external terminal processor (110) and the image of the road captured by the sensor device (13) into an artificial intelligence model based on image processing, thereby performing part or all of a situation judgment operation regarding the road. Accordingly, the display unit (21) can output data regarding the situation judgment performed, thereby providing the driver with information regarding the road situation.

[0389] According to another embodiment of the present invention, the external terminal processor (110) of the external terminal (100) can process a general-purpose artificial intelligence model. At this time, the external terminal processor (110) can receive an image of the road captured by the sensor device (13) through a communication protocol, and the external terminal processor (110) can input the received image of the road into the general-purpose artificial intelligence model to perform a situation judgment operation regarding the road. At this time, the AR device processor unit (23) can receive data regarding the situation judgment performed by the external terminal processor (110) through a communication protocol. The display unit (21) can output the data regarding the received situation judgment to provide the driver with information regarding the road situation.

[0390] According to another embodiment of the present invention, the external terminal processor (110) of the external terminal (100) can process a general-purpose artificial intelligence model. In addition, the AR device processor unit (23) can directly process an artificial intelligence model based on image processing. In addition, the processor (11) can directly process an artificial intelligence model based on image processing. At this time, the processor (11) can perform a situation judgment operation on part or all of the road using the image of the road captured by the sensor device (13) as input to the artificial intelligence model based on image processing. At this time, the external terminal processor (110) can receive at least one or more of the situation judgment on part or all of the road performed by the processor (11) and the image of the road captured by the sensor device (13) through a communication protocol, and can perform a situation judgment operation on part or all of the road using at least one or more of the situation judgment on part or all of the road performed by the received processor (11) and the image of the road captured by the sensor device (13) as input to the general-purpose artificial intelligence model. At this time, the AR device (20) directly processes an artificial intelligence model based on image processing, and receives at least one of a part or all of the situation judgment regarding the road performed by the processor (11), an image of the road captured by the sensor device (13), and a part or all of the situation judgment performed by the external terminal processor (110), and performs a part or all of the situation judgment operation regarding the road based on this.

[0391] According to another embodiment of the present invention, the external terminal processor (110) of the external terminal (100) can process a general-purpose artificial intelligence model. In addition, the AR device processor unit (23) can directly process an artificial intelligence model based on image processing. At this time, the external terminal processor (110) can communicate an image of the road captured by the sensor device (13) through a communication protocol and use this as input for the general-purpose artificial intelligence model to perform a partial or full situation judgment operation regarding the road. The AR device (20) can communicate an image of the road captured by the sensor device (13) and data regarding a partial or full situation judgment regarding the road performed by the external terminal processor (110) through a communication protocol and use this as input for the image processing-based artificial intelligence model to perform a partial or full situation judgment operation regarding the road.

[0392] As illustrated, the electronic device (10) constituting the infrastructure of multiple sensor devices installed on the road can be installed in a form mounted on a smart pole, and the electronic device (10) constituting the infrastructure of multiple sensor devices installed on the road can capture images of road conditions, etc., process the data, and in real time, communication (101) between the electronic device and the AR device can be performed with the AR device (20) worn by the driver of the vehicle. Specifically, the communication (101) between the electronic device and the AR device will be mainly performed in a form in which the data processed by the electronic device (10) constituting the infrastructure of multiple sensor devices installed on the road is transmitted to the AR device (20) using a communication protocol. However, an embodiment in which the AR device (20) analyzes the state of the vehicle (30) of the driver currently wearing the AR device or the state of the vehicle driver, etc., and the electronic device (10) constituting the infrastructure of multiple sensor devices installed on the road receives the analyzed data is also possible. In addition, it is also possible to assist the driver in driving safely by having the AR device processor (23) of the AR device (20) worn by the driver analyze the current status of the vehicle (30) of the driver wearing the AR device or the status of the driver and directly transmit the information to the driver.

[0393] For example, if the AR device processor unit (23) detects the drowsy driving of a vehicle driver, data indicating that drowsy driving has been detected can be transmitted to an electronic device (10) constituting a plurality of sensor device infrastructures installed on the road through communication (101) between the electronic device and the AR device, and the electronic device (10) constituting a plurality of sensor device infrastructures installed on the road can receive the data and transmit the data to another electronic device (10-1) or an external terminal (100) shown in FIG. 1b, thereby transmitting information to other vehicles (30-2, 30-3, etc.) of drivers wearing AR devices to be careful of the vehicle (30) of the driver wearing the AR device. In addition, it is also possible for the AR device processor unit (23) of the AR device (20) worn by the vehicle driver to detect drowsy driving of the vehicle (30) of the driver currently wearing the AR device, analyze the data, and directly transmit information about drowsy driving to the vehicle driver, thereby assisting the safe driving of the vehicle driver. Accordingly, the electronic device (10) that constitutes the infrastructure of multiple sensor devices installed on the road can directly receive the data from the AR device (20) and transmit information to other vehicles (30-2, 30-3, etc.) of drivers wearing AR devices to be careful of the vehicle (30) of the driver wearing the AR device.

[0394] Embodiment of Fig. 3

[0395] FIG. 3 is a drawing for explaining an embodiment in which, when there is a pedestrian located in a blind spot (31) of a vehicle (30) of a driver wearing an AR device according to one embodiment of the present invention, an AR device (20) receives data from an electronic device (10) constituting a plurality of sensor device infrastructures indicating that a pedestrian is located in a blind spot (31).

[0396] In this specification, the driver assistance system using edge AI is described as an AR device (20) that provides AR worn by a vehicle driver. However, it includes not only an AR device (20) that provides AR worn by a vehicle driver, but also all embodiments in which drivers of transportation such as airplanes, helicopters, bicycles, and motorcycles wear AR devices, and is not limited thereto.

[0397] According to one embodiment of the present invention, an electronic device (10) constituting a plurality of sensor device infrastructures can communicate with an AR device (20) via a communication protocol. In addition, an electronic device (10) constituting a plurality of sensor device infrastructures can communicate with an external terminal (100) via a communication protocol. In addition, an AR device (20) and an external terminal (100) can communicate with each other via a communication protocol. That is, an electronic device (10), an AR device (20), and an external terminal (100) constituting a plurality of sensor device infrastructures can communicate with each other via a communication protocol.

[0398] As illustrated, the display unit (21) can output data related to the location of a pedestrian or object in the blind spot (31) of a moving vehicle as an augmented reality image.

[0399] As illustrated, a vehicle (30) of a driver wearing an AR device may have a blind spot (31). A blind spot (31) of a vehicle may refer to an area where the driver has difficulty directly checking the view from inside the vehicle, and may primarily correspond to the side, rear, and rear sides of the vehicle. If an object or pedestrian suddenly appears in a blind spot (31) while the driver is driving, it may affect traffic safety, so a system that detects and warns the driver of a blind spot (31) may be required.

[0400] As illustrated, the electronic device (10) constituting the infrastructure of multiple sensor devices installed on the road can be installed in a form mounted on a smart pole, and the electronic device (10) constituting the infrastructure of multiple sensor devices installed on the road can capture a pedestrian or an object in the blind spot (31) of the driver while driving and process the corresponding data. Accordingly, communication (101) between the electronic device and the AR device can be performed in real time with the AR device (20) worn by the driver of the vehicle. For example, the communication (101) between the electronic device and the AR device can mainly use a communication protocol to transmit data processed by the electronic device (10) constituting the infrastructure of multiple sensor devices installed on the road to the AR device (20), and the AR device (20) can display a warning sign through the display unit (21) to indicate that a pedestrian is currently located in the blind spot (31) and to be careful, and can transmit a guidance voice.

[0401] According to another embodiment of the present invention, the AR device processor unit (23) of the AR device (20) worn by the driver of a vehicle can detect and recognize in real time that a pedestrian is located in the blind spot (31). In this case, the AR device (20) can display a warning sign through the display unit (21) to indicate that a pedestrian is currently located in the blind spot (31), and can transmit a guidance voice. In addition, when the AR device (20) of a moving vehicle detects that a pedestrian is located in the blind spot (31), the AR device (20) of other vehicles located at a distance less than a threshold distance from the blind spot (31) can display a warning sign to indicate that a pedestrian is currently located in the blind spot (31), and can communicate to transmit a guidance voice. In addition, when an AR device (20) in a moving vehicle detects that a pedestrian is located in a blind spot (31), it is also possible to receive the corresponding data from the AR devices (20) of other vehicles located at a distance less than a critical distance from the blind spot (31) by communicating with an electronic device (10) or an external terminal (100) that constitutes a plurality of sensor device infrastructures installed on the road.

[0402] Embodiment of Fig. 4

[0403] FIG. 4A is a diagram for explaining an embodiment in which, when a contact accident occurs between a vehicle (30-5, 30-6) of a driver wearing an AR device according to an embodiment of the present invention, an external terminal (100) recognizes this and transmits information about whether a contact accident has occurred to an AR device (20) of a vehicle (30-1, 30-2, 30-3, 30-4) of a driver wearing another AR device. FIG. 4B is a diagram showing an augmented reality image projected in the form of a 2D map in the field of vision of a vehicle driver when, when a contact accident occurs between a vehicle (30-5, 30-6) of a driver wearing an AR device according to an embodiment of the present invention, the AR device determines that a contact accident has occurred and communicates this to an external terminal (100), and the external terminal (100) recognizes this and transmits information about whether a contact accident has occurred to an AR device (20) of a vehicle (30-1, 30-2, 30-3, 30-4) of a driver wearing another AR device.

[0404] FIG. 4c is a drawing showing an augmented reality image (21-1) indicating the point of occurrence of the contact accident projected into the field of vision of a vehicle driver when, in the event of a contact accident between a vehicle (30-5, 30-6) of a driver wearing an AR device according to one embodiment of the present invention, the AR device determines that there has been a contact accident and communicates this to an external terminal (100), and the external terminal (100) recognizes this and transmits information about whether or not a contact accident has occurred to an AR device (20) of a vehicle (30-1, 30-2, 30-3, 30-4) of a driver wearing another AR device.

[0405] In this specification, the driver assistance system using edge AI is described as an AR device (20) that provides AR worn by a vehicle driver. However, it includes not only an AR device (20) that provides AR worn by a vehicle driver, but also all embodiments in which drivers of transportation such as airplanes, helicopters, bicycles, and motorcycles wear AR devices, and is not limited thereto.

[0406] As illustrated, a case is explained assuming that a collision accident occurs between a vehicle (30-5, 30-6) of a driver wearing an AR device or a road condition becomes congested. In the attached drawing, a collision accident between a vehicle (30-5, 30-6) of a driver wearing an AR device is depicted, but it is not limited to a collision accident and may include all road events including accidents and congestion. In addition, although not shown in the attached drawing, all or some drivers of the vehicle (30-5, 30-6) of the driver wearing the AR device and the vehicle (30-1, 30-2, 30-3, 30-4) of the driver wearing other AR devices are wearing the AR device (20) of the present invention.

[0407] As illustrated, in the case where a collision accident occurs between a vehicle (30-5, 30-6) of a driver wearing an AR device, the AR device (20) worn by the driver of the vehicle (30-5, 30-6) of the driver wearing the AR device can capture the current collision accident, and the AR device processor (23) can process the data and transmit / receive the data to an external terminal (100). The external terminal (100) can receive the data, recognize that a collision accident has occurred between the vehicle (30-5, 30-6) of the driver wearing the AR device through the AI ​​processor (115) of the external terminal processor (110), and transmit the collision accident occurrence data to the AR device (20) worn by the driver of the vehicle (30-1, 30-2, 30-3, 30-4) of the driver wearing another AR device through the communication module (170).

[0408] According to one embodiment of the present invention, the AR device (20) worn by the driver of a vehicle (30-1, 30-2, 30-3, 30-4) of a driver wearing another AR device can visually display through the display unit (21) that a collision accident has occurred near the road on which the driver is currently driving, and can provide information on how much time delay there will be in reaching the destination because the collision accident has occurred near the road on which the driver is currently driving.

[0409] As illustrated in FIG. 4b, a driver wearing an AR device (20) can view an augmented reality image (21-1) through the display unit (21) and visually confirm that a collision accident has occurred ahead. According to one embodiment of the present invention, guidance such as a detour to another road can be provided on the display unit (21), and the AR device (20) can provide a function to provide real-time route guidance, so that a wearer of the AR device (20) can prepare in advance for a collision accident ahead by using another road or driving safely through the augmented reality image (21-1).

[0410] As shown in Fig. 4c, a driver wearing an AR device (20) can view an augmented reality image (21-1) through the display unit (21), and can also visually check in detail the point where a contact accident occurred in front.

[0411] Embodiment of Fig. 5

[0412] FIG. 5a is a drawing for explaining an embodiment of transmitting vehicle usage method information of a vehicle (30) of a driver wearing an AR device (20) from an external terminal (100) according to one embodiment of the present invention to a vehicle driver.

[0413] In this specification, the driver assistance system using edge AI is described as an AR device (20) that provides AR worn by a vehicle driver. However, it includes not only an AR device (20) that provides AR worn by a vehicle driver, but also all embodiments in which drivers of transportation such as airplanes, helicopters, bicycles, and motorcycles wear AR devices, and is not limited thereto.

[0414] FIG. 5b is a drawing showing an augmented reality image (21-1) projected into the field of vision of a vehicle driver when information on how to use a vehicle (30) of a driver wearing an AR device is transmitted to the vehicle driver from an external terminal (100) according to one embodiment of the present invention.

[0415] According to one embodiment of the present invention, a vehicle driver can wear an AR device (20), and since the AR device (20) is capable of data communication with an external terminal (100), the AR device (20) can receive a vehicle operation method for a specific vehicle of the vehicle driver from the external terminal (100) and sequentially display the method on the display unit (21) of the AR device (20), thereby allowing the vehicle driver to learn the driving method of the corresponding vehicle.

[0416] According to another embodiment of the present invention, a vehicle driver can wear an AR device (20), and the AR device processor unit (23) detects the vehicle, recognizes it, and sequentially displays a vehicle operation method for a specific vehicle of the vehicle driver on the display unit (21) of the AR device (20), thereby allowing the vehicle driver to learn how to drive the vehicle.

[0417] As illustrated in FIG. 5b, when a driver of a vehicle is riding in a vehicle (30) of a driver wearing an AR device, an augmented reality image (21-1) may be projected into the driver's field of vision when the driver wears the AR device (20). If the driver is inexperienced in driving the vehicle, the projected augmented reality image (21-1) may provide guidance on how to operate the vehicle. For example, the augmented reality image (21-1) may display the words "First of all, please fasten your seat belt" with an arrow pointing toward the seat belt, and a guidance voice may be simultaneously output from the AR device (20). As a result, the driver of the vehicle may recognize that he or she must fasten his or her seat belt first. Then, when the driver of the vehicle fastens his or her seat belt, the AR device processor unit (23) may recognize that the driver of the vehicle has fastened his or her seat belt, and the display unit (21) may guide the next step of operation. In this case, the augmented reality image (21-1) can provide a phrase such as "Next, please adjust the side mirrors," and when the vehicle driver adjusts the side mirrors appropriately, the AR device processor unit (23) can recognize that the vehicle driver has adjusted the side mirrors appropriately. Next, the augmented reality image (21-1) can provide a phrase such as "Next, please release the parking brake," and at this time, the augmented reality image (21-1) can display the parking brake portion using an effect such as an arrow or highlight so that the vehicle driver can recognize where the parking brake is located. At this time, when the vehicle driver releases the parking brake, the AR device processor unit (23) can recognize that the vehicle driver has released the parking brake.Next, the augmented reality image (21-1) can provide a phrase such as “Next, please turn on the engine,” and at this time, as shown in FIG. 5B, the augmented reality image (21-1) can display the engine part using an effect such as an arrow or highlight so that the driver of the vehicle can recognize where the engine part is located.

[0418] However, the present invention is not limited to these embodiments, and the method of operating the vehicle may differ depending on the type of vehicle, and the method of operating the vehicle may also differ depending on the model of the vehicle. Therefore, the AR device (20) may receive information reflecting the type of vehicle, the model of the vehicle, the degree of the driver's inexperience in driving, etc. from the external terminal (100) and provide an augmented reality image (21-1) accordingly. In addition, instead of the AR device (20) receiving information reflecting the type of vehicle, the model of the vehicle, the degree of the driver's inexperience in driving, etc. from the external terminal (100), the AR device processor unit (23) may detect the model of the vehicle, etc. on its own, recognize it, and provide an augmented reality image (21-1) accordingly.

[0419] Embodiment of Fig. 6

[0420] FIG. 6 is a drawing showing an augmented reality image (21-1) that provides distance information from a preceding vehicle among one embodiment of an augmented reality image (21-1) displayed on a display unit (21) of an AR device (20) according to the present invention.

[0421] In this specification, the driver assistance system using edge AI is described as an AR device (20) that provides AR worn by a vehicle driver. However, it includes not only an AR device (20) that provides AR worn by a vehicle driver, but also all embodiments in which drivers of transportation such as airplanes, helicopters, bicycles, and motorcycles wear AR devices, and is not limited thereto.

[0422] As illustrated, the display unit (21) can output data related to securing a safe distance from other vehicles located below a critical distance while driving as an augmented reality image.

[0423] According to one embodiment of the present invention, when a vehicle (30) of a driver wearing an AR device gets closer to another vehicle or a vehicle (30-1) of a driver wearing another AR device by a predetermined distance or less while driving, an image indicating the distance to the vehicle in front with an arrow may be provided along with a phrase and a guidance voice such as "The distance to the vehicle in front is getting narrower. Slow down" through an augmented reality image (21-1) displayed on the display unit (21) of the AR device (20). According to another embodiment, when the distance to the vehicle in front gets closer to a predetermined distance or less, an augmented reality image (21-1) displayed on the display unit (21) of the AR device (20) may simultaneously output a phrase such as "The distance to the vehicle in front is getting narrower. Slow down" along with a guidance voice of the AR device (20), thereby guiding the driver of the vehicle to recognize the distance to the vehicle in front and exercise caution. In this case, the AR device (20) can recognize that the vehicle is approaching a certain distance or less while driving through communication with an electronic device (10) or an external terminal (100) that constitutes a plurality of sensor device infrastructures, but it is also possible for the AR device processor unit (23) to determine that the vehicle is approaching a certain distance or less while driving on its own and to simultaneously output a guidance voice of the AR device (20) along with a phrase such as "The distance from the vehicle in front is narrow. Slow down" through an augmented reality image (21-1) displayed on the display unit (21) of the AR device (20), thereby guiding the vehicle driver to recognize the distance from the vehicle in front and be careful.

[0424] Embodiment of Fig. 7

[0425] FIG. 7 is a drawing showing an augmented reality image (21-1) that highlights a lane on a rainy day among one example of an augmented reality image (21-1) displayed on a display unit (21) of an AR device (20) according to the present invention.

[0426] In this specification, the driver assistance system using edge AI is described as an AR device (20) that provides AR worn by a vehicle driver. However, it includes not only an AR device (20) that provides AR worn by a vehicle driver, but also all embodiments in which drivers of transportation such as airplanes, helicopters, bicycles, and motorcycles wear AR devices, and is not limited thereto.

[0427] As shown, the display unit (21) can output data related to road lanes or road surface guide signs as an augmented reality image.

[0428] According to one embodiment of the present invention, when the weather is bad, such as when it rains or snows, while the vehicle (30) of a driver wearing an AR device is driving, the driver's field of vision may be obstructed, and the lane may appear blurry. In this case, the lane may be highlighted or a clear image of the lane may be provided on the lane through an augmented reality image (21-1) appearing on the display unit (21) of the AR device (20). According to another embodiment, when the weather is bad, such as when it rains or snows, while the vehicle (30) of a driver wearing an AR device is driving, the guidance text on the road may be highlighted or a clear image of the guidance text may be provided through an augmented reality image (21-1) appearing on the display unit (21) of the AR device (20). In this case, the AR device (20) can recognize that the weather is bad, such as rain or snow, through communication with an electronic device (10) or an external terminal (100) that constitutes a plurality of sensor device infrastructures, but it is also possible for the AR device processor unit (23) to determine that the weather is bad, such as rain or snow, and to highlight or clearly provide an image of the guidance text on the road through an augmented reality image (21-1) that appears on the display unit (21) of the AR device (20).

[0429] Embodiments of Figs. 8, 9, and 10

[0430] FIG. 8 is a drawing for explaining an embodiment of guiding a vehicle driver to an optimal parking space considering the destination of the vehicle driver through communication with an external terminal (100) and electronic devices (10-1, 10-2) constituting a plurality of sensor device infrastructures when the vehicle (30) of the driver wearing the AR device according to the present invention enters a parking lot.

[0431] In this specification, the driver assistance system using edge AI is described as an AR device (20) that provides AR worn by a vehicle driver. However, it includes not only an AR device (20) that provides AR worn by a vehicle driver, but also all embodiments in which drivers of transportation such as airplanes, helicopters, bicycles, and motorcycles wear AR devices, and is not limited thereto.

[0432] FIG. 9 is a drawing showing an augmented reality image (21-1) that appears to guide the driver to an optimal parking space considering the destination of the driver through communication with an external terminal (100) and electronic devices (10-1, 10-2) constituting a plurality of sensor device infrastructures when the driver's vehicle (30) wearing the AR device according to the present invention enters a parking lot.

[0433] FIG. 10 is a drawing showing an augmented reality image (21-1) that provides a movement path from a parking space in a building to a destination when a vehicle (30) of a driver wearing an AR device according to the present invention enters a parking lot, and the optimal parking space considering the destination of the driver is guided through communication with an external terminal (100) and electronic devices (10-1, 10-2) constituting a plurality of sensor device infrastructures.

[0434] As illustrated, the display unit (21) can output data related to the shortest distance parking space between the driver's destination and the parking location as an augmented reality image.

[0435] As illustrated, according to one embodiment of the present invention, when a vehicle driver has a destination (40) within a building such as a department store, outlet, or other complex shopping center, the distance to the destination (40) within the building of the specific vehicle driver may become relatively shorter depending on the parking location. Accordingly, as illustrated in FIG. 8, an external terminal (100) communicates with an AR device (20) or an electronic device (10) constituting a plurality of sensor device infrastructures with respect to the building such as the complex shopping center, so that the AR device (20) can guide the driver to an optimal parking location by taking into consideration the current parking situation and the optimal route to the destination (40) within the building of the specific vehicle driver.

[0436] As illustrated in FIG. 9, the vehicle driver can be guided to the optimal parking location determined by the AR device (20) in consideration of the current parking situation and the optimal route to the destination (40) within the building of the specific vehicle driver through an augmented reality image (21-1). Specifically, the driver can be visually guided to the optimal parking location in real time through an arrow on the augmented reality image (21-1) indicating the vehicle (30) of the driver currently wearing the AR device. As illustrated in FIG. 10, for example, if the destination (40) within the building of the specific vehicle driver is a movie theater within the building, if the driver parks in the current parking space provided by the AR device (20) through the augmented reality image (21-1), the driver can reach the destination (40) within the building of the specific vehicle driver through a certain route, and the driver can recognize that the parking space is the shortest distance to the destination (40) within the building of the specific vehicle driver. Accordingly, the augmented reality image (21-1) can provide a route to a destination (40) within a building for a specific vehicle driver, and can simultaneously provide a phrase and guidance voice such as "If you want to go to the movie theater, this is the closest place to park!!"

[0437] According to another embodiment of the present invention, when there is no optimal parking space, the AR device processor unit (23) can recommend to the driver of the vehicle a parking space with the shortest distance to the destination (40) within the building of the specific driver of the vehicle among the remaining parking spaces, and the driver of the vehicle can recognize that the parking space is the shortest distance to the destination (40) within the building of the specific driver of the vehicle. Accordingly, the augmented reality image (21-1) can provide a path to the destination (40) within the building of the specific driver of the vehicle.

[0438] According to another embodiment of the present invention, as illustrated in FIG. 8, a plurality of electronic devices (10-1, 10-2) constituting a plurality of sensor device infrastructures can be positioned within a parking lot, and communication can be established between the electronic devices (10-1, 10-2) constituting the plurality of sensor device infrastructures, communication can be established between an external terminal (100) and the electronic devices (10-1, 10-2) constituting the plurality of sensor device infrastructures, communication can be established between an AR device (20) and the electronic devices (10-1, 10-2) constituting the plurality of sensor device infrastructures, and communication can be established between the AR device (20) and the external terminal (100). In this case, through communication between the external terminal (100) and the electronic devices (10-1, 10-2) constituting the plurality of sensor device infrastructures, it is possible to determine an optimal parking space by communicating with the AR device (20) in real time in consideration of vacant spaces in the parking lot, preferred parking spaces of the vehicle driver, and the final destination of the vehicle driver. Alternatively, an augmented reality image (21-1) may be provided by highlighting optimal parking spots along with the location of the vehicle (30) of the driver wearing the AR device in the current parking lot. In addition, the AR device processor unit (23) may determine the optimal parking spot through communication with the electronic devices (10-1, 10-2) constituting the multiple sensor device infrastructure, without going through communication between the external terminal (100) and the electronic devices (10-1, 10-2) constituting the multiple sensor device infrastructure.

[0439] On-site facility control infrastructure utilizing edge AI

[0440] Overall composition

[0441] FIG. 11 is a diagram illustrating a field facility control infrastructure utilizing edge AI according to one embodiment of the present invention.

[0442] As illustrated in Fig. 11, the field facility control infrastructure (10-2) utilizing edge AI may include an edge AI infrastructure (200-2) and an integrated control server (300-2).

[0443] Specifically, the edge AI infrastructure (200-2) can receive location information (20-2) from at least one user terminal (100-2) and sense facility environment information (40-2) from an on-site facility (500-2). In addition, the edge AI infrastructure (200-2) can identify, through the integrated control server (300-2), personal information (30-2) transmitted to the integrated control server (300-2) by the user terminal (100-2) from which the location information (20-2) was received. In addition, the edge AI infrastructure can generate facility control information (50-2) based on at least one of the location information (20-2), personal information (30-2), and facility environment information (40-2), and transmit the same to the integrated control server (300-2).

[0444] According to the present invention, the integrated control server (300-2) can receive personal information (30-2) from at least one user terminal (100-2) of a user, and store the received personal information (30-2). In addition, the integrated control server (300-2) can receive facility control information (50-2) from the edge AI infrastructure (200-2), and control the field facility (500-2) based on the facility control information (50-2). In addition, the integrated control server (300-2) can generate user terminal control information (60-2) based on the facility control information (50-2), and transmit the user terminal control information (60-2) to the user terminal (100-2) so that the user terminal control information (60-2) is output to the user terminal (100-2).

[0445] According to the present invention, location information (20-2) may mean information about the location of the user terminal (100-2) and information about the location of a user using the user terminal (100-2).

[0446] According to the present invention, personal information (30-2) includes information about a user terminal (100-2) that transmits personal information (30-2) to an integrated control server (300-2). In addition, personal information (30-2) may include information about a user using the user terminal (100-2) and reservation details for the user's on-site facility (500-2) entered into the user terminal (100-2).

[0447] According to the present invention, there are cases where the user terminal (100-2) does not transmit personal information (30-2) to the integrated control server (300-2). In this case, the personal information (30-2) includes information that the user terminal (100-2) did not transmit any personal information (30-2) to the integrated control server (300-2).

[0448] According to the present invention, the identification of personal information (30-2) by the edge AI system (210-2) through the integrated control server (300-2) may mean integrating user-related information input into the user terminal (100-2) with location information (20-2). Specifically, the step of the edge AI system (210-2) identifying personal information (30-2) through the integrated control server (300-2) includes the step of confirming personal information (30-2) transmitted by the user terminal (100-2) from which the location information (20-2) was received to the integrated control server (300-2).

[0449] According to the present invention, the step of identifying personal information (30-2) through the integrated control server (300-2) by the edge AI system (210-2) includes the step of integrating information related to at least one of the user's name, age, address, mobile phone number, service reservation details, and consent to use of personal information, which is input into the user terminal (100-2) located at the point included in the location information (20-2), with the location information (20-2).

[0450] According to the present invention, the edge AI system (210-2) identifying personal information (30-2) may mean that, if the user terminal (100-2) does not transmit personal information (30-2) to the integrated control server (300-2), the user terminal (100-2) is confirmed to be a user terminal (100-2) in which personal information (30-2) is not stored in the integrated control server (300-2).

[0451] According to the present invention, the facility environment information (40-2) may refer to information about the environment of the on-site facility (500-2). For example, the facility environment information (40-2) may refer to information about the temperature, humidity, illuminance, air quality, and sound of the on-site facility (500-2). In addition, the facility environment information (40-2) may refer to information about the occurrence of a disaster, such as a fire, power outage, or earthquake, occurring in the on-site facility (500-2). In addition, the facility environment information (40-2) may refer to information about a crime occurring within the on-site facility (500-2).

[0452] According to the present invention, facility control information (50-2) may be information configured to enable the integrated control server (300-2) to control the field facility (500-2). In addition, facility control information (50-2) may be information configured to enable the integrated control server (300-2) to transmit user terminal control information (60-2) to the user terminal (100-2).

[0453] According to the present invention, the user terminal control information (60-2) may be information configured to control the user terminal (100-2) so that information corresponding to the facility control information (50-2) is output to the user terminal (100-2). For example, the user terminal (100-2) may receive the user terminal control information (60-2) and display a user interface corresponding to the user terminal control information (60-2). In addition, the user terminal (100-2) may receive the user terminal control information (60-2) and output a voice corresponding to the user terminal control information (60-2).

[0454] Hereinafter, a field facility control infrastructure (10-2) utilizing an edge AI system according to one embodiment of the present invention will be described in detail with reference to the drawings attached to this specification.

[0455] User terminal

[0456] According to the present invention, the user terminal (100-2) may include a mobile phone, a smart phone, a laptop computer, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, a tablet PC, an ultrabook, a wearable device (e.g., a smart watch), a glass-type terminal (smart glass), a head mounted display (HMD), etc., but is not limited thereto.

[0457] According to the present invention, a user terminal (100-2) may refer to any type of entity(ies) in a system having a mechanism for communicating with a computing device. For example, the user terminal (100-2) may include a PC, a laptop, a mobile terminal, a smart phone, a tablet PC, an AI speaker, an AI TV, a wearable device, etc. In addition, the user terminal (100-2) may include any type of terminal capable of connecting to a wired / wireless network. In addition, the user terminal (100-2) may include any server implemented by at least one of an agent, an Application Programming Interface (API), and a plug-in. In addition, the user terminal (100-2) may include an application source and / or a client application.

[0458] Application

[0459] According to one embodiment of the present invention, a user terminal (100-2) may be installed with an application (110-2) that can be utilized for facility use. Specifically, the user terminal (100-2) may provide a user interface through which a user can register for membership via the application (110-2). Furthermore, the user terminal (100-2) may provide a user interface through which a user can consent to the use of their personal information via the application (110-2). Furthermore, the user terminal (100-2) may provide a user interface through which a map of the interior of the on-site facility (500-2) can be displayed via the application (110-2).

[0460] According to the present invention, the user terminal (100-2) can provide a user interface capable of displaying the user's current location through the application (110-2). Furthermore, the user terminal (100-2) can provide a user interface capable of displaying a route from the user's current location to a specific location within the field facility (500-2) through the application (110-2).

[0461] According to the present invention, the user terminal (100-2) can provide a user interface that can display the location of the emergency entrance door of the on-site facility (500-2) through the application (110-2). Furthermore, the user terminal (100-2) can provide a user interface that allows the user to reserve services provided at the on-site facility (500-2) through the application (110-2). Furthermore, the user terminal (100-2) can provide a user interface that allows the user to check the user's reserved services through the application (110-2).

[0462] According to the present invention, the user terminal (100-2) can provide, through the application (110-2), a user interface that can display the location of the parking lot within the on-site facility (500-2) closest to the service provision location reserved by the user, and a user interface that can display the distribution of users according to the space within the on-site facility (500-2). However, this is not limited thereto.

[0463] According to the present invention, the user terminal (100-2) can provide, through the application (110-2), sound that can guide a map inside the field facility (500-2) by voice, sound that can guide the user's current location by voice, and sound that can guide a route from the user's current location to a specific location inside the field facility (500-2) by voice.

[0464] According to the present invention, the user terminal (100-2) can provide, through the application (110-2), a sound that can guide the location of the emergency entrance door of the on-site facility (500-2) by voice, a sound that can guide the user by voice so that the user can reserve a service provided at the on-site facility (500-2), and a sound that can guide the user by voice so that the user can confirm the user's reserved service.

[0465] According to the present invention, the user terminal (100-2) can provide, through an application (110-2), audio that can provide voice guidance on the location of the parking lot within the on-site facility (500-2) closest to the service provision location reserved by the user, and audio that can provide voice guidance on the distribution of users according to the space within the on-site facility (500-2). However, this is not limited thereto.

[0466] Edge AI Infrastructure

[0467] FIG. 12a is a diagram illustrating an edge AI infrastructure comprising at least one edge AI system according to the present invention. FIG. 12b is a diagram illustrating an edge AI system comprising a head device and at least one node device according to the present invention.

[0468] As described above, the field facility control infrastructure (10-2) utilizing the edge AI system may include the edge AI infrastructure (200-2).

[0469] As illustrated in FIGS. 12A and 12B , the edge AI infrastructure (200-2) may include at least one edge AI system (210-2). The edge AI system (210-2) may include at least one node device (220-2) and a head device (230-2). Hereinafter, the edge AI infrastructure (200-2), the edge AI system (210-2), the node device (220-2), and the head device (230-2) will be described in detail with reference to FIGS. 12A to 12F .

[0470] Edge AI system

[0471] As illustrated in FIG. 12a, the edge AI infrastructure (200-2) may include at least one edge AI system (210-2).

[0472] As illustrated in FIG. 12b, the edge AI system (210-2) may include at least one node device (220-2) and a head device (230-2).

[0473] Hereinafter, the node device (220-2) and the head device (230-2) according to the present invention will be described in detail with reference to FIGS. 12c and 12f.

[0474] Node device

[0475] FIG. 12c is a diagram showing a node device configured to receive location information from a user terminal according to the present invention and sense facility environment information from a field facility.

[0476] As illustrated in (a) of FIG. 12c, the node device (220-2) can communicate with at least one user terminal (100-2) of a user and receive location information (20-2) from the user terminal (100-2). For example, the node device (220-2) can obtain location information (20-2) using at least one of a UWB-based tracking method and a Wi-Fi waveform pattern analysis method. However, the present invention is not limited thereto.

[0477] According to the present invention, a UWB-based tracking method achieves high resolution by shortening signal transmission times, thereby enabling high-precision location tracking. Furthermore, UWB-based tracking is less affected by walls or obstacles, making it easier to track location in indoor environments compared to GPS. Furthermore, UWB-based tracking consumes less power, making it suitable for mobile devices, edge devices, and wearables where battery life is critical.

[0478] According to the present invention, the Wi-Fi waveform pattern analysis method can utilize existing Wi-Fi infrastructure, thereby reducing the cost of additional hardware installation. Furthermore, the Wi-Fi waveform pattern analysis method can update location information at a shorter interval than GPS. Furthermore, due to the characteristic of Wi-Fi signals that can maintain a strong signal even indoors, the Wi-Fi waveform pattern analysis method is more advantageous for location tracking in indoor environments than GPS, which weakens its signal indoors.

[0479] As illustrated in (b) of FIG. 12c, the node device (220-2) may include a sensor unit (221-2). Specifically, the node device (220-2) may sense facility environment information (40-2) through the sensor unit (221-2). For example, the sensor unit (221-2) may be at least one environmental sensor selected from the group consisting of a camera, an infrared camera, a temperature sensor, an illuminance sensor, a vibration sensor, an air sensor, and an acoustic sensor. However, the present invention is not limited thereto.

[0480] According to one embodiment of the invention, the edge AI system (210-2) may include at least one node device (220-2), and the sensor unit (221-2) of each node device (220-2) among the at least one node device (220-2) may be a different sensor.

[0481] head device

[0482] Figure 12d is a diagram illustrating a head device including a network unit, a memory unit, and a processor unit according to the present invention. Figure 12e is a diagram illustrating a head device configured to transmit facility control information according to the present invention to an integrated control server. Figure 12f is a schematic diagram illustrating one or more network functions for generating facility control information according to the present invention.

[0483] As illustrated in FIG. 12D, the head device (230-2) may include a network unit (231-2), a memory unit (232-2), and a processor unit (233-2). Specifically, the head device (230-2) may communicate with an external device or server, such as a user terminal (100-2), a node device (220-2), or an integrated control server (300-2), through the network unit (231-2). In addition, the head device (230-2) may store information received from the outside through the memory unit (232-2) or store an analysis model, such as an artificial intelligence model (234-2). In addition, the head device (230-2) may process information received from the outside through the processor unit (233-2) or drive an analysis model, such as an artificial intelligence model (234-2), stored in the memory unit (232-2).

[0484] As illustrated in FIG. 12e, the head device (230-2) can communicate with at least one node device (220-2) and an integrated control server (300-2) via a network unit (231-2). Specifically, the head device (230-2) can receive location information (20-2) and facility environment information (40-2) from the node device (220-2) via the network unit (231-2). In addition, specifically, the head device (230-2) can identify personal information (30-2) transmitted to the integrated control server (300-2) by a user terminal (100-2) that has received location information (20-2) via the network unit (231-2) via the integrated control server (300-2). In addition, specifically, the head device (230-2) can transmit facility control information (50-2) to the integrated control server (300-2) via the network unit (231-2). Here, the facility control information (50-2) may be information generated by the head device (230-2) based on at least one of location information (20-2), personal information (30-2), and facility environment information (40-2).

[0485] According to the present invention, the head device (230-2) can generate facility control information (50-2) by using at least one of location information (20-2), personal information (30-2), and facility environment information (40-2) as input to the artificial intelligence model (234-2). Here, the facility control information (50-2) may mean information configured to enable the integrated control server (300-2) to control the field facility (500-2) and / or information configured to enable the integrated control server (300-2) to transmit user terminal control information (60-2) to the user terminal (100-2).

[0486] According to the present invention, the artificial intelligence model (234-2) utilized by the head device (230-2) to generate facility control information (50-2) may be stored in the memory unit (232-2) of the head device (230-2). In addition, the processor unit (233-2) of the head device (230-2) may drive the artificial intelligence model (234-2) stored in the memory unit (232-2). When the head device (230-2) receives or identifies at least one of location information (20-2), personal information (30-2), and facility environment information (40-2), the head device (230-2) may utilize the artificial intelligence model (234-2) driven by the processor unit (233-2) to process the same, thereby generating facility control information (50-2).

[0487] As illustrated in FIG. 12F, the artificial intelligence model (234-2) of the head device (230-2) is composed of one or more network functions, and the one or more network functions may be composed of a set of interconnected computational units, which may generally be referred to as 'nodes'. These 'nodes' may also be referred to as 'neurons'. The one or more network functions are composed of at least one node. The nodes (or neurons) constituting the one or more network functions may be interconnected by one or more 'links'.

[0488] According to the present invention, one or more nodes connected through links can form a relationship between input nodes and output nodes relatively within a neural network. The concept of input nodes and output nodes is relative, and any node in an output node relationship with respect to one node can also be in an input node relationship with respect to another node, and vice versa. As described above, the relationship between input nodes and output nodes can be created based on links. One or more output nodes can be connected to one input node through links, and vice versa.

[0489] According to the present invention, in a relationship between input nodes and output nodes connected through a single link, the value of the output node can be determined based on data input to the input node. Here, the node interconnecting the input node and the output node can have a weight. The weight can be variable and can be varied by a user or an algorithm so that the neural network can perform a desired function. For example, when one or more input nodes are interconnected to one output node through respective links, the output node can determine the output node value based on the values ​​input to the input nodes connected to the output node and the weights set for the links corresponding to the respective input nodes.

[0490] According to the present invention, a neural network is formed by interconnecting one or more nodes through one or more links, thereby forming input and output node relationships within the neural network. Furthermore, the characteristics of the neural network can be determined based on the number of nodes and links within the neural network, the correlations between the nodes and links, and the weight values ​​assigned to each link. For example, if two neural networks exist with the same number of nodes and links and different weight values ​​between the links, the two neural networks can be recognized as different from each other.

[0491] According to the present invention, some of the nodes constituting a neural network can form a layer based on their distances from the initial input node. For example, a set of nodes that are n distances from the initial input node can form n layers. The distance from the initial input node can be defined by the minimum number of links that must be passed through to reach the node from the initial input node. However, this definition of a layer is arbitrary for the purpose of explanation, and the order of a layer within a neural network can be defined in a different way than described above. For example, the layer of nodes can also be defined by their distance from the final output node.

[0492] According to the present invention, the initial input node may refer to one or more nodes in a neural network into which data is directly input without going through links in their relationship with other nodes. Alternatively, the initial input node may refer to nodes that do not have other input nodes connected by links in the relationship between nodes based on links in the neural network. Similarly, the final output node may refer to one or more nodes in a neural network that do not have output nodes in their relationship with other nodes.

[0493] According to the present invention, hidden nodes may refer to nodes that constitute a neural network other than the initial input node and the final output node. A neural network according to one embodiment of the present invention may have more nodes in the input layer than in the hidden layer closer to the output layer, and may be a neural network in which the number of nodes decreases as it progresses from the input layer to the hidden layer.

[0494] According to the present invention, a neural network may include one or more hidden layers. The hidden nodes of the hidden layers may take as input the output of the previous layer and the output of the surrounding hidden nodes. The number of hidden nodes for each hidden layer may be the same or different. The number of nodes of the input layer may be determined based on the number of data fields of the input data and may be the same as or different from the number of hidden nodes. Input data input to the input layer may be operated by the hidden nodes of the hidden layer and may be output by the output layer, which is a fully connected layer (FCL).

[0495] In the present invention, the network function may include a deep neural network (DNN). A deep neural network (DNN) may refer to a neural network that includes multiple hidden layers in addition to an input layer and an output layer. Using a deep neural network, it is possible to identify latent structures in data. Specifically, it is possible to identify the latent structures of photos, text, videos, voices, and music (e.g., what objects are in a photo, what the content and emotion of a text are, what the content and emotion of a voice are, etc.).

[0496] According to the present invention, the deep neural network may include a convolutional neural network (CNN), a recurrent neural network (RNN), an auto encoder, a generative adversarial network (GAN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a Q network, a U network, a Siamese network, a transformer, a vision transformer (ViT), a mobile vision transformer (Mobile ViT), and the like. The description of the above-described deep neural network is merely an example, and the present invention is not limited thereto.

[0497] In the present invention, the network function may include an autoencoder. An autoencoder may be a type of artificial neural network that outputs output data similar to input data. The autoencoder may include at least one hidden layer, and an odd number of hidden layers may be positioned between the input and output layers.

[0498] According to the present invention, the number of nodes in each layer may be reduced from the number of nodes in the input layer to an intermediate layer called a bottleneck layer (encoding), and then expanded in a symmetrical manner from the bottleneck layer to the output layer (symmetrical to the input layer).

[0499] According to the present invention, the nodes of the dimensionality reduction layer and the dimensionality restoration layer may or may not be symmetrical. The autoencoder can perform nonlinear dimensionality reduction. The number of input layers and output layers may correspond to the number of sensors remaining after preprocessing the input data.

[0500] According to the present invention, in an autoencoder structure, the number of nodes in a hidden layer included in the encoder may have a structure in which the number of nodes decreases as it moves away from the input layer. If the number of nodes in the bottleneck layer (the layer with the fewest nodes located between the encoder and the decoder) is too small, a sufficient amount of information may not be transmitted, so it may be maintained at a certain number or more (e.g., more than half of the input layer).

[0501] According to the present invention, a neural network can be trained using at least one of supervised learning, unsupervised learning, and semi-supervised learning. Training of a neural network is aimed at minimizing output errors. Training of a neural network involves repeatedly inputting training data into a neural network, calculating the output of the neural network and the target error for the training data, and backpropagating the error of the neural network from the output layer of the neural network toward the input layer in a direction to reduce the error, thereby updating the weights of each node of the neural network.

[0502] According to the present invention, in the case of supervised learning, training data with correct answers labeled for each training data is used (i.e., labeled training data), whereas in the case of unsupervised learning, the correct answers may not be labeled for each training data. That is, for example, in the case of supervised learning for data classification, the training data may be data in which categories are labeled for each training data. The labeled training data is input to a neural network, and an error can be calculated by comparing the output (category) of the neural network with the labels of the training data.

[0503] As another example, in unsupervised learning for data classification, an error can be calculated by comparing the input training data with the neural network output. The calculated error is backpropagated through the neural network (i.e., from the output layer to the input layer), and the connection weights of each node in each layer of the neural network can be updated based on this backpropagation. The amount of change in the connection weights of each node can be determined by the learning rate.

[0504] According to the present invention, the neural network's calculations on input data and backpropagation of errors can constitute a learning cycle (epoch). The learning rate can be applied differently depending on the number of iterations of the neural network's learning cycle. For example, a high learning rate can be used early in the neural network's training process to quickly achieve a certain level of performance, thereby increasing efficiency. A lower learning rate can be used later in the training process to increase accuracy.

[0505] According to the present invention, in learning a neural network, learning data may generally be a subset of actual data (i.e., data to be processed using a learned neural network), and therefore, a learning cycle may exist in which errors for the learning data decrease but errors for the actual data increase.

[0506] According to the present invention, overfitting is a phenomenon in which excessive training on training data leads to increased errors in actual data. For example, a neural network trained on yellow cats and then fails to recognize cats of any color other than yellow could be a form of overfitting.

[0507] According to the present invention, overfitting can increase errors in AI algorithms. Various optimization methods can be used to prevent overfitting. Furthermore, methods such as increasing the training data, regularization, or dropout, which omits some nodes from the network during the learning process, can be applied to prevent overfitting.

[0508] According to the present invention, the terms computational model, neural network, network function, and neural network may be used interchangeably. (Hereinafter, they are collectively referred to as neural networks.) The data structure may include a neural network. And the data structure including the neural network may be stored in a computer-readable medium. The data structure including the neural network may also include data input to the neural network, weights of the neural network, hyperparameters of the neural network, data obtained from the neural network, activation functions associated with each node or layer of the neural network, and a loss function for learning the neural network.

[0509] According to the present invention, a data structure including a neural network may include any of the components described above. That is, the data structure including a neural network may be configured to include all or any combination of data input to the neural network, weights of the neural network, hyperparameters of the neural network, data obtained from the neural network, activation functions associated with each node or layer of the neural network, and loss functions for training the neural network. In addition to the aforementioned components, the data structure including a neural network may include any other information that determines the characteristics of the neural network.

[0510] According to the present invention, the data structure may include all forms of data used or generated in the computational process of a neural network, and is not limited to the aforementioned. The computer-readable medium may include a computer-readable recording medium and / or a computer-readable transmission medium. The neural network may be composed of a set of interconnected computational units, which may generally be referred to as nodes. These nodes may also be referred to as neurons. The neural network is composed of at least one node.

[0511] According to the present invention, the artificial intelligence model (234-2) can be trained to receive at least one of location information (20-2), personal information (30-2), and facility environment information (40-2) and output facility control information (50-2). For example, there is a case where the artificial intelligence model (234-2) receives personal information (30-2) of a user who has reserved a specific service of an on-site facility (500-2) and location information (20-2) including information that the user has arrived at the entrance of the on-site facility (500-2). In this case, the artificial intelligence model (234-2) can be trained to output facility control information (50-2), which is information related to controlling the entrance door of the on-site facility (500-2). In addition, the artificial intelligence model (234-2) can be trained to output facility control information (50-2), which is information related to the location information of the parking lot closest to the place where the reservation service is provided by the user terminal (100-2). Specifically, the processor unit (233-2) of the head device (230-2) can control the facility control information (50-2) output by the artificial intelligence model (234-2) to be displayed on the display of the user terminal (100-2).

[0512] Placement of node devices and head devices

[0513] According to one embodiment of the present invention, at least one node device (220-2) may be placed at a predetermined distance inside or outside the field facility (500-2). However, the present invention is not limited to the field facility control infrastructure (10-2) utilizing an edge AI system in which the distance between at least one node device (220-2) is predetermined, and it is not excluded that at least one node device (220-2) may be placed at different distances from each other.

[0514] According to one embodiment of the present invention, the head devices (230-2) may be placed at a certain distance inside or outside the field facility (500-2). In addition, each head device (230-2) may be placed so as to be able to communicate with a certain number of node devices (220-2). However, the present invention is not limited to the facility control infrastructure (10-2) utilizing the edge AI system in which the distance between the head devices (230-2) is constant, and it is not excluded that the head devices (230-2) may be installed at different distances from each other. In addition, the present invention is not limited to the facility control infrastructure (10-2) utilizing the edge AI system in which the number of node devices (220-2) with which each head device (230-2) can communicate is constant.

[0515] Integrated Control Server

[0516] Figure 13a is a diagram illustrating an integrated control server configured to receive personal information from a user terminal according to the present invention and to allow an edge AI system to identify the personal information through the integrated control server. Figure 13b is a diagram illustrating an integrated control server configured to control field facilities based on facility control information according to the present invention.

[0517] As illustrated in FIG. 13A, the integrated control server (300-2) can receive personal information (30-2) input from a user terminal (100-2) via the user terminal (100-2). Here, inputting the personal information (30-2) into the user terminal (100-2) can be performed via an application (110-2) that can be utilized for utilizing the on-site facility (500-2). In addition, the integrated control server (300-2) can store the received personal information (30-2). In addition, the integrated control server (300-2) can transmit the stored personal information (30-2) to the edge AI system (200-2).

[0518] As illustrated in FIG. 13b, the integrated control server (300-2) can receive facility control information (50-2) from the edge AI system (200-2). Furthermore, the integrated control server (300-2) can generate user terminal control information (60-2) based on the facility control information (40-2) and transmit the user terminal control information (60-2) to the user terminal (100-2). Furthermore, the integrated control server (300-2) can control the on-site facility (500-2) based on the facility control information (50-2).

[0519] Digital Twin System

[0520] FIG. 14 is a diagram showing a digital twin system that can receive facility control information from an integrated control server according to the present invention and create a digital twin for a field facility based on the facility control information.

[0521] As illustrated in FIG. 14, the on-site facility control infrastructure (10-2) utilizing the edge AI system may further include a digital twin system (400-2). The digital twin system (400-2) may include a receiving unit (410), a visualization module (420-2), and a display unit (430-2). Specifically, the digital twin system (400-2) may receive facility control information (50-2) from the integrated control server (300-2) through the receiving unit (410). In addition, the digital twin system (400-2) may combine the facility control information (50-2) received through the visualization module (420-2) with the spatial information of the on-site facility (500-2), and may create a digital twin model for the on-site facility (500-2) that visualizes at least one of the facility control information (50-2) and the spatial information of the on-site facility (500-2). Additionally, the digital twin system (400-2) can display a digital twin model for the field facility (500-2) created through the display unit (430-2).

[0522] In the present invention, a digital twin model may refer to a virtual representation of an actual physical object, system, process, or environment in a digital space. Digital twin technology enables various simulations, analyses, and predictions through interactions between physical and virtual objects through real-time data exchange.

[0523] That is, the digital twin model according to the present invention can reproduce the physical space of the field facility (500-2) as a corresponding digital space, and display personal information (30-2) and location information (20-2) of users of the field facility (500-2), information on service reservation details of the field facility (500-2), information on the operating status of devices of the field facility (500-2), information on the internal environment of the field facility (500-2), etc., in the reproduced digital space.

[0524] A method for controlling on-site facilities using at least one edge AI system and an integrated control server.

[0525] FIG. 15a is a diagram illustrating a field facility control method utilizing at least one edge AI system and an integrated control server according to one embodiment of the present invention. FIG. 15b is a diagram for detailing the field facility control method illustrated in FIG. 15a.

[0526] As illustrated in FIG. 15a, a field facility control method (70-2) utilizing at least one edge AI system and an integrated control server may include a human information (30-2) acquisition step (S100-2), a location information (20-2) acquisition step (S200-2), a facility environment information (40-2) acquisition step (S300-2), a human information (30-2) and location information (20-2) integration step (S400-2), a facility control information (50-2) generation step (S500-2), a field facility (500-2) control step (S600-2), a user terminal (100-2) control step (S700-2), and a digital twin model generation step (S800-2). Here, the human information (30-2) acquisition step (S100-2), the location information (20-2) acquisition step (S200-2), the facility environment information (40-2) acquisition step (S300-2), and the human information (30-2) and location information (20-2) integration step (S400-2) are not necessarily performed sequentially and may be performed in parallel. In addition, the field facility control (500-2) step (S600-2), the user terminal (100-2) control step (S700-2), and the digital twin model creation step (S800-2) are not necessarily performed sequentially and may be performed in parallel.

[0527] Hereinafter, with reference to FIG. 15b, a field facility control method (70-2) utilizing at least one edge AI system and an integrated control server according to one embodiment of the present invention will be described in detail.

[0528] As illustrated in FIG. 15b, the personal information (30-2) acquisition step (S100) may include a step (S110) in which a user terminal (100-2) receives personal information (30-2), a step (S120-2) in which the user terminal (100-2) transmits the personal information (30-2) to an integrated control server (300-2), a step (S130-2) in which the integrated control server (300-2) receives the personal information (30-2), and a step (S140-2) in which the integrated control server (300-2) stores the personal information (30-2).

[0529] According to the present invention, the step (S200) of obtaining location information (20-2) may include a step (S210) in which the user terminal (100-2) transmits the location information (20-2) to the edge AI system (210-2) and a step (S220-2) in which the edge AI system (210-2) receives the location information (20-2) from the user terminal (100-2).

[0530] According to the present invention, the step (S300-2) of obtaining facility environment information (40-2) may include a step (S310-2) in which the edge AI system (210-2) senses the facility environment information (40-2).

[0531] According to the present invention, the step (S400-2) of integrating personal information (30-2) and location information (20-2) may include a step (S410-2) in which the edge AI system (210-2) identifies personal information (30-2) transmitted from the user terminal (100-2) from which the location information (20-2) was received to the integrated control server (300-2) by the integrated control server (300-2).

[0532] According to the present invention, the facility control information (50-2) generation step (S500) may include a step (S510-2) in which the edge AI system (210-2) generates the facility control information (50-2) based on at least one of location information (20-2), personal information (30-2), and facility environment information (40-2), a step (S520-2) in which the edge AI system (210-2) transmits the facility control information (50-2) to the integrated control server (300-2), and a step (S530) in which the integrated control server (300-2) receives the facility control information (50-2) from the edge AI system (210-2).

[0533] According to the present invention, the field facility (500-2) control step (S600-2) may include a step (S610-2) in which the integrated control server (300-2) controls the field facility (500-2) based on the facility control information (50-2).

[0534] According to the present invention, the user terminal (100-2) control step (S700-2) may include a step (S710-2) in which the integrated control server (300-2) generates user terminal control information (60-2) based on facility control information (50-2), a step (S720-2) in which the integrated control server (300-2) transmits the user terminal control information (60-2) to the user terminal (100-2), a step (S730-2) in which the user terminal (100-2) receives the user terminal control information (60-2) from the integrated control server (300-2), and a step (S740-2) in which the user terminal (100-2) outputs the user terminal control information (60-2).

[0535] According to the present invention, the digital twin model creation step (S800-2) may include a step (S810-2) in which the integrated control server (300-2) transmits facility control information (50-2) to the digital twin system (400-2), a step (S820-2) in which the digital twin system (400-2) creates a digital twin model for the field facility (500-2) based on the facility control information (50-2), and a step (S830-2) in which the digital twin system (400-2) displays the digital twin model.

[0536] An example of a field facility control infrastructure utilizing edge AI.

[0537] FIGS. 16A to 16F are diagrams illustrating one embodiment of a field facility control infrastructure utilizing edge AI according to the present invention.

[0538] Specifically, FIGS. 16a to 16f are drawings illustrating an embodiment of a field facility control infrastructure (10-2) related to automatic control of entrance / exit doors and parking lot guidance.

[0539] As illustrated in FIG. 16a, the integrated control server (300-2) may receive personal information (30-2) from the user terminal (100-2) of a user who has reserved a service provided at an on-site facility (500-2) and store the personal information (30-2). Specifically, the personal information (30-2) may refer to information related to one or more of the name, age, address, mobile phone number, service reservation details, and consent to the use of personal information of the user who has reserved the service.

[0540] As illustrated in FIG. 16b, there may be a plurality of node devices (220-2). For example, the node devices (220-2) may include a first node device (220a-2), a second node device (220b-2), a third node device (220c-2), and a fourth node device (220d-2). Here, the third node device (220c-2) may receive location information (20-2) from the user terminal (100-2) of the user when the user is located near the entrance of the field facility (500-2). In addition, the head device (230-2) may receive location information (20-2) from the third node device (220c-2).

[0541] As illustrated in FIG. 16c, the head device (230-2) can identify personal information (30-2) transmitted by the user terminal (100-2) that received the location information (20-2) to the integrated control server (300-2) through the integrated control server (300-2). Specifically, the head device (230-2) can integrate the location information (20-2) and the personal information (30-2) to obtain information related to at least one of the name, age, address, mobile phone number, service reservation details, and consent to the use of personal information of the user located at the location included in the location information (20-2). For example, the head device (230-2) can obtain information that a user who has reserved a service provided at the on-site facility (500-2) has arrived near the entrance of the on-site facility (500-2).

[0542] As illustrated in FIG. 16d, the head device (230-2) can generate facility control information (50-2) using location information (20-2) and personal information (30-2) as inputs to the artificial intelligence model (234-2), and transmit the facility control information (50-2) to the integrated control server (300-2). For example, the facility control information (50-2) can be configured so that the integrated control server (300-2) controls the entrance door of the field facility (500-2). In addition, the facility control information (50-2) can be configured so that the integrated control server (300-2) transmits user terminal control information (60-2) to the user terminal (100-2).

[0543] As illustrated in FIG. 16e, when the integrated control server (300-2) receives facility control information (50-2) from the head device (230-2), the integrated control server (300-2) can control the door of the on-site facility (500-2) to open. In addition, the integrated control server (300-2) can control the speaker device of the door to output a voice message such as “Welcome, Mr. OOO.”

[0544] As illustrated in FIG. 16f, when the integrated control server (300-2) receives facility control information (50-2) from the head device (230-2), the integrated control server (300-2) can generate user terminal control information (60-2) based on the facility control information (50-2) and transmit the user terminal control information (60-2) to the user terminal (100-2). The user terminal (100-2) can receive the user terminal control information (60-2) and output the user terminal control information (60-2). For example, the user terminal (100-2) can display a user interface that indicates the shortest path from the entrance of the on-site facility (500-2) to the second parking lot, which is the parking lot within the on-site facility (500-2) that is closest to the reservation service provision location.

[0545] According to one embodiment of the present invention, the digital twin system (400-2) receives facility control information (50-2) from the integrated control server (300-2), and generates a digital twin model of the field facility (500-2) based on the facility control information (50-2) and spatial information of the field facility (500-2) and displays the same. For example, when the digital twin system (400-2) receives the facility control information (50-2), the digital twin system (400-2) can generate and display a digital twin model of the field facility (500-2) that includes user personal information, real-time location information, and information on whether the entrance door of the field facility (500-2) is open or closed.

[0546] Another example of a field facility control infrastructure leveraging edge AI.

[0547] FIGS. 17a to 17e are drawings illustrating another embodiment of a field facility control infrastructure utilizing edge AI according to the present invention.

[0548] Specifically, FIGS. 17a to 17e are drawings illustrating an example of a field facility control infrastructure (10-2) related to a power outage situation within a field facility (500-2).

[0549] As illustrated in FIG. 17A, there may be a plurality of node devices (220-2). For example, the node devices (220-2) may include a first node device (220a-2), a second node device (220b-2), a third node device (220c-2), and a fourth node device (220d-2). Here, the second node device (220b-2) may receive location information (20-2) from a user terminal (100-2) of a user located in a field facility (500-2). In addition, the head device (230-2) may receive location information (20-2) from the second node device (220b-2).

[0550] As illustrated in FIG. 17b, when a power outage occurs in a specific space within a field facility (500-2), the second node device (220b-2) located in the specific space can sense facility environment information (40-2) indicating that a power outage occurred in a specific space within the field facility (500-2). At this time, the sensor unit (not illustrated) of the second node device (220b-2) may be a light sensor, but is not limited thereto. Meanwhile, the head device (230-2) can receive facility environment information (40-2) indicating that a power outage occurred in a specific space within the field facility (500-2) from the second node device (220b-2).

[0551] As illustrated in FIG. 17c, the head device (230-2) can generate facility control information (50-2) using location information (20-2) and facility environment information (40-2) as inputs to the artificial intelligence model (234-2), and transmit the facility control information (50-2) to the integrated control server (300-2). For example, the facility control information (50-2) can be configured so that the integrated control server (300-2) controls the emergency power grid and emergency lighting devices in a space where a power outage has occurred. In addition, the facility control information (50-2) can be configured so that the integrated control server (300-2) transmits user terminal control information (60-2) to the user terminal (100-2).

[0552] As illustrated in FIG. 17d, there are cases where the integrated control server (300-2) receives facility control information (50-2) from the head device (230-2). At this time, the integrated control server (300-2) can control the emergency power grid so that the emergency power grid located in the space where the power outage occurred is activated. In addition, the integrated control server (300-2) can control an emergency light device to output light that can indicate the path to the emergency exit in the space where the power outage occurred, or voice that can guide the path to the emergency exit.

[0553] As illustrated in FIG. 17e, when the integrated control server (300-2) receives facility control information (50-2) from the head device (230-2), the integrated control server (300-2) can generate user terminal control information (60-2) based on the facility control information (50-2) and transmit the user terminal control information (60-2) to the user terminal (100-2). In addition, the user terminal (100-2) can receive the user terminal control information (60-2) and output the user terminal control information (60-2). For example, the user terminal (100-2) can display a user interface that can indicate the occurrence of a power outage within the on-site facility (500-2) and the shortest path from the user's location to the emergency exit.

[0554] According to one embodiment of the present invention, the digital twin system (400-2) receives facility control information (50-2) from the integrated control server (300-2), and generates a digital twin model of the field facility (500-2) based on the facility control information (50-2) and spatial information of the field facility (500-2) and displays the same. For example, when the digital twin system (400-2) receives facility control information (50-2), the digital twin system (400-2) can generate and display a digital twin model of the field facility (500-2) that includes information on the occurrence of a power outage in the field facility (500-2), the location of the power outage, the number of people at the location of the power outage, and whether the emergency power grid is in operation.

[0555] Another example of a field facility control infrastructure leveraging edge AI.

[0556] FIGS. 18a to 18f are drawings illustrating another embodiment of a field facility control infrastructure utilizing edge AI according to the present invention.

[0557] Specifically, FIGS. 18a to 18f are drawings illustrating an example of an on-site facility control infrastructure (10-2) related to a rapid response to a crime occurring on an emergency staircase within an on-site facility (500-2).

[0558] As illustrated in FIG. 18a, the integrated control server (300-2) can receive personal information (30-2) from the user terminal (100-2) of the first user using the on-site facility (500-2) and store the personal information (30-2). Specifically, the personal information (30-2) may refer to information related to one or more of the name, age, address, mobile phone number, service reservation details, and consent to the use of personal information of the first user using the on-site facility (500-2).

[0559] As illustrated in FIG. 18b, there may be a plurality of node devices (220-2). For example, the node devices (220-2) may include a first node device (220a-2), a second node device (220b-2), a third node device (220c-2), a fourth node device (220d-2), and a fifth node device (220e-2). Here, the second node device (220b-2) may receive location information (20-2) from the user terminal (100-2) of the first user when the first user is located on an emergency staircase within the field facility (500-2). In addition, the head device (230-2) may receive location information (20-2) from the second node device (220b-2).

[0560] As illustrated in FIG. 18c, the head device (230-2) can identify the personal information (30-2) transmitted by the user terminal (100-2) of the first user, from which the location information (20-2) has been received, to the integrated control server (300-2) through the integrated control server (300-2). Specifically, the head device (230-2) can integrate the location information (20-2) and the personal information (30-2) to obtain information related to at least one of the name, age, address, mobile phone number, service reservation details, and consent to the use of personal information of the first user located at the point included in the location information (20-2). For example, the head device (230-2) can obtain information that the first user using the on-site facility (500-2) is located on the emergency stairs within the on-site facility (500-2).

[0561] As illustrated in FIG. 18d, when a scream occurs in an emergency staircase within a field facility (500-2), a second node device (220b-2) located near the point where the scream occurred can sense facility environment information (40-2) indicating that the scream occurred at a specific location on the emergency staircase within the field facility (500-2). At this time, the sensor unit (not illustrated) of the second node device (220b-2) may be an acoustic sensor, but is not limited thereto. Meanwhile, the head device (230-2) can receive facility environment information (40-2) indicating that a scream occurred at a specific location on the emergency staircase within the field facility (500-2) from the second node device (220b-2).

[0562] As illustrated in FIG. 18e, the head device (230-2) can generate facility control information (50-2) by using location information (20-2), personal information (30-2), and facility environment information (40-2) as inputs to the artificial intelligence model (234-2), and transmit the facility control information (50-2) to the integrated control server (300-2). For example, the facility control information (50-2) can be configured to allow the integrated control server (300-2) to control a speaker device located in a space where a screaming sound occurred.

[0563] As illustrated in FIG. 18f, when the integrated control server (300-2) receives facility control information (50-2) from the head device (230-2), the integrated control server (300-2) can control the speaker device located in the space where the screaming sound occurred to generate a certain sound. For example, the speaker device can output the sound “Step back” to cause the second user who caused the screaming sound of the first user to move away from the first user.

[0564] According to one embodiment of the present invention, the digital twin system (400-2) receives facility control information (50-2) from the integrated control server (300-2), and generates a digital twin model of the field facility (500-2) based on the facility control information (50-2) and spatial information of the field facility (500-2) and displays the same. For example, when the digital twin system (400-2) receives the facility control information (50-2), the digital twin system (400-2) can generate and display a digital twin model of the field facility (500-2) that includes information about the occurrence of a scream on an emergency staircase within the field facility (500-2), the location of the scream, and the first user who generated the scream.

[0565] According to one embodiment of the present invention, the manager of the on-site facility (500-2) can remotely detect a user's scream on the emergency stairs within the on-site facility (500-2) via the digital twin system (400-2). Accordingly, the manager can take immediate action, such as dispatching safety personnel or reporting the incident to the police.

[0566] Another example of a field facility control infrastructure leveraging edge AI.

[0567] FIGS. 19a to 19f are drawings illustrating another embodiment of a field facility control infrastructure utilizing edge AI according to the present invention.

[0568] Specifically, FIGS. 19a to 19f are drawings illustrating an example of an on-site facility control infrastructure related to early prevention of crime occurring in an emergency staircase within an on-site facility (500-2).

[0569] As illustrated in FIG. 19a, the integrated control server (300-2) may receive personal information (30a-2) from the user terminal (100a-2) of the first user using the on-site facility (500-2) and store the personal information (30a-2). Specifically, the personal information (30a-2) may refer to information related to one or more of the name, age, address, mobile phone number, service reservation details, and consent to the use of personal information of the first user using the on-site facility (500-2). Meanwhile, if the second user does not input personal information (300b-2) related to the use of the on-site facility (500-2) through the user terminal (100b-2), the integrated control server (300-2) may not receive personal information (300b-2) from the user terminal (100-2) of the second user.

[0570] As illustrated in FIG. 19b, there may be a plurality of node devices (220-2). For example, the node devices (220-2) may include a first node device (220a-2), a second node device (220b-2), a third node device (220c-2), a fourth node device (220d-2), and a fifth node device (220e-2). Here, the first node device (220a-2) may receive location information (20a-2) from the user terminal (100a-2) of the first user when the first user enters the emergency stairs in the on-site facility (500-2). In addition, the first node device (220a-2) may receive location information (20b-2) from the user terminal (100b-2) of the second user when the second user approaches the emergency stairs entrance in the on-site facility (500-2). Additionally, the head device (230-2) can receive location information (20a-2, 20b-2) from the first node device (220a-2).

[0571] As illustrated in FIG. 19c, the head device (230-2) can identify personal information (30a-2, 30b-2) transmitted to the integrated control server (300-2) by the user terminal (100a-2, 100b-2) that received the location information (20a-2, 20b-2) through the integrated control server (300-2). Specifically, the head device (230-2) can integrate the location information (20a-2) and the personal information (30a-2) to obtain information related to at least one of the name, age, address, mobile phone number, service reservation details, and consent to use of personal information of the first user located at the point included in the location information (20a-2). For example, the head device (230-2) can obtain information that the first user using the on-site facility (500-2) entered the emergency stairs within the on-site facility (500-2). Additionally, the head device (230-2) can integrate location information (20b-2) and personal information (30b-2) to obtain information about a second user located at a location included in the location information (20b-2). For example, the head device (230-2) can obtain information that a second user who does not use the on-site facility (500-2) is following the first user and approaching the emergency staircase entrance within the on-site facility (500-2).

[0572] As illustrated in FIG. 19d, the head device (230-2) can generate facility control information (50-2) using location information (20a-2, 20b-2) and personal information (30a-2, 30b-2) as inputs to the artificial intelligence model (234-2), and transmit the facility control information (50-2) to the integrated control server (300-2). For example, the facility control information (50-2) can be configured so that the integrated control server (300-2) controls a speaker device located in a space where the first user is located within the emergency stairs. In addition, the facility control information (50-2) can be configured so that the integrated control server (300-2) transmits user terminal control information (60-2) to the user terminal (100-2).

[0573] As illustrated in FIG. 19e, there are cases where the integrated control server (300-2) receives facility control information (50-2) from the head device (230-2). At this time, the integrated control server (300-2) can control the speaker device located in the space where the first user is located within the emergency stairs to generate a certain sound. For example, the speaker device can output the sound "Please step back" to cause the second user, who is attempting to enter following the first user, to move away from the first user.

[0574] As illustrated in FIG. 19f, there are cases where the integrated control server (300-2) receives facility control information (50-2) from the head device (230-2). At this time, the integrated control server (300-2) can generate user terminal control information (60-2) based on the facility control information (50-2) and transmit the user terminal control information (60-2) to the user terminal (100a-2) of the first user. The user terminal (100a-2) can receive the user terminal control information (60-2) and output the user terminal control information (60-2). For example, the user terminal (100a-2) can display a user interface that can indicate that a second user is following the first user and is attempting to enter the emergency stairs.

[0575] According to one embodiment of the present invention, the digital twin system (400-2) receives facility control information (50-2) from the integrated control server (300-2), and generates a digital twin model of the field facility (500-2) based on the facility control information (50-2) and spatial information of the field facility (500-2) and displays the same. For example, when the digital twin system (400-2) receives the facility control information (50-2), the digital twin system (400-2) can generate and display a digital twin model of the field facility (500-2) that includes the locations of the first user and the second user, the movement paths of the first user and the second user, and information about the second user whose personal information (30b-2) is not confirmed.

[0576] According to one embodiment of the present invention, the manager of the on-site facility (500-2) can remotely detect, via the digital twin system (400-2), that an unregistered person has followed a facility user into the emergency stairs of the on-site facility (500-2). Accordingly, the manager can take immediate action, such as dispatching security personnel or reporting the incident to the police.

[0577] Another example of a field facility control infrastructure leveraging edge AI.

[0578] FIGS. 20A to 20E are drawings illustrating another embodiment of a field facility control infrastructure utilizing edge AI according to the present invention.

[0579] Specifically, FIGS. 20a to 20e are drawings illustrating one embodiment of a field facility control infrastructure (10-2) related to an automatic payment system.

[0580] As illustrated in FIG. 20a, the integrated control server (300-2) may receive personal information (30-2) from the user terminal (100-2) of a user who has made a reservation at a restaurant within the on-site facility (500-2) and store the personal information (30-2). Specifically, the personal information (30-2) may refer to information related to one or more of the name, age, address, mobile phone number, service reservation details, and consent to the use of personal information of the user who has made the reservation.

[0581] As illustrated in FIG. 20b, there may be a plurality of node devices (220-2). For example, the node devices (220-2) may include a first node device (220a-2), a second node device (220b-2), and a third node device (220c-2). Here, the first node device (220a-2) and the second node device (220b-2) may receive location information (20-2, 20'-2) from the user terminal (100-2) of the user when the user finishes eating and moves from inside the restaurant to outside the restaurant. In addition, the head device (230-2) may receive location information (20-2, 20'-2) from the first node device (220a-2) and the second node device (220b-2).

[0582] As illustrated in FIG. 20c, the head device (230-2) can identify personal information (30-2) transmitted by the user terminal (100-2) that received the location information (20-2, 20'-2) to the integrated control server (300-2) through the integrated control server (300-2). Specifically, the head device (230-2) can integrate the location information (20-2, 20'-2) and the personal information (30-2) to obtain information related to at least one of the name, age, address, mobile phone number, service reservation details, and consent to the use of personal information of the user located at the location included in the location information (20-2, 20'-2). For example, the head device (230-2) can obtain information that a user who made a reservation at a restaurant has finished eating and moved from inside the restaurant to outside the restaurant.

[0583] As illustrated in FIG. 20d, the head device (230-2) can generate facility control information (50-2) by using location information (20-2) and personal information (30-2) as inputs to an artificial intelligence model (234-2), and transmit the facility control information (50-2) to an integrated control server (300-2). For example, the facility control information (50-2) can be configured so that the integrated control server (300-2) transmits user terminal control information (60-2) to the user terminal (100-2).

[0584] As illustrated in FIG. 20e, there are cases where the integrated control server (300-2) receives facility control information (50-2) from the head device (230-2). At this time, the integrated control server (300-2) can generate user terminal control information (60-2) based on the facility control information (50-2) and transmit the user terminal control information (60-2) to the user terminal (100-2). The user terminal (100-2) can receive the user terminal control information (60-2) and output the user terminal control information (60-2). For example, the user terminal (100-2) can display a user interface that displays a payment window for a restaurant where the user has made a reservation.

[0585] According to one embodiment of the present invention, the digital twin system (400-2) receives facility control information (50-2) from the integrated control server (300-2), and generates a digital twin model of the on-site facility (500-2) based on the facility control information (50-2) and spatial information of the on-site facility (500-2) and displays the same. For example, when the digital twin system (400-2) receives the facility control information (50-2), the digital twin system (400-2) can generate and display a digital twin model of the on-site facility (500-2) that includes information on the number of people inside the restaurant, the location of at least one user who made a reservation at the restaurant, movement path and personal information, and whether at least one user who made a reservation at the restaurant made a payment.

[0586] Another example of a field facility control infrastructure leveraging edge AI.

[0587] FIGS. 21A to 21K are drawings illustrating another embodiment of a field facility control infrastructure utilizing edge AI according to the present invention.

[0588] Specifically, FIGS. 21a to 21k are drawings illustrating one embodiment of a field facility control infrastructure (10-2) related to conference room management within a field facility (500-2).

[0589] As illustrated in FIG. 21a, the integrated control server (300-2) may receive personal information (30a-2, 30b-2, 30c-2) from the user terminals (100a-2, 100b-2, 100c-2) of the first user, second user, and third user who have reserved use of a conference room within the on-site facility (500-2), and store the personal information (30a-2, 30b-2, 30c-2). Specifically, the personal information (30a-2, 30b-2, 30c-2) may refer to information related to one or more of the name, age, address, mobile phone number, service reservation details, and consent to the use of personal information of the user who has reserved the service.

[0590] As illustrated in FIG. 21b, there may be a plurality of at least one node device (220-2). For example, at least one node device (220-2) may include a first node device (220a-2), a second node device (220b-2), and a third node device (220c-2). Here, the third node device (220c-2) may receive location information (20a-2, 20b-2, 20c-2) from the user terminal (100a-2, 100b-2, 100c-2) of the user when the user is located near the entrance door of the conference room within the on-site facility (500-2). In addition, the head device (230-2) may receive location information (20a-2, 20b-2, 20c-2) from the third node device (220c-2).

[0591] As illustrated in FIG. 21c, the head device (230-2) can identify personal information (30a-2, 30b-2, 30c-2) transmitted to the integrated control server (300-2) by the user terminal (100a-2, 100b-2, 100c-2) from which location information (20a-2, 20b-2, 20c-2) was received through the integrated control server (300-2). Specifically, the head device (230-2) can obtain information related to at least one of the name, age, address, mobile phone number, service reservation details, and consent to use of personal information of a user located at a location included in the location information (20a-2, 20b-2, 20c-2) by integrating location information (20a-2, 20b-2, 20c-2) and personal information (30a-2, 30b-2, 30c-2). For example, the head device (230-2) can obtain information that a first user, a second user, and a third user who have reserved use of a conference room within an on-site facility (500-2) have arrived near the entrance of the conference room.

[0592] As illustrated in FIG. 21d, the head device (230-2) can generate facility control information (50-2) by using location information (20a-2, 20b-2, 20c-2) and personal information (30a-2, 30b-2, 30c-2) as inputs to the artificial intelligence model (234-2), and transmit the facility control information (50-2) to the integrated control server (300-2). For example, the facility control information (50-2) can be configured to allow the integrated control server (300-2) to control the conference room's entrance door, conference room screen, lights, air purifier, and beam projector. Additionally, facility control information (50-2) can be configured so that the integrated control server (300-2) transmits user terminal control information (60-2) to user terminals (100a-2, 100b-2, 100c-2).

[0593] As illustrated in FIG. 21e, when the integrated control server (300-2) receives facility control information (50-2) from the head device (230-2), the integrated control server (300-2) can control the entrance door of the conference room to open it. In addition, the integrated control server (300-2) can control the screen, lights, air purifier, and beam projector in the conference room to turn them on.

[0594] As illustrated in FIG. 21f, there are cases where the integrated control server (300-2) receives facility control information (50-2) from the head device (230-2). At this time, the integrated control server (300-2) can generate user terminal control information (60-2) based on the facility control information (50-2) and transmit the user terminal control information (60-2) to the user terminals (100a-2, 100b-2, 100c-2).

[0595] According to the present invention, user terminals (100a-2, 100b-2, 100c-2) can receive user terminal control information (60-2) and output user terminal control information (60-2). For example, user terminals (100a-2, 100b-2, 100c-2) can display a user interface indicating how to use a conference room. In addition, when the first user, the second user, and the third user who have reserved the use of the conference room are all located within the conference room, the user terminals (100a-2, 100b-2, 100c-2) can display a user interface indicating that all conference participants are present.

[0596] As illustrated in FIG. 21g, if the air quality inside the conference room deteriorates during a meeting, the first node device (220a-2) can sense facility environment information (40-2) regarding the air quality inside the conference room. At this time, the sensor unit (not shown) of the first node device (220a-2) may be an air sensor, but is not limited thereto. Meanwhile, the head device (230-2) can receive facility environment information (40-2) regarding the air quality inside the conference room from the first node device (220a-2).

[0597] As illustrated in FIG. 21h, the head device (230-2) may generate facility control information (50-2) by using location information (20a-2, 20b-2, 20c-2) and personal information (30a-2, 30b-2, 30c-2) and facility environment information (40-2) for the first user, the second user, and the third user as inputs to the artificial intelligence model (234-2), and transmit the facility control information (50-2) to the integrated control server (300-2). For example, the facility control information (50-2) may be configured to allow the integrated control server (300-2) to control an air purifier in a conference room.

[0598] As illustrated in FIG. 21h, when the integrated control server (300-2) receives facility control information (50-2) from the head device (230-2), the integrated control server (300-2) can control the air purifier in the conference room to operate at a higher intensity.

[0599] As illustrated in FIG. 21i, the first node device (220a-2) can receive location information (20a-2, 20b-2, 20c-2) from the user terminals (100a-2, 100b-2, 100c-2) when the first user, the second user, and the third user are located near the conference room door to exit the conference room after finishing the conference. In addition, the head device (230-2) can receive location information (20a-2, 20b-2, 20c-2) from the first node device (220a-2).

[0600] As illustrated in FIG. 21i, the head device (230-2) can generate facility control information (50-2) by using location information (20a-2, 20b-2, 20c-2) and personal information (30a-2, 30b-2, 30c-2) as inputs to the artificial intelligence model (234-2), and transmit the facility control information (50-2) to the integrated control server (300-2). For example, the facility control information (50-2) can be configured to allow the integrated control server (300-2) to control the conference room entrance door.

[0601] As illustrated in FIG. 21i, when the integrated control server (300-2) receives facility control information (50-2) from the head device (230-2), the integrated control server (300-2) can control the conference room door so that the conference room door is opened.

[0602] As illustrated in FIG. 21j, when the first user, the second user, and the third user of the third node device (220c-2) go outside the conference room, location information (20a-2, 20b-2, 20c-2) can be received from the user terminals (100a-2, 100b-2, 100c-2). In addition, the head device (230-2) can receive location information (20a-2, 20b-2, 20c-2) from the third node device (220c-2).

[0603] As illustrated in FIG. 21j, the head device (230-2) can generate facility control information (50-2) by using location information (20a-2, 20b-2, 20c-2) and personal information (30a-2, 30b-2, 30c-2) as inputs to the artificial intelligence model (234-2), and transmit the facility control information (50-2) to the integrated control server (300-2). For example, the facility control information (50-2) can be configured to allow the integrated control server (300-2) to control a screen, lights, air purifier, and beam projector in a conference room.

[0604] As illustrated in FIG. 21j, when the integrated control server (300-2) receives facility control information (50-2) from the head device (230-2), the integrated control server (300-2) can control the screen, lights, air purifier, and beam projector in the conference room to turn off the screen, lights, air purifier, and beam projector.

[0605] As illustrated in FIG. 21k, when the expiration of the conference room reservation times of the first user, the second user, and the third user is imminent during the conference, the head device (230-2) may generate facility control information (50-2) by using location information (20a-2, 20b-2, 20c-2) and personal information (30a-2, 30b-2, 30c-2) as inputs to the artificial intelligence model (234-2), and transmit the facility control information (50-2) to the integrated control server (300-2). For example, the facility control information (50-2) may be configured so that the integrated control server (300-2) transmits user terminal control information (60-2) to the user terminals (100a-2, 100b-2, 100c-2). Here, the head device (230-2) can obtain information about the user's conference room reservation time and the remaining time until the reservation time expires through the user's personal information (30a-2, 30b-2, 30c-2). In addition, the head device (230-2) can obtain information that the user is still in the middle of a meeting through the user's location information (20a-2, 20b-2, 20c-2).

[0606] As illustrated in FIG. 21k, when the integrated control server (300-2) receives facility control information (50-2) from the head device (230-2), the integrated control server (300-2) can generate user terminal control information (60-2) based on the facility control information (50-2) and transmit the user terminal control information (60-2) to the user terminals (100a-2, 100b-2, 100c-2). The user terminals (100a-2, 100b-2, 100c-2) can receive the user terminal control information (60-2) and output the user terminal control information (60-2). For example, the user terminals (100a-2, 100b-2, 100c-2) can display a user interface that indicates a method for adding a conference room reservation time.

[0607] According to one embodiment of the present invention, the digital twin system (400-2) receives facility control information (50-2) from the integrated control server (300-2), and generates a digital twin model of the on-site facility (500-2) based on the facility control information (50-2) and the spatial information of the on-site facility (500-2) and displays the same. For example, there is a case where the digital twin system (400-2) receives facility control information (50-2). At this time, the digital twin system (400-2) can generate and display a digital twin model of the on-site facility (500-2) that includes real-time location and personal information of people who have reserved a conference room, the number of people in the conference room, whether devices in the conference room are running, and whether the conference room door is open or closed.

[0608] Meanwhile, the specific terms, numbers, field facility (500-2) and user terminal (100-2) control methods described above are merely examples described to explain the field facility control infrastructure (10-2) according to one embodiment of the present invention, and the present invention is not limited thereto.

[0609] In addition, various modified examples of the field facility control infrastructure (10-2) and the field facility control method (70-2) according to one embodiment of the present invention can be utilized for various facilities of a large-scale complex facility that are not described in detail by this specification. For example, the various facilities of the large-scale complex facility can include an exhibition hall, a restaurant, a clothing store, a sports facility, a convenience store, a grocery store, a food court, a movie theater, a gym, a swimming pool, a spa, a beauty salon, a hospital, a pharmacy, a bank, a cafe, a restaurant, a hotel, a performance hall, a library, an academy, an amusement park, a game center, a golf course, a tennis court, a large bookstore, a bakery, a park, a walking path, a convention center, a daycare center, an elementary school, a middle school, a high school, a university, a museum, an art gallery, a zoo, an aquarium, a garden, a baseball field, a soccer field, a skating rink, a bowling alley, a club, a bar, a karaoke room, an electric vehicle charging station, etc.

[0610] Configuration of an edge AI-based vehicle parking management system

[0611] Figure 22 is a block diagram showing the configuration of an edge AI-based vehicle parking management system (100-3) according to the present invention.

[0612] According to the present invention, the edge AI-based vehicle parking management system (100-3) may include at least one electronic device (110-3). While FIG. 22 illustrates only two electronic devices (110-1 and 110-2), the present invention is not limited thereto.

[0613] According to the present invention, at least one electronic device (110-3) may include a sensor unit (111-3), a network unit (112-3), a memory unit (113-3), and a processor unit (114-3).

[0614] Figure 23 is a block diagram illustrating another configuration of an edge AI-based vehicle parking management system (100-3) according to the present invention. As illustrated, at least one electronic device (110-3) may further include an output unit (115-3) in addition to a sensor unit (111-3), a network unit (112-3), a memory unit (113-3), and a processor unit (114-3).

[0615] According to the present invention, the sensor unit (111-3) may be composed of a single sensor, but is not limited thereto, and may also be composed of multiple sensors. Specifically, the sensor unit (111-3) may include a camera sensor, an acoustic sensor, an infrared sensor, etc. In particular, the sensor unit (111-3) may be a visible light camera sensor or an infrared camera sensor installed to capture a specific area.

[0616] According to the present invention, the sensor unit (111-3) may be configured as a sensor that performs the functions of both a visible light camera and an infrared camera. Furthermore, the sensor unit (111-3) may be configured as a camera sensor that captures sequential images in time or repeatedly captures still images at regular intervals.

[0617] According to the present invention, the sensor unit (111-3) can acquire first image information (10-3) including a vehicle (200-3) and / or a parking lot (300-3). Specifically, the first image information (10-3) can include a shape of the vehicle (200-3) as viewed from all sides and / or a license plate of the vehicle (200-3), and can include a structure of the parking lot (300-3) and / or a parking space.

[0618] Figure 28 is a conceptual diagram showing information transmission and reception between at least one electronic device (110-3) according to the present invention.

[0619] According to the present invention, at least one electronic device (110-3) can transmit and receive information to and from each other via a network unit (112-3). Specifically, the network unit (112-3) can perform wired and / or wireless communication with an external device, and the wireless communication can include at least one of infrared communication, RF, Zigbee, and Bluetooth.

[0620] According to the present invention, the network unit (112-3) can receive second image information (20-3) from at least one other electronic device (110-3). Specifically, the network unit (112-3) can transmit at least one of first image information (10-3), parking-related information (30-3), route information (40-3), and parking fee information (50-3) to at least one other electronic device (110-3).

[0621] According to the present invention, the memory unit (113-3) may be a volatile memory or a non-volatile memory, and may be referred to as a 'database', a 'storage unit', etc. Specifically, the memory unit (113-3) may store at least one of data acquired by the sensor unit (111-3), data transmitted by the network unit (112-3) from another electronic device, and data processed by the processor unit (114-3).

[0622] According to the present invention, the memory unit (113-3) may store the code of a program executed by the processor unit (114-3), or store intermediate results and temporary data generated while the processor unit (114-3) processes the data. Specifically, an edge AI model may be implemented in the memory unit (113-3).

[0623] According to the present invention, the memory unit (113-3) can store at least one of the first image information (10-3) acquired by the sensor unit (111-3), the second image information (20-3) received by the network unit (112-3), the parking-related information (30-3), the route information (40-3) and the parking fee information (50-3) processed by the processor unit (114-3).

[0624] According to the present invention, the processor unit (114-3) can control at least one electronic device (110-3). The processor unit (114-3) can interpret commands, participate in data movement, perform operations, and execute code stored in the memory unit (113-3).

[0625] According to the present invention, the processor unit (114-3) can directly process data acquired by the sensor unit (111-3), execute image processing-based artificial intelligence algorithms, and execute edge AI models stored in the memory unit (113-3). Therefore, edge AI technology is implemented, significantly improving data processing speed, data security, and data protection.

[0626] According to the present invention, the processor unit (114-3) can process parking-related information (30-3) using the first image information (10-3) and / or the second image information (20-3) as inputs to the edge AI model. Specifically, the processor unit (114-3) can generate route information (40-3) of the vehicle (200-3) based on the parking-related information (30-3). In addition, the processor unit (114-3) can generate parking fee information (50-3) of the vehicle (200-3) based on the parking-related information (30-3).

[0627] According to the present invention, the output unit (115-3) may be an audio reproduction device or a device that displays visual information. In addition, the output unit (115-3) may include both the audio reproduction device and the device that displays visual information.

[0628] According to the present invention, the audio reproduction device can convert an electrical signal into an acoustic signal and output sound. Specifically, at least one of parking-related information (30-3), route information (40-3), and parking fee information (50-3) generated by the processor unit (114-3) can be output in the form of a voice, warning sound, and / or sound effect.

[0629] According to the present invention, the device for displaying the visual information may be referred to as a 'display', an 'electric board', etc. In addition, the device for displaying the visual information may be configured in an LED (Light Emitting Diode) manner and / or an LCD (Liquid Crystal Display) manner. Specifically, the device for displaying the start information may output at least one of first image information (10-3), second image information (20-3), parking-related information (30-3), route information (40-3), and parking fee information (50-3) in a visual form.

[0630] Operation of an edge AI-based vehicle parking management system

[0631] Figure 24 is an exemplary diagram for explaining the operation of an edge AI-based vehicle parking management system (100-3) according to the present invention.

[0632] According to the present invention, the sensor unit (111-3) of at least one electronic device (110-3) can detect a situation in which a vehicle (200-3) has entered a parking lot (300-3). The sensor unit (111-3) may be directed only toward the vehicle (200-3), only toward a portion of the parking lot (300-3), or toward both the vehicle (200-3) and a portion of the parking lot (300-3).

[0633] According to the present invention, when a vehicle (200-3) enters a parking lot (300-3) in which an edge AI-based vehicle parking management system (100-3) is installed, the sensor unit (111-3) can obtain first image information (10-3).

[0634] According to the present invention, the network unit (112-3) can receive second image information (20-3) from at least one other electronic device (110-3). The second image information (20-3) may be acquired by the sensor unit (111-3) of at least one other electronic device (110-3) in the same manner as the first image information (10-3) is acquired.

[0635] According to the present invention, the memory unit (113-3) can store the first image information (10-3) and / or the second image information (20-3) so that the first image information (10-3) and / or the second image information (20-3) can be processed by the processor unit (114-3).

[0636] According to the present invention, the processor unit (114-3) can process parking-related information (30-3) based on the first image information (10-3) and / or the second image information (20-3). At this time, the parking-related information (30-3) can be processed using the first image information (10-3) and / or the second image information (20-3) as input to the edge AI model.

[0637] According to the present invention, the edge AI model may include an artificial intelligence model based on image processing, and the processor unit (114-3) may recognize the vehicle (200-3) from a view of the vehicle (200-3) from all sides through the artificial intelligence model based on image processing, and recognize the identifier of the vehicle (200-3) from the license plate of the vehicle.

[0638] According to the present invention, the processor unit (114-3) can recognize the structure of the parking lot (300-3) through an image processing-based artificial intelligence model, and can recognize individual parking spaces (310-3, 320-3, 330-3) included in the parking lot (300-3). Specifically, the processor unit (114-3) can determine whether a vehicle is parked in an individual parking space (320-3) through an image processing-based artificial intelligence model.

[0639] According to the present invention, parking-related information (30-3) may include at least one of real-time location information (31-3) of a vehicle (200-3), target parking space information (32-3) of a vehicle (200-3), identifier information (33-3) of a vehicle (200-3), structure information (34-3) of a parking lot (300-3), entry time information (35-3) of a vehicle (200-3), exit time information (36-3) of a vehicle (200-3), and parking time information (37-3) of a vehicle (200-3).

[0640] According to the present invention, real-time location information (31-3) may be information obtained in real time regarding the coordinates of the space where the vehicle (200-3) is located within the parking lot (300-3). Specifically, since the processor unit (114-3) obtains information in real time, the coordinates of the vehicle (200-3) can be immediately obtained even when the vehicle (200-3) is moving.

[0641] According to the present invention, target parking space information (32-3) may be information regarding the parking space closest to the vehicle (200-3) or a reserved parking space. Specifically, a reserved parking space may refer to a parking space that the driver of the vehicle (200-3) has previously reserved for parking.

[0642] According to the present invention, the identifier information (33-3) of the vehicle (200-3) may be a string including numbers and / or letters recognized from the license plate of the vehicle (200-3). In addition, the structural information (34-3) of the parking lot (300-3) may be information regarding the location, number, and size of individual parking spaces (310-3, 320-3, 330-3) included in the parking lot (300-3).

[0643] Figure 25 is an exemplary diagram for explaining the operation and special parking area of ​​an edge AI-based vehicle parking management system (100-3) according to the present invention.

[0644] As illustrated in Figure 25, the parking lot (300-3) may include special areas such as a no-parking zone (340-3), a disabled parking zone (350), a women-only parking zone (not shown), and a fire truck access zone (not shown). Accordingly, since it is necessary to provide guidance to the driver of the vehicle (200-3), the structure information (34-3) of the parking lot (300-3) may include information regarding the special areas.

[0645] According to the present invention, when the target parking space information (32-3) is the parking space closest to the vehicle (200-3), the closest parking space may be a parking space excluding the special zone.

[0646] According to the present invention, the entry time information (35-3) may be information regarding the time at which the vehicle (200-3) entered the parking lot (300-3). Furthermore, the exit time information (36-3) may be information regarding the time at which the vehicle (200-3) exited the parking lot (300-3). Furthermore, the parking time information (37-3) may be information regarding the actual time at which the vehicle (200-3) was parked.

[0647] According to the present invention, when processing parking-related information (30-3), the processor unit (114-3) can determine whether the vehicle (200-3) is parked in a parking space included in the target parking space information (32-3) through an image processing-based artificial intelligence model. Accordingly, the parking time information (37-3) may be a measurement of the time from the time the vehicle (200-3) actually parked in the parking space to the time it left the parking space.

[0648] According to the present invention, the processor unit (114-3) can generate route information (40-3) of the vehicle (200-3) based on parking-related information (30-3).

[0649] According to the present invention, path information (40-3) can be obtained from real-time location information (31-3), target parking space information (32-3), and structure information (34-3) of a parking lot (300-3). Specifically, it can be information regarding a vehicle movement path from the coordinates of a vehicle (200-3) obtained through real-time location information (31-3) to the coordinates of a parking space included in the target parking space information (32-3).

[0650] According to the present invention, the processor unit (114-3) can generate parking fee information (50-3) of a vehicle (200-3) based on parking-related information (30-3).

[0651] According to the present invention, parking fee information (50-3) can be obtained from at least one of entry time information (35-3), exit time information (36-3), and parking time information (37-3). Specifically, the amount may be the amount calculated by multiplying the time from the entry time to the exit time by the hourly fee, or the amount calculated by multiplying the parking time by the hourly fee. However, the above amounts are provided as examples of parking fee calculation and are not limited to the above-described calculation method.

[0652] According to the present invention, parking fee information (50-3) may include information on differential amounts considering parking space preference, distance from building entrances and exits, distance from parking lot entrances and exits, presence of shade, etc., and / or information on total parking fees.

[0653] According to the present invention, route information (40-3) and parking fee information (50-3) need to be easily provided to the driver of the vehicle (200-3). Therefore, the route information (40-3) and parking fee information (50-3) may be provided in a form that can be provided through a user terminal, such as a smartphone, tablet, or AR device, carried by the driver of the vehicle (200-3).

[0654] Figure 29 is a drawing showing path information (40-3) according to the present invention.

[0655] According to the present invention, route information (40-3) is provided to the driver of the vehicle (200-3) via a smartphone. In addition, the route information (40-3) may be provided in a simplified form as structural information (34-3) of the parking lot (300-3), or may be provided as a background for at least a portion of the first image information (10-3) and / or the second image information (20-3).

[0656] According to the present invention, route information (40-3) can display the current location and destination of the vehicle in graphics and / or text, and can display the vehicle's movement path using arrows. However, FIG. 29 and the above description are merely examples of providing route information (40-3), and are not limited thereto.

[0657] Figure 30 is a drawing showing parking fee information (50-3) according to the present invention.

[0658] According to the present invention, parking fee information (50-3) is provided to the driver of a vehicle (200-3) via a smartphone. Specifically, the parking fee information (50-3) may include information regarding parking start time, parking end time, differential fee, and total fee.

[0659] According to the present invention, parking fee information (50-3) may include information regarding a button that connects to a payment service so that the driver of the vehicle (200-3) can immediately pay the total fee. However, FIG. 30 and the above description are merely examples of the provision of parking fee information (50-3), and are not limited thereto.

[0660] Figure 26 is an exemplary diagram illustrating the operation of an edge AI-based vehicle parking management system (100-3) according to the present invention and the role of an output unit (115-3). According to the present invention, at least one electronic device (110-3) included in the edge AI-based vehicle parking management system (100-3) may further include an output unit (115-3).

[0661] According to the present invention, the output unit (115-3) may be an audio reproduction device or a device that displays visual information. Alternatively, it may include both the audio reproduction device and the device that displays visual information.

[0662] According to the present invention, the audio reproduction device can output audio signals in the form of voices, warning sounds, and / or sound effects. For example, when a vehicle enters a parking space, the audio reproduction device can output voices and / or sound effects such as "The vehicle has entered the parking space," and when a reserved vehicle enters the parking space, "The reserved vehicle has entered the parking space." Furthermore, when a vehicle is parked in a no-parking zone, the audio reproduction device can output voices and / or warning sounds such as "Please move to another area."

[0663] According to the present invention, the audio reproduction device can output route information (40-3) in a voice such as, "Go straight for 30m and then turn left." In addition, the audio reproduction device can output parking fee information (50-3) in a voice such as, "The total parking fee is 13,000 won." In addition, the audio reproduction device can output a voice such as, "Goodbye" when the vehicle exits the parking lot. However, the above-described is merely an example for explaining the role of the output unit (115-3) and is not limited thereto.

[0664] According to the present invention, the device for displaying the visual information can visually display at least one of the first image information (10-3), the second image information (20-3), the parking-related information (30-3), the route information (40-3), and the parking fee information (50-3). For example, the device for displaying the visual information can display the vehicle (200-3) included in the first image information (10-3) as viewed from all sides or the identifier information (33-3) of the vehicle (200-3) when a vehicle enters the parking lot. In addition, the device for displaying the visual information can display information about the reserved parking space when a reserved vehicle enters the parking lot, information indicating that parking is not possible when the vehicle is parked in a no-parking zone (340-3), route information (40-3), parking fee information (50-3) when the vehicle exits the parking lot, etc. However, the above is merely an example for explaining the role of the output unit (115-3) and is not limited thereto.

[0665] Edge AI-based vehicle parking management method

[0666] Figure 31 is a flowchart illustrating an edge AI-based vehicle parking management method (S100-3) according to the present invention.

[0667] According to the present invention, an edge AI-based vehicle parking management method (S100-3) may include an image information acquisition step (S110-3) of acquiring image information including a vehicle (200-3) and / or a parking lot (300-3), a vehicle recognition step (S120-3) of recognizing a vehicle (200-3) based on the acquired image information, a parking space recognition step (S130-3) of recognizing a parking space included in the parking lot (300-3) based on the acquired image information, and a parking determination step (S140-3) of determining whether a vehicle (200-3) is parked in the parking space.

[0668] According to the present invention, the edge AI-based vehicle parking management method (S100-3) may further include a parking information storage step (S150-3) of storing an identifier of the vehicle and / or an identifier of the parking space in a memory unit when the recognized vehicle is parked in the recognized parking space.

[0669] According to the present invention, the vehicle recognition step (S120-3), the parking space recognition step (S130-3), and the parking judgment step (S140-3) can be performed using the acquired image information as input to the edge AI model.

[0670] According to the present invention, the processor unit can acquire an image of a vehicle (200-3) viewed from all sides and / or an image including a vehicle license plate through the image information acquisition step (S110-3). In addition, the processor unit can recognize the vehicle and / or the vehicle identifier by using the image as input to an image processing-based artificial intelligence model. In addition, the processor unit can acquire an image of a parking lot including individual parking spaces.

[0671] Figure 27 (a) is a drawing showing the front part of a vehicle according to the present invention, and Figure 27 (b) is a drawing showing the rear part of a vehicle according to the present invention.

[0672] According to the present invention, the processor unit can identify a vehicle identifier from an image including information about the front and / or rear of the vehicle, such as in FIG. 27 (a) and / or FIG. 27 (b), through an image processing-based artificial intelligence model. For example, the vehicle identifier may be '12ga 4567'.

[0673] According to the present invention, the vehicle recognition step (S120-3) can recognize the vehicle (200-3) by using the acquired image information as input to an edge AI model, which is an image processing-based artificial intelligence model. The image processing-based artificial intelligence model can learn various shapes, colors, and sizes of the vehicle and accurately recognize the vehicle (200-3) within the image information.

[0674] According to the present invention, the parking space recognition step (S130-3) can recognize the parking space by using the acquired image information as input to an edge AI model, which is an image processing-based artificial intelligence model. The image processing-based artificial intelligence model can learn the various shapes, sizes, and locations of parking spaces and accurately recognize parking spaces within the image information.

[0675] According to the present invention, the processor unit can assign a unique identifier to the recognized parking space. The identifier of the parking space may be a string containing numbers and / or letters, such as an identifier of a vehicle (200-3).

[0676] According to the present invention, the parking determination step (S140-3) can determine whether the vehicle (200-3) recognized in the vehicle recognition step (S120-3) and the parking space recognition step (S130-3) and the parking space overlap each other. For example, the processor unit can generate a bounding box of the recognized vehicle (200-3), generate a bounding box of the recognized parking space, and compare whether the bounding box of the vehicle (200-3) and the bounding box of the parking space overlap by a predetermined area or more. Through the above-described comparison process, it can be determined whether the vehicle (200-3) is parked in the parking space. However, the above-described is merely an example for explaining the parking determination step (S140-3) and is not limited thereto.

[0677] According to the present invention, the edge AI-based vehicle parking management method (S100-3) may further include a parking information storage step (S150-3) of storing an identifier of the vehicle (200-3) and / or an identifier of the parking space in a memory unit when the vehicle (200-3) is parked in the recognized parking space. Specifically, in addition to the identifier of the vehicle (200-3) and / or the identifier of the parking space, information regarding specifications of the vehicle (200-3), the date and time the vehicle (200-3) was parked, driver information of the vehicle (200-3), the time the vehicle (200-3) entered the parking lot, and the time the vehicle (200-3) exited the parking lot may be further stored in the memory unit.

[0678] According to the present invention, by storing the above-described information in the memory, the driver of the vehicle (200-3) and the operator of the parking lot (300-3) can check various statistical information in real time, such as the degree of parking lot crowding, real-time status of parking spaces, parking preference in parking spaces, frequency of parking, time and date, etc. For example, statistics regarding at least one of the busiest time of the parking lot (300-3), the most popular parking space, the least popular parking space, and the vehicle (200-3) using the parking lot (300-3) can be provided. The above-described information is merely an example for explaining the information stored in the memory, and is not limited thereto.

[0679] Configuration of a personal mobility device parking management device based on edge AI

[0680] Figure 32 is a block diagram illustrating the configuration of an edge AI-based personal mobility device parking management device (100-4) according to the present invention. According to the present invention, the edge AI-based personal mobility device parking management device (100-4) may include a first sensor unit (110-4), a second sensor unit (120-4), a processor unit (130-4), and a memory unit (140-4).

[0681] Figure 33 is a block diagram illustrating another configuration of an edge AI-based personal mobility device parking management device (100-4) according to the present invention. According to the present invention, the edge AI-based personal mobility device parking management device (100-4) may further include an output unit (150-4) in addition to the first sensor unit (110-4), the second sensor unit (120-4), the processor unit (130-4), and the memory unit (140-4).

[0682] According to the present invention, the first sensor unit (110-4) and the second sensor unit (120-4) may each be composed of one sensor, but are not limited thereto, and may be composed of multiple sensors. Specifically, the first sensor unit (110-4) and the second sensor unit (120-4) may include a camera sensor, an acoustic sensor, an infrared sensor, and the like. In particular, the first sensor unit (110-4) and the second sensor unit (120-4) may be a visible light camera sensor and / or an infrared camera sensor installed to capture a specific area.

[0683] According to the present invention, the first sensor unit (110-4) and the second sensor unit (120-4) may include a camera sensor that captures sequential images in time series or repeatedly captures still images at regular intervals.

[0684] According to the present invention, the first sensor unit (110-4) can obtain wide-area image information (10-4) including a personal transportation means (200-4) and / or a parking lot (300-4).

[0685] According to the present invention, the personal transportation means (200-4) may refer to any of various types of transportation means, including a vehicle, an electric kickboard, a bicycle, an electric bicycle, a motorcycle, and a scooter. Furthermore, the personal transportation means (200-4) may be referred to as "personal mobility."

[0686] According to the present invention, the personal mobility device (200-4) is depicted as a vehicle in FIGS. 35 to 38, but this is merely an example for explaining the operation of the edge AI-based personal mobility device parking management device (100-4), and is not limited thereto.

[0687] According to the present invention, the wide-area image information (10-4) may include various visual data spanning a wide range in space and time. Specifically, the wide-area image information (10-4) may include the overall shape of the personal mobility device (200-4) and / or the parking lot (300-4). Accordingly, the processor unit (130-4) may recognize the overall shape of the personal mobility device (200-4) included in the wide-area image information (10-4), and determine the parking intention of the personal mobility device (200-4) by identifying the movement path, movement direction, and speed of the personal mobility device (200-4).

[0688] According to the present invention, the second sensor unit (120-4) can obtain focused image information (20-4) including a personal mobility device (200-4).

[0689] According to the present invention, the focused image information (20-4) may include various visual data spanning a narrower spatiotemporally than the wide-area image information (10-4). Specifically, the focused image information (20-4) may include an enlarged front view of the personal mobility device (200-4) or an enlarged rear view of the personal mobility device (200-4). Accordingly, the focused image information (20-4) may include information regarding the license plate (210-4) of the personal mobility device (200-4). Accordingly, the processor unit (130-4) may recognize the identifier of the personal mobility device (200-4) included in the focused image information (20-4).

[0690] According to the present invention, the processor unit (130-4) can control an edge AI-based personal mobility device parking management device (100-4). Specifically, the processor unit (130-4) can interpret commands, participate in data transfer, or perform calculations.

[0691] According to the present invention, the processor unit (130-4) can execute the code stored in the memory unit (140-4). Specifically, the processor unit (130-4) can execute the edge AI model installed in the memory unit (140-4). Accordingly, the processor unit (130-4) can execute an image processing-based artificial intelligence algorithm using the wide-area image information (10-4) and focused image information (20-4) acquired by the first sensor unit (110-4) and the second sensor unit (120-4) as inputs to the edge AI model. Accordingly, an edge AI technology that can significantly improve data processing speed, data security, and data protection can be implemented.

[0692] According to the present invention, the processor unit (130-4) can determine the parking intention of the personal mobility device (200-4) by using wide-area image information (10-4) as input to the edge AI model.

[0693] According to the present invention, the processor unit (130-4) can recognize the identifier (30-4) of the personal mobility device (200-4) by using the focused image information (20-4) as input to the edge AI model. Specifically, the processor unit (130-4) can process parking-related information (40-4) of the personal mobility device (200-4) by using the wide-area image information (10-4) and / or the focused image information (20-4) as input to the edge AI mode...

Claims

1. In an edge AI device for object tracking and collecting object-specific spatial stay information, A sensor unit for sensing objects in a designated area; A communication unit that communicates with other edge AI devices; memory section; and Processor unit; including, The above processor unit The sensor unit analyzes the object sensed by the sensor unit to output object analysis information, and stores the movement path information and residence time information of the analyzed object in the memory unit. It is configured to create spatial residence information for each object based on the object analysis information output above, the stored movement path information, and the stored residence time information. The above object analysis information Information consisting of at least one of the size information of the object, the appearance information of the object, the color information of the object, the age information of the object, the gender information of the object, or the race information of the object. Edge AI device for object tracking and collecting object-specific spatial dwell information.

2. In paragraph 1, The above processor unit, If the object moves and the sensor unit can no longer sense the object in the designated area, at least one of the output object analysis information, the stored movement path information, the stored residence time information, and the object-specific spatial residence information created is transmitted to the other edge AI device through the communication unit. Edge AI device for object tracking and collecting object-specific spatial dwell information.

3. In the field facility control infrastructure utilizing edge AI, At least one edge AI system; Including, At least one edge AI system as described above, At least one node device configured to receive location information in real time from at least one user terminal; A head device configured to receive the location information from the node device and track the location of at least one user terminal in real time based on the received location information; and An integrated control server configured to receive personal information from at least one user terminal and store the received personal information; The above head device, The personal information transmitted by the at least one user terminal from which the above location information is received to the integrated control server is configured to be identified from the integrated control server. Field facility control infrastructure utilizing edge AI.

4. In paragraph 3, The above head device, It is configured to generate space-specific residence time information within the field facility for at least one user terminal based on the location information. Field facility control infrastructure utilizing edge AI.

5. In paragraph 4, The above head device, It is configured to generate facility control information by using at least one of the above location information, the above personal information, and the above space-specific residence time information as inputs to an artificial intelligence model, and to transmit the generated facility control information to the integrated control server. Field facility control infrastructure utilizing edge AI.

6. In paragraph 5, The above facility control information is: The above integrated control server is configured to transmit user terminal control information to at least one user terminal, The above facility control information is: The above integrated control server is configured to control the above field facility, Field facility control infrastructure utilizing edge AI.

7. As a server device for providing personalized artificial intelligence, memory section; Processor unit; and including the Department of Communications; The above memory section Save at least one artificial intelligence model, The above processor unit, configured to create an artificial intelligence platform storing at least one artificial intelligence model; The above communication department, It is configured to transmit the first artificial intelligence model stored in the artificial intelligence platform to the edge device, The first artificial intelligence model transmitted to the edge device is configured to receive a second artificial intelligence model learned based on the first-first personalized information from the edge device, configured to transmit the second artificial intelligence model to another edge device; A server device for providing personalized artificial intelligence.

8. In paragraph 7, The above communication department, configured to receive a third artificial intelligence model additionally learned based on the second personalized information from the other edge device, A server device for providing personalized artificial intelligence.

9. In paragraph 7, The above first artificial intelligence model is, It is configured to perform multi-modal learning based on field data, The above second artificial intelligence model or the above third artificial intelligence model, Configured to perform multi-modal learning based on personalized information, A server device for providing personalized artificial intelligence.

10. In an electronic device for performing situational judgment operations in the field, Sensor section; A memory unit for storing an artificial intelligence model based on image processing; and including a processor unit; The above sensor unit senses first field sensing information, wherein the first field sensing information includes at least one of first field sound information, first field text information, and first field image information, The above processor unit is configured to generate first field situation analysis information using the first field sensing information sensed by the sensor unit as input to the image processing-based artificial intelligence model, and to perform a situation judgment operation based on the generated first field situation analysis information. The above sensor part configured to sense second field sensing information, wherein the second field sensing information includes at least one of second field acoustic information, second field text information and second field image information; The above processor unit, The second field sensing information is used as input to the artificial intelligence model based on the stored image processing to generate second field situation analysis information. Based on the above second field situation analysis information, verification is performed on the above first field situation analysis information, Configured to perform the situation judgment operation based on the above first field situation analysis information and the verification result. An electronic device for performing situational assessment operations in the field.

11. In Article 10, The electronic device further comprises a network section; The above network section configured to receive third field situation analysis information generated by said other electronic device from another electronic device; The above processor unit, Configured to perform the situation judgment operation based on at least one of the first field situation analysis information generated above, the verification result performed above, and the third field situation analysis information received above. An electronic device for performing situational assessment operations in the field.

12. In paragraph 11, The above network section, The first field situation analysis information generated above is transmitted to a server, and the server is configured to receive update information of the artificial intelligence model generated based on the first field situation analysis information from the server. The above processor unit, configured to update the artificial intelligence model based on the above update information; An electronic device for performing situational assessment operations in the field.

13. In a personal mobility device parking management device based on edge AI, A first sensor unit for obtaining wide-area image information including the personal transportation means and / or the parking lot; Memory unit where edge AI model is implemented; A processor unit that determines the parking intention of the personal mobility device by using the acquired wide-area image information as input to the edge AI model; and A second sensor unit that obtains focused image information including the personal transportation device when there is an intention to park the personal transportation device; Including, The above processor unit is configured to recognize the identifier of the personal mobility device by using the acquired focused image information as input to the edge AI model, The above memory section, configured to store at least one of the wide-area image information, the focused image information and the identifier of the personal mobility device; Edge AI-based personal mobility parking management device.

14. In paragraph 13, If there is an intention to park the above personal transportation device, The above processor unit, It is configured to process parking-related information using the above wide-area image information and / or the above focused image information as input to the edge AI model, The above edge AI model, Including an artificial intelligence model based on image processing, Edge AI-based personal mobility parking management device.

15. In paragraph 14, The above parking information is: It includes at least one of target parking space information of the personal transportation means, entry time information of the personal transportation means, exit time information of the personal transportation means, and parking time information of the personal transportation means. The target parking space information for the above personal mobility device is: Information about the parking space closest to the above personal mobility device, Edge AI-based personal mobility parking management device.

16. In the edge AI-based vehicle parking management system, At least one electronic device; Including, At least one of said electronic devices, A sensor unit that acquires first image information including the vehicle and / or parking lot; A network unit capable of receiving second image information including the vehicle and / or the parking lot from at least one other electronic device; Memory unit where edge AI model is implemented; and A processor unit configured to process parking-related information using the first image information and / or the second image information as input to the edge AI model; Including, The above memory section configured to store at least one of the first image information, the second image information and the parking-related information, The above network section configured to transmit said first image information and / or said parking related information to at least one other electronic device; The above edge AI model Configured to include an artificial intelligence model based on image processing, Edge AI-based vehicle parking management system.

17. In paragraph 16, The above parking information is: It includes at least one of real-time location information of the vehicle, target parking space information of the vehicle, identifier information of the vehicle, structure information of the parking lot, entry time information of the vehicle, exit time information of the vehicle, and parking time information of the vehicle. The target parking space information for the above vehicle is: Information about the parking space nearest to the vehicle or the reserved parking space, Edge AI-based vehicle parking management system.

18. In the field facility control infrastructure utilizing edge AI, At least one edge AI system configured to generate facility control information; and An integrated control server configured to receive facility control information from at least one edge AI system and control the field facility based on the facility control information; Including, At least one edge AI system as described above, It is configured to receive location information from at least one user terminal of a user, identify personal information transmitted by the user terminal from which the location information was received to the integrated control server, and sense facility environment information related to at least one user. The above facility control information is: Information generated by at least one edge AI system based on one or more of the location information, the personal information, and the facility environment information. Field facility control infrastructure utilizing edge AI.

19. In paragraph 18, The above facility control information is: The above integrated control server is configured to transmit user terminal control information to the user terminal, The above user terminal, It includes an output module, receives the user terminal control information from the integrated control server, and is configured to output the user terminal control information through the output module. The above edge AI system, At least one node device configured to receive the location information from the user terminal and sense the facility environment information; A head device configured to receive the location information and the facility environment information from at least one node device, identify personal information transmitted by the user terminal from which the location information was received to the integrated control server from the integrated control server, and generate the facility control information by using at least one of the location information, the personal information, and the facility environment information as an input to an artificial intelligence model; Including, Field facility control infrastructure utilizing edge AI.

20. In an edge AI device for object tracking and collecting information on the object's spatial facility stay, A sensor unit for sensing objects in a designated area; A communication unit that communicates with other edge AI devices, other terminals, or external servers; A memory section storing unique information necessary for identifying the above object; and Processor unit; including, The above processor unit Match the unique information stored in the memory unit to the object sensed by the sensor unit, and store the movement path information and residence time information of the matched object in the memory unit. It is configured to create spatial residence information of the sensed object based on the unique information required for identification of the matched object, the stored movement path information, and the stored residence time information. The above processor unit If the unique information stored in the memory unit does not match the object sensed by the sensor unit, Configured to transmit external access-related information to the other terminal, the external server, or the other edge AI device through the above communication unit. Edge AI for object tracking and collecting information on the spatial facility stay of said objects.

Images