Twin space-based object sharing method and system

Abstract

The present invention relates to a twin space-based object sharing method and system, and the method comprises the steps of: identifying an object present in a real space and calculating an object-specific location, through an object detection sensor; collecting object information including the object-specific location through an edge server that is connected to the object detection sensor and is located near the real space; collecting and updating the object information through a cloud server connected to the edge server; and sharing, through the cloud server, the updated object information in at least one virtual space that has implemented the real space.

Description

Twin-space-based object sharing method and system

[0001] The present invention relates to an object sharing technology, and more specifically, to a twin-space-based object sharing method and system capable of identifying objects existing in a real space based on an object detection sensor and sharing object information in a virtual space based on the location of each object.

[0002]

[0003] Recently, various content utilizing virtual information, such as virtual reality (VR) or augmented reality (AR), has been proposed. Virtual reality (VR) is a virtual space created using artificial technologies such as computers, which resembles reality but possesses characteristics that are different from reality. Augmented reality (AR), a branch of VR, refers to a technique that synthesizes virtual information onto real-world objects viewed by the user, allowing the user to perceive the virtual information as part of reality.

[0004] To create virtual spaces using virtual reality or augmented reality, the process of recognizing objects existing in the real space is essential. This object recognition technology typically relies on marker recognition, visualizing various information based on these markers. However, marker-based object recognition suffers from the limitation of the active space, as it fails to track the viewpoint position when the reference marker moves out of the recognition range.

[0005] Accordingly, there is a growing demand for basic technologies that can recognize indoor spaces and objects and secure connectivity between the virtual and real worlds by augmenting additional information to recognized objects. Accordingly, research is being conducted on systems that determine and share the locations of dynamic objects using fixed or mobile cameras or position sensors.

[0006]

[0007] [Prior Art Literature]

[0008] [Patent Document]

[0009] Korean Patent Publication No. 2021-0129883 (September 30, 2021)

[0010]

[0011] One embodiment of the present invention provides a twin-space-based object sharing method and system capable of identifying objects existing in a real space and sharing object information in a virtual space based on the location of each object.

[0012] One embodiment of the present invention provides a twin-space based object sharing method and system capable of tracking the location of each object frame by frame through an object detection model based on images collected through an object detection sensor.

[0013]

[0014] Among the embodiments, a twin-space based object sharing method includes: a step of identifying an object existing in a real space through an object detection sensor and calculating a location for each object; a step of collecting object information including the location for each object through an edge server connected to the object detection sensor and located near the real space; a step of collecting and updating the object information through a cloud server connected to the edge server; and a step of sharing the updated object information on at least one virtual space that implements the real space through the cloud server.

[0015] The step of calculating the position of each object may include a step of classifying the identified object as a fixed object or a moving object; a step of collecting the status and operation log of the object together with the position of each object in the case of the fixed object; and a step of determining the type of the object together with the position of each object in the case of the moving object.

[0016] The step of calculating the position of each object may include a step of tracking the position of each object for each frame using an object detection model based on images collected through the object detection sensor.

[0017] The step of collecting and updating the object information may include a step of collecting and updating the location of each user participating in the real space and the virtual space through the cloud server.

[0018] The step of sharing the updated object information may include a step of directly accessing the edge server to share the updated object information of the virtual space when a preset time has elapsed based on the most recent sharing time of the object information within the virtual space.

[0019] The step of sharing the updated object information may include the step of receiving a message regarding a specific moving object from a VR user participating in the virtual space through the cloud server; and the step of sharing the message to an AR user participating in the real space through the edge server.

[0020] The step of sharing to the AR user may include a step of sharing the message through the AR user's AR screen when a moving object associated with the message exists in the AR user's field of view (FOV) for a preset period of time.

[0021] Among the embodiments, a twin space-based object sharing system includes an object detection sensor that identifies an object existing in a real space and calculates a location for each object; an edge server that collects object information including the location for each object from the object detection sensor; and a cloud server that collects and updates the object information from the edge server and shares the updated object information on at least one virtual space that implements the real space.

[0022]

[0023] The disclosed technology may have the following effects. However, this does not mean that a particular embodiment must include all or only the following effects, and therefore the scope of the disclosed technology should not be construed as being limited thereby.

[0024] A twin-space-based object sharing method and system according to one embodiment of the present invention can identify objects existing in a real space and share object information in a virtual space based on the location of each object.

[0025] The twin-space-based object sharing method and system according to the present invention can track the location of each object frame by frame through an object detection model based on images collected through an object detection sensor.

[0026]

[0027] FIG. 1 is a drawing illustrating an object sharing system according to the present invention.

[0028] Fig. 2 is a drawing explaining the cloud server system configuration of the object sharing system of Fig. 1.

[0029] Fig. 3 is a drawing explaining the functional configuration of the object sharing system of Fig. 1.

[0030] Figure 4 is a flowchart illustrating an object sharing system according to the present invention.

[0031] FIG. 5 is a drawing illustrating one embodiment of an object sharing system according to the present invention.

[0032] FIG. 6 is a drawing illustrating one embodiment of an object sharing system according to the present invention.

[0033]

[0034] The description of the present invention is merely an example for structural and functional explanation, and therefore, the scope of the present invention should not be construed as being limited by the embodiments described in the text. That is, since the embodiments can be modified in various ways and can take various forms, the scope of the present invention should be understood to include equivalents that can realize the technical idea. In addition, the purposes or effects presented in the present invention do not mean that a specific embodiment must include all of them or only such effects, and therefore, the scope of the present invention should not be construed as being limited thereby.

[0035] Meanwhile, the meaning of the terms described in this application should be understood as follows.

[0036] Terms such as "first" and "second" are intended to distinguish one component from another, and the scope of the rights should not be limited by these terms. For example, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component.

[0037] When a component is said to be "connected" to another component, it should be understood that while it may be directly connected to that other component, there may also be other components intervening. Conversely, when a component is said to be "directly connected" to another component, it should be understood that there are no other intervening components. Similarly, other expressions describing relationships between components, such as "between" and "directly between," or "adjacent to" and "directly adjacent to," should be interpreted similarly.

[0038] Singular expressions should be understood to include plural expressions unless the context clearly indicates otherwise, and terms such as "comprises" or "have" should be understood to specify the presence of a feature, number, step, operation, component, part or combination thereof, but not to exclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0039] For each step, the identifiers (e.g., a, b, c, etc.) are used for convenience of explanation and do not describe the order of the steps. The steps may occur in a different order than stated unless the context clearly dictates a specific order. That is, the steps may occur in the same order as stated, may be performed substantially simultaneously, or may be performed in the opposite order.

[0040] The present invention can be implemented as computer-readable code on a computer-readable recording medium. The computer-readable recording medium includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices. Furthermore, the computer-readable recording medium can be distributed across network-connected computer systems, so that the computer-readable code can be stored and executed in a distributed manner.

[0041] Unless otherwise defined, all terms used herein have the same meaning as commonly understood by those of ordinary skill in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted to be consistent with their meaning within the context of the relevant technology, and should not be interpreted as having an idealized or overly formal meaning unless explicitly defined herein.

[0042]

[0043] FIG. 1 is a drawing illustrating an object sharing system according to the present invention.

[0044] Referring to FIG. 1, the object sharing system (100) may include an object detection sensor (110), a user terminal (111), an edge server (120), a cloud server (130), and a database (150).

[0045] The object detection sensor (110) may correspond to a computing device that can identify objects existing in real space and share the objects in virtual space through the object sharing system (100). Here, the twin space may correspond to an augmented reality space and a virtual reality space. The object detection sensor (110) may be connected to and operable with the object sharing system (100), and may include a sensor that can detect objects in an indoor space such as a living room or room where a user is located, and a camera module that photographs the indoor space. For example, the object detection sensor (110) may be implemented by being included in an iPhone, iPad, etc.

[0046] Additionally, the object detection sensor (110) may be connected to the object sharing system (100) and may be installed and run with a predetermined program or application necessary to collect information about virtual space and objects. That is, the object detection sensor (110) may be installed with a camera module capable of detecting objects in an indoor space. Through this, the object detection sensor (110) may utilize various services provided by the object sharing system (100).

[0047] The user terminal (111) may correspond to a computing device that can utilize the object sharing system (100). The user terminal (111) may be implemented as a smartphone, laptop, or computer that is connected to and operable with the object sharing device (140), but is not necessarily limited thereto and may also be implemented as various devices such as a tablet PC or VR / AR device.

[0048] Additionally, the user terminal (111) may include a Lidar sensor for measuring the shape and size of an indoor space, such as a living room or room, where the user is located, and a camera module for photographing the indoor space. For example, the user terminal (111) may be implemented as an iPhone, iPad, or the like equipped with a Lidar sensor.

[0049] Additionally, the user terminal (111) may be connected to an object sharing device (140) and may have installed and executed a predetermined program or application necessary to collect data regarding virtual space and objects. That is, an application capable of detecting object locations through a lidar sensor may be installed on the user terminal (111). Through this, the user terminal (111) may utilize various services provided by the 3D asset creation device (130).

[0050] The edge server (120) may be implemented as a server corresponding to a computer or program capable of providing distributed processing performance in a distributed open architecture. The edge server (120) may be connected to the object detection sensor (110) and the cloud server (130) via a wireless network such as Bluetooth or WiFi, and may transmit and receive data with the object detection sensor (110) and the cloud server (130) via the network.

[0051] The cloud server (130) may be implemented as a computer or a server corresponding to a program that transmits a three-dimensional image including a virtual space to an AR / VR device (not shown in FIG. 1) via a network. In addition, the cloud server (130) may receive a request from an AR / VR device and directly execute an application program installed in the cloud server (130), and transmit an object image of the real space or virtual space of the corresponding application in the virtual space to the XR device. In another embodiment, the cloud server (130) may receive a request from an AR / VR device and execute an application program installed in a remote object detection sensor (110), and may transmit an object image of the real space or virtual space of the corresponding application in the virtual space to the AR / VR device based on captured data received from the object detection sensor (110). In another embodiment, the object detection sensor (110) may receive a request from an AR / VR device and execute an application installed on the object detection sensor (110), and may generate capture data during the execution of the application and transmit the captured data to a cloud server (130), and the cloud server (130) may transmit an object image of a real space or a virtual space to an XR device based on the captured data received from the object detection sensor (110).

[0052] The cloud server (130) can be connected to the XR device via a wireless network such as Bluetooth, WiFi, or 5G communication, and can transmit and receive data with the XR device via the network.

[0053] In addition, the cloud server (130) may be implemented as a server corresponding to a computer or program that can identify objects existing in a real space through an object detection sensor (110) and collect and share information about the objects. The cloud server (130) may be connected to the edge server (120) via a wireless network such as Bluetooth or WiFi, and may transmit and receive data with the edge server (120) via the network.

[0054] The database (150) may correspond to a storage device that stores various information required during the operation of the object sharing device (140). For example, the database (150) may store images, videos, and object information acquired through the object detection sensor (110), and may store object detection model information for twin space-based object sharing. However, the database is not necessarily limited thereto, and may store information collected or processed in various forms during the process in which the object sharing device (140) performs the twin space-based object sharing method using the object detection sensor (110).

[0055] In one embodiment, the object sharing system may be implemented as an object sharing device according to the present invention. The object sharing system is described herein assuming an object sharing device.

[0056]

[0057] Fig. 2 is a drawing explaining the cloud server system configuration of the object sharing system of Fig. 1.

[0058] Referring to FIG. 2, the cloud server (130) may include a processor (210), memory (230), a user input / output unit (250), and a network input / output unit (270).

[0059] The processor (210) can execute a procedure for processing each step in the process of operating the cloud server (130), manage the memory (230) that is read or written in this process, and schedule a synchronization time between the volatile memory and the non-volatile memory in the memory (230). The processor (210) can control the overall operation of the object sharing system (100), and can be electrically connected to the memory (230), the user input / output unit (250), and the network input / output unit (270) to control the data flow therebetween. The processor (210) can be implemented as a CPU (Central Processing Unit) or GPU (Graphics Processing Unit) of the object sharing device (140).

[0060] The memory (230) may include an auxiliary memory device implemented with a non-volatile memory such as an SSD (Solid State Disk) or an HDD (Hard Disk Drive) and used to store all data required for the cloud server (130), and may include a main memory device implemented with a volatile memory such as a RAM (Random Access Memory). In addition, the memory (230) may store a set of commands that execute a twin space-based object sharing method according to the present invention by being executed by an electrically connected processor (210).

[0061] The user input / output unit (250) includes an environment for receiving user input and an environment for outputting specific information to the user, and may include, for example, an input device including an adapter such as a touchpad, a touch screen, a virtual keyboard, or a pointing device, and an output device including an adapter such as a monitor or a touch screen. In one embodiment, the user input / output unit (250) may correspond to a computing device connected via remote access, and in such a case, the cloud server (130) may be implemented as an independent server.

[0062] The network input / output unit (270) provides a communication environment for connecting to an external device via a network, and may include, for example, an adapter for communication such as a Local Area Network (LAN), a Metropolitan Area Network (MAN), a Wide Area Network (WAN), and a Value Added Network (VAN). In addition, the network input / output unit (270) may be implemented to provide a short-range communication function such as WiFi or Bluetooth, or a wireless communication function of 4G or higher for wireless transmission of data.

[0063]

[0064] Fig. 3 is a drawing explaining the functional configuration of the object sharing system of Fig. 1.

[0065] Referring to FIG. 3, the object sharing system (100) can perform a twin-space-based object sharing method according to the present invention. To this end, the object sharing system (100) can include an object detection sensor (110), an edge server (120), a cloud server (130), and a control unit (not shown in FIG. 3).

[0066] At this time, the object sharing system (100) according to the embodiment of the present invention does not have to include all of the above functional configurations at the same time, and may be implemented by omitting some of the above configurations or selectively including some or all of the above configurations according to each embodiment.

[0067]

[0068] The object detection sensor (110) can identify objects existing in real space and calculate the location of each object. Here, the object may correspond to an object, animal, or person identified in a specific space by the object detection sensor (110). The object detection sensor (110) can identify at least one object in real space and calculate the location of each object. Here, the object detection sensor (110) can calculate the location of each object based on the distance between each object and the object detection sensor (110), and is not necessarily limited thereto, and can calculate the location of each object based on a specific reference point in the real space. In one embodiment, the object detection sensor (110) can calculate the movement distance from the initial point in time for each identified object to determine whether or not it has moved, and can distinguish each object and store it in the edge server (120).

[0069] In one embodiment, the object detection sensor (110) classifies the identified object as a stationary object or a moving object, and in the case of a stationary object, collects the status and operation log of the object together with the object-specific location, and in the case of a moving object, determines the type of the object together with the object-specific location. Here, a stationary object may correspond to an object whose movement is not identified within a reference time, and may correspond, for example, to a machine or device located in a specific space. If the identified object corresponds to a stationary object, the object detection sensor (110) may collect the status and operation log of the object. Here, the status of the object may correspond to an operation status or an idle status of the object. In addition, the operation log may correspond to load, temperature, noise, etc. of the object, and is not necessarily limited thereto, and may correspond to an operation signal according to the object characteristics. If the identified object is a stationary object, the object detection sensor (110) may be connected to each stationary object and collect status and operation log data for each stationary object. In one embodiment, the object detection sensor (110) can collectively collect data for each fixed object based on a system that integrates and manages at least one fixed object.

[0070] In one embodiment, the object detection sensor (110) can classify the type of the identified object if it corresponds to a moving object. Here, the object detection sensor (110) can classify each moving object as a person, a movable object, etc., but is not necessarily limited thereto. In one embodiment, the object detection sensor (110) can distinguish between a person and a movable object by separately classifying and processing a moving object corresponding to a person. Here, the object detection sensor (110) can define the user terminal (111) as a moving object corresponding to a person and separately classify and process it.

[0071] In one embodiment, the object detection sensor (110) can track the position of each object frame by frame through an object detection model based on the collected image. Here, the object detection model may correspond to an artificial intelligence model designed to receive image data as input and output identified objects in the image through an inference process. The object detection sensor (110) can generate object information such as object position (bounding box), object size, and object type from objects output in the image through the object detection model. In one embodiment, the object detection sensor (110) can generate object information for each frame by performing an object identification operation for each consecutive frame of the image based on the object detection model. Here, the object detection sensor (110) can track the position of a specific object by detecting changes in the object information for each frame. In addition, the object detection sensor (110) can extract a frame image for the corresponding image by performing a sampling process at a specific cycle on the collected image and track a specific object for each frame.

[0072] The edge server (120) can be connected to the object detection sensor (110) to collect object information including the location of each object. Here, the edge server (120) can collect the location of each moving object by detecting the location of the user terminal (111). That is, the edge server (120) can collect object information including the location of each object based on direct communication between the AR / VR device worn by the user and the cloud server (130). Here, in the case of a user using an AR device, the edge server (120) can collect location information from the cloud server (130) by passing through the edge server (120).

[0073] The cloud server (130) can be connected to the edge server (120) to collect and update object information. Here, the cloud server (130) can collect at least one piece of object information through the object detection sensor (110) and integrate and manage the object information collected through the edge server (120). In addition, the cloud server (130) can perform updates on specific object information by receiving the integrated object information from the edge server (120). In one embodiment, the cloud server (130) can receive an update request from a user terminal (111) and perform updates on specific object information.

[0074] In one embodiment, the cloud server (130) can collect and update the locations of each user participating in the real space and the virtual space. Here, the cloud server (130) can collect the location of a specific user based on the location of the user terminal (111) in the real space. In addition, the cloud server (130) can update the location of a specific user participating in the virtual space based on the location of the user terminal (111) collected in the real space. In one embodiment, the cloud server (130) can reflect the user's location within the virtual space by collecting and updating the location of a specific user participating in the virtual space in real time.

[0075] The cloud server (130) can share updated object information on at least one virtual space that implements a real space. Here, the cloud server (130) can reflect the location of a moving object within the real space to the virtual space. In one embodiment, when object information in the virtual space is updated, the cloud server (130) can reflect it by displaying it in augmented reality in the real space based on the object information. In addition, the cloud server (130) can share whether or not the object information between the real space and the virtual space has been updated by sharing the object information with the user terminal (111).

[0076] In one embodiment, the cloud server (130) can directly access the edge server (120) to share updated object information of the virtual space when a preset time has elapsed from the most recent shared time of object information within the virtual space. Here, the cloud server (130) can update object information for a specific virtual space in real time based on a preset cycle. In addition, the cloud server (130) can access the edge server (120) based on an input from the user terminal (111) and share updated object information of the virtual space.

[0077] In one embodiment, the cloud server (130) may receive a message regarding a specific mobile object from a VR user participating in a virtual space and share the message with an AR user participating in a real space via the edge server (120). Here, the AR user may correspond to a user designated by the VR user, but is not necessarily limited thereto, and may correspond to at least one user existing in the real space. In addition, the message may correspond to, for example, a caution, delivery, sharing, or important message regarding a specific mobile object, but is not necessarily limited thereto, and may correspond to a command regarding a specific mobile object. The cloud server (130) may receive a message for a specific mobile object and share it with an AR user participating in the real space, thereby requesting the AR user to perform an action associated with the message regarding the specific mobile object.

[0078] In one embodiment, the cloud server (130) may share a message through the AR user's AR screen if a moving object associated with the message exists in the AR user's field of view (FOV) for a preset period of time during the process of sharing the message to the AR user. For example, the cloud server (130) may analyze the AR user's field of view and, if the AR user's field of view gazes at a moving object associated with a specific message for a preset period of time, visualize the message contained in the moving object. Here, the cloud server (130) may display the message containing the moving object on the user terminal (111), but is not necessarily limited thereto, and may display the specific message by visualizing it near the moving object.

[0079] In one embodiment, the cloud server (130) may receive one or more messages from a VR user in a virtual space and provide the messages to an AR user in a real space. Here, the cloud server (130) may share the messages with the AR user in the real space via the edge server (120). In addition, the cloud server (130) may assign one or more messages to each AR user based on input from the user terminal (111). Here, the cloud server (130) may provide the assigned messages by displaying them on the terminal of a specific AR user, but is not limited thereto and may also visualize them in the vicinity of the AR user. In addition, the cloud server (130) may request execution of an unassigned message (i.e., an unassigned message) by sharing it with all AR users.

[0080] In one embodiment, the cloud server (130) may provide vibration and a notification sound to the user terminal (111) when a message regarding a specific moving object is visualized. That is, the cloud server (130) may provide visual, auditory, and tactile message notification functions to the AR user when a message regarding a specific moving object is visualized.

[0081] In one embodiment, the cloud server (130) can reflect the content of an AR user's message in the virtual space. For example, if the AR user moves a specific fixed object in response to the message, the cloud server (130) can reflect and modify the location of the fixed object in the virtual space. Furthermore, the cloud server (130) is not limited to this, and can also reflect the content of changes in the configuration of objects in the real space based on the AR user's message in the virtual space.

[0082] The control unit (not shown in FIG. 3) controls the overall operation of the object sharing device (130) and can manage the control flow or data flow between the object detection sensor (110), the edge server (120), and the cloud server (130).

[0083]

[0084] Figure 4 is a flowchart illustrating an object sharing system according to the present invention.

[0085] Referring to FIG. 4, the object sharing system (100) can be performed through a plurality of steps. Specifically, the object sharing system (100) identifies objects existing in a real space based on an object detection sensor (110) and calculates the location of each object (step S410). The object sharing system (100) collects object information including the location of each object from the object detection sensor (110) based on an edge server (120) (step S430). The object sharing system (100) collects and updates object information from the edge server (120) based on a cloud server (130), and shares the updated object information on at least one virtual space that implements a real space (step S450).

[0086]

[0087] FIG. 5 is a drawing illustrating one embodiment of an object sharing system according to the present invention.

[0088] Referring to FIG. 5, the object sharing system (100) can identify objects existing in an actual space based on an object detection sensor (110). Here, the object sharing system (100) can identify one or more objects, classify each object as a moving object or a fixed object, and then calculate a location based on the characteristics of each object.

[0089] Next, the object sharing system (100) can transmit the location information of the collected moving object and the information of the fixed object to the edge server (120). Here, the object sharing system (100) can collect object information through the edge server (120) and manage the collected object information in an integrated manner. Next, the object sharing system (100) can transmit the collected object information to the cloud server (130) and create a virtual space in the cloud server (130) based on the collected object information. Here, the object sharing system (100) can collect and update the location of each user participating in the real space and the virtual space through the cloud server (130). Here, the object sharing system (100) can receive a message regarding a specific moving object from a VR user and share the message to an AR user participating in the real space through the edge server (120). That is, the object sharing system (100) can request the AR user to perform the message by receiving a message related to a moving object or a fixed object from the VR user and transmitting the message to the cloud server (130) and the edge server (120).

[0090] In one embodiment, the object sharing system (100) can determine the importance of a message regarding a specific mobile object and perform a designated action based on the importance. For example, if the importance of a message regarding a specific mobile object is high, the object sharing system (100) can deliver the message to each user participating in the real space via the edge server (120) and update the real space in real time. Here, a message with high importance may correspond to location data of a user participating in the real space or a mobile object, and is not necessarily limited thereto, and may correspond to data that needs to be immediately reflected in the virtual space of a user participating in the virtual space.

[0091] In one embodiment, the object sharing system (100) may transmit a message regarding a specific moving object to a cloud server (130) if the message has a low importance. Here, a low importance message may correspond to data generated in a real space that does not require immediate reflection or data that needs to be accumulated and used. The object sharing system (100) may transmit a low importance message to the cloud server (130) so that the virtual space can be updated as needed. That is, the object sharing system (100) may reflect and update a low importance message in the virtual space through the user terminal (111) or each user participating in the virtual space.

[0092]

[0093] FIG. 6 is a drawing illustrating one embodiment of an object sharing system according to the present invention.

[0094] Referring to FIG. 6, the object sharing system (100) can implement a virtual space based on object information received from a cloud server (130) and share the object information with an AR user. Here, the object information in the virtual space can be updated periodically, and is not necessarily limited thereto, and can be updated through interaction with a VR user. In one embodiment, the object sharing system (100) can perform an update operation based on the object information of the cloud server (130). Then, the object sharing system (100) can update based on the most recent time point at which the object information was received from the cloud server. In one embodiment, the object sharing system (100) can request the VR user to receive object information directly from the edge server (120) when a preset time has passed based on the most recent time point at which the object information was shared. Accordingly, the object sharing system (100) can update the object information in the virtual space for the corresponding VR user and visualize it on the user terminal (111) or in the vicinity of the corresponding moving object.

[0095]

[0096] Although the present invention has been described above with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various modifications and changes may be made to the present invention without departing from the spirit and scope of the present invention as set forth in the claims below.

[0097]

[0098] [Explanation of symbols]

[0099] 100: Object Sharing System

[0100] 110: Object detection sensor 111: User terminal

[0101] 120: Edge Server 130: Cloud Server

[0102] 140: Object sharing device 150: Database

[0103] 210: Processor 230: Memory

[0104] 250: User input / output section 270: Network input / output section

Claims

1. A step of identifying objects existing in real space and calculating the location of each object using an object detection sensor; A step of collecting object information including the location of each object through an edge server connected to the object detection sensor and located near the actual space; A step of collecting and updating the object information through a cloud server connected to the edge server; and A twin space-based object sharing method, comprising: a step of sharing the updated object information on at least one virtual space that implements the actual space through the cloud server; 2. In the first paragraph, the step of calculating the location of each object is A step of classifying the identified object as a fixed object or a moving object; In the case of the above fixed object, a step of collecting the status and operation log of the object together with the location of each object; and A twin space-based object sharing method, characterized in that it includes a step of determining the type of the object together with the location of each object in the case of the above moving object.

3. In the second paragraph, the step of calculating the location of each object is A twin-space-based object sharing method, characterized by including a step of tracking the location of each object frame by frame through an object detection model based on images collected through the object detection sensor.

4. In paragraph 1, the step of collecting and updating the object information is A twin space-based object sharing method, characterized by including a step of collecting and updating the location of each user participating in the real space and the virtual space through the cloud server.

5. In the first paragraph, the step of sharing the updated object information is A twin space-based object sharing method, characterized in that it includes a step of directly accessing the edge server and sharing updated object information of the virtual space when a preset time has elapsed from the most recent sharing time of the object information within the virtual space.

6. In the first paragraph, the step of sharing the updated object information is A step of receiving a message regarding a specific moving object from a VR user participating in the virtual space through the cloud server; and A twin-space based object sharing method, characterized by including a step of sharing the message to an AR user participating in the actual space through the edge server.

7. In paragraph 6, the step of sharing with the AR user A twin-space based object sharing method, characterized in that it comprises a step of sharing the message through the AR screen of the AR user when a moving object associated with the message exists in the field of view (FOV) of the AR user for a preset period of time.

8. Object detection sensor that identifies objects existing in real space and calculates the location of each object; An edge server that collects object information including the location of each object from the object detection sensor; and A twin space-based object sharing system, comprising: a cloud server that collects and updates the object information from the edge server and shares the updated object information on at least one virtual space that implements the actual space.

Images