Method, system and non-transitory computer-readable recording medium for supporting the sharing of experiences between users

Abstract

According to one aspect of the present invention, there is provided a method for supporting the sharing of experiences between users, the method including the steps of: acquiring spatial information regarding shared content to be the subject of experience sharing; determining a virtual space in which to provide the shared content to the user by referring to the spatial information regarding the shared content and time information related to the user; and, when copyrighted content is generated by the user in the virtual space, allowing the copyrighted content to be shared based on a location that matches the location where the copyrighted content is generated in the virtual space.

Description

[Technical field]

[0001] The present invention relates to a method, a system and a non-transitory computer-readable medium for supporting the sharing of experiences between users. [Background technology]

[0002] Recently, there has been an increasing demand for sharing video content that one has experienced or filmed with others through social network services (SNS) and the like.

[0003] As an example of related art in this regard, the technology disclosed in Patent Document 1 may be cited. A system for providing a platform for SNS content production and sharing services has been introduced, which includes a server providing a platform for SNS content production and sharing services, and a terminal that provides a user input interface to a user through the platform and receives post information to be shared on the SNS from the user, wherein the server receives the post information input by the user from at least one terminal, generates one or more squares based thereon, and controls the generated one or more squares to be shared through the SNS, and the squares are SNS content generated by converting the post information input by the user by the server according to predetermined criteria.

[0004] However, according to the above-mentioned conventional technology and other technologies introduced so far, users could only play the content shared by other users and exchange opinions about it in a separate web page, bulletin board, comment space, etc., so experiences were shared passively through the shared content, making interactive communication between users difficult. In addition, it was difficult to create additional content based on content created by other users without specialized knowledge of content editing.

[0005] Therefore, the inventors propose new and advanced technology that enables active experiences to be had based on content created and shared by other users, enables creation of new content based on that content, enables interactive communication with other users, and enables time capsule creation of the content by adding a concept of time to the content. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Korean Patent No. 10-1713256 Summary of the Invention [Problem to be solved by the invention]

[0007] An object of the present invention is to solve all of the problems of the prior art mentioned above.

[0008] Another object of the present invention is to provide a method for supporting a user to experience shared content in a realistic manner in a virtual space related to the shared content.

[0009] Another object of the present invention is to support the creation of new content in a virtual space related to shared content.

[0010] Another object of the present invention is to support interactive communication between multiple users in a virtual space related to shared content.

[0011] Another object of the present invention is to provide a user experience in which an augmented reality image and a virtual reality image are spatially linked.

[0012] Another object of the present invention is to support the experience of a virtual space related to shared content based on a specific point in time or in chronological order.

[0013] Another object of the present invention is to support a plurality of users existing at different times to share different experiences in a virtual space related to shared content. [Means for solving the problem]

[0014] A representative configuration of the present invention for achieving the above object is as follows.

[0015] According to one aspect of the present invention, a method for supporting experience sharing between users is provided, the method including the steps of acquiring spatial information regarding shared content that is the subject of experience sharing, determining a virtual space in which to provide the shared content to the user by referring to the spatial information regarding the shared content and time information related to the user, and, when copyrighted content is generated by the user in the virtual space, sharing the copyrighted content based on a location that matches a location where the copyrighted content is generated in the virtual space.

[0016] According to another aspect of the present invention, there is provided a system for supporting experience sharing between users, the system including: a spatial information acquisition unit that acquires spatial information regarding shared content that is the subject of experience sharing; a virtual space determination unit that determines a virtual space in which to provide the shared content to the user by referring to the spatial information regarding the shared content and time information related to the user; and a sharing management unit that, when copyrighted content is generated by the user in the virtual space, allows the copyrighted content to be shared based on a location that matches a location where the copyrighted content is generated in the virtual space.

[0017] In addition, other methods for embodying the present invention, other systems, and a non-transitory computer readable recording medium having a computer program for carrying out the methods are also provided. Effect of the Invention

[0018] According to the present invention, it becomes possible to experience shared content realistically in a virtual space related to the shared content.

[0019] The present invention also allows for the creation of new content in a virtual space related to the shared content.

[0020] Furthermore, the present invention allows multiple users to communicate interactively in a virtual space related to shared content.

[0021] In addition, the present invention can provide a user experience in which an augmented reality image and a virtual reality image are spatially linked.

[0022] In addition, the present invention can support the experience of a virtual space related to shared content based on a specific point in time or in chronological order.

[0023] Furthermore, according to the present invention, a plurality of users existing at different times can share different experiences in a virtual space related to shared content. [Brief description of the drawings]

[0024] [Figure 1] 1 is a diagram showing a schematic configuration of an entire system for supporting experience sharing between users according to an embodiment of the present invention. [Diagram 2] 2 is a diagram illustrating in detail the internal configuration of a support system according to an embodiment of the present invention; [Diagram 3] 1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; [Figure 4A] 1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; [Figure 4B] 1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; [Figure 4C] 1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; [Figure 4D] 1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; [Figure 5A] 1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; [Figure 5B] 1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; [Figure 6A] 1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; [Figure 6B] 1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; [Figure 7] 1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; [Figure 8]1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; [Figure 9] 1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; [Figure 10] 1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; [Figure 11] 1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; [Figure 12A] 1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; [Figure 12B] 1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; [Figure 13A] 1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; [Figure 13B] 1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; [Figure 13C] 1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; [Figure 13D] 1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; [Figure 13E] 1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; [Figure 14A] 1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; [Figure 14B] 1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; [Figure 14C] 1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; [Figure 15A] 1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; [Figure 15B] 1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; [Figure 16] 1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; [Figure 17A] 1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; [Figure 17B] 1 is a diagram illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] The following detailed description of the present invention will be made with reference to the accompanying drawings, which show, by way of example, specific embodiments in which the present invention may be practiced. Such embodiments are described in detail to enable those skilled in the art to fully practice the present invention. It should be understood that the various embodiments of the present invention are different from one another but are not necessarily mutually exclusive. For example, specific shapes, structures and characteristics described herein may be embodied in one embodiment without departing from the spirit and scope of the present invention. It should also be understood that the location or arrangement of individual components within each embodiment may be changed without departing from the spirit and scope of the present invention. Therefore, the following detailed description is not to be taken in a limiting sense, and the scope of the present invention should be understood to include the scope of the claims and all equivalents thereto. In the drawings, like reference numerals indicate the same or similar components throughout the various aspects.

[0026] In this specification, the term "content" refers to digital information or individual information elements that are accessible through a communication network and include characters, codes, voice, audio, images, videos, etc. Such content may include, for example, data such as text, images, videos, audio, links (e.g., web links), etc., or a combination of at least two of such data. More specifically, such content may include content related to characters, volumetric images, objects configured with point clouds or polygons, etc.

[0027] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, various preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings in order to enable those skilled in the art to which the present invention pertains to easily practice the present invention.

[0028] Overall system configuration

[0029] FIG. 1 is a diagram showing a schematic configuration of an entire system for supporting experience sharing between users according to an embodiment of the present invention.

[0030] As shown in FIG. 1, the overall system according to an embodiment of the present invention may include a communication network 100, a support system 200, and a device 300.

[0031] First, the communication network 100 according to an embodiment of the present invention may be configured regardless of the type of communication, such as wired communication or wireless communication, and may be configured as various communication networks, such as a local area network (LAN), a metropolitan area network (MAN), and a wide area network (WAN). Preferably, the communication network 100 in this specification may be the well-known Internet or World Wide Web (WWW). However, the communication network 100 is not necessarily limited thereto, and may include at least a part of a well-known wired / wireless data communication network, a well-known telephone network, or a well-known wired / wireless television communication network.

[0032] For example, communication network 100 may be a wireless data communication network that embodies, at least in part, conventional communication methods such as radio frequency (RF) communication, WiFi communication, cellular (e.g., LTE) communication, Bluetooth communication (more specifically, Bluetooth Low Energy (BLE) communication), infrared communication, ultrasonic communication, and the like.

[0033] Next, the support system 200 according to an embodiment of the present invention may be a digital device having a memory means and a microprocessor to provide a computing capability, and such a support system 200 may be a server system.

[0034] The support system 200 according to one embodiment of the present invention can communicate with a device 300 (described later) through a communication network 100, acquire spatial information regarding shared content to be a subject of experience sharing, determine a virtual space in which to provide the shared content to a user by referring to the spatial information regarding the shared content and time information related to a user, and when a copyrighted content is created by a user in the virtual space, share the copyrighted content based on a location that matches the location where the copyrighted content is created in the virtual space.

[0035] The shared content to be the subject of experience sharing according to an embodiment of the present invention may refer to content generated based on a real-world location or a virtual location and permitted to be shared (e.g., viewed, watched, experienced, etc.) by at least one user. For example, such shared content may be content generated based on a real-world location, such as a CCTV video or image, a 360-degree camera video or image, a smartphone video or image, a black box video or image, a drone video or image, a 3D point cloud (or a point cloud image or image), a 3D mesh (or a mesh image or image), etc., or may be content generated based on a virtual location, such as a Virtual Reality (VR) video.

[0036] The configuration and functions of the support system 200 according to the present invention will be described in detail below. Meanwhile, although the support system 200 has been described above, such description is merely an example, and it will be apparent to those skilled in the art that at least a part of the functions and components required for the support system 200 may be realized or included in a device 300 or an external system (not shown) as necessary.

[0037] Next, the device 300 (e.g., a user's device) according to an embodiment of the present invention is a digital device including a function for connecting to the support system 200 through the communication network 100 and communicating therewith. Any portable digital device having a memory means and a computing capability including a microprocessor, such as a smartphone, a tablet PC, smart glasses (e.g., such smart glasses support augmented reality (AR), virtual reality (VR), mixed reality (MR), extended reality (XR), etc.), a head mounted display (e.g., such head mounted display supports augmented reality (AR), virtual reality (VR), mixed reality (MR), extended reality (XR), etc.), may be adopted as the device 300 according to the present invention. In addition, according to an embodiment of the present invention, the device 300 may further include a photographing and scanning module (e.g., a camera, a LiDAR) for generating shared content, a display module (e.g., an LCD, an LED display, or an OLED display) for providing a virtual space or shared content, etc.

[0038] Meanwhile, the device 300 according to an embodiment of the present invention may include an application for supporting a function for supporting experience sharing between users according to the present invention. Such an application may be downloaded from the support system 200 or an external application distribution server (not shown).

[0039] Support system configuration

[0040] The internal configuration of the support system 200 and the functions of each component thereof, which perform important functions for implementing the present invention, will now be described in detail.

[0041] FIG. 2 is a diagram illustrating in detail the internal configuration of a support system 200 according to an embodiment of the present invention.

[0042] As shown in FIG. 2, the support system 200 according to an embodiment of the present invention may include a space information acquisition unit 210, a virtual space determination unit 220, a content providing unit 230, a sharing management unit 240, a communication unit 250, and a control unit 260. According to an embodiment of the present invention, the space information acquisition unit 210, the virtual space determination unit 220, the content providing unit 230, the sharing management unit 240, the communication unit 250, and the control unit 260 of the support system 200 may be program modules, at least some of which communicate with an external system. Such program modules may be included in the support system 200 in the form of an operating system, an application program module, or other program modules, and may be physically stored in various known storage devices. In addition, such program modules may be stored in a remote storage device that can communicate with the support system 200. Meanwhile, such program modules include routines, subroutines, programs, objects, components, data structures, etc. that perform specific tasks or execute specific abstract data types according to the present invention, which will be described later, but are not limited thereto.

[0043] First, the spatial information acquisition unit 210 according to an embodiment of the present invention may perform a function of acquiring spatial information related to shared content that is a target of experience sharing. The spatial information related to the shared content according to an embodiment of the present invention may include information on geography, location, object, theme, etc. of a real-world place or a virtual place related to the shared content.

[0044] For example, when the shared content is generated by photographing a real-world location, the spatial information acquisition unit 210 may acquire, as spatial information related to the shared content, at least one of a position (absolute coordinates (e.g., geographic coordinates such as GPS coordinates) or relative coordinates) corresponding to the real-world location where the shared content is photographed, a direction in which the real-world location is photographed in the shared content (e.g., a direction angle or azimuth for a subject in the shared content), an object (e.g., a subject) existing in the real-world location where the shared content is photographed, a point cloud for a space or object (e.g., a building existing in the real-world location) scanned in the real-world location where the shared content is photographed, and a theme identified in the shared content. Meanwhile, a theme according to an embodiment of the present invention means a main object identified in the shared content or an object that can be a predetermined standard, and may be a concept including not only tangible objects but also intangible objects (e.g., sky, sea, atmosphere, etc.). For example, such a theme may be an object that occupies the largest proportion (or ratio) in the shared content or an object that has a certain level of recognition among the public. Here, such recognition may be determined, for example, by referring to social network services (SNS), web databases, etc., or may be determined based on the results of analysis of public information (e.g., public information, SNS information, etc.) or private information (e.g., personal information) using artificial intelligence (e.g., deep learning).

[0045] More specifically, when the shared content is a video content generated by shooting the Eiffel Tower in Paris, France, the spatial information acquisition unit 210 can specify the Eiffel Tower, which has a recognition level above a predetermined level, as a theme, and can acquire information on at least one of the shape (such a shape may be a concept including all shapes based on spatial data such as an image shape or a point cloud), position, and direction of the Eiffel Tower in the shared content (i.e., information on the theme) as spatial information on the shared content. In addition, the spatial information acquisition unit 210 can acquire information on at least one of the position, height, width, and area of ​​the Eiffel Tower (i.e., information on the object) as spatial information on the shared content.

[0046] Meanwhile, the spatial information acquisition unit 210 according to one embodiment of the present invention may perform machine learning or deep learning based on information regarding the shape, position, size, area, width, volume, etc. of an object (i.e., information regarding attributes of the object) to identify or estimate information regarding the object in the shared content.

[0047] Next, the virtual space determination unit 220 according to one embodiment of the present invention may perform a function of determining a virtual space in which to provide shared content to a user by referring to space information related to the shared content and time information related to the user.

[0048] According to an embodiment of the present invention, the virtual space determination unit 220 may generate a virtual space by referring to space information related to the shared content. Here, the virtual space according to an embodiment of the present invention is a virtual space (such a space may be configured as a two-dimensional or three-dimensional space) that is realized to be the same as or similar to a real-world place or a virtual place related to the shared content, and may be generated at the same scale as a space specified or limited in the shared content, or may be generated at a scale further expanded by a predetermined range from the space.

[0049] For example, the virtual space determination unit 220 may identify a plurality of images (or pictures) related to a place related to the shared content by referring to spatial information related to the shared content, and generate the virtual space based on the identified plurality of images (or pictures) (on the other hand, when the shared content is two-dimensional, the shared content may be reconstructed in three dimensions based on the plurality of images). More specifically, the virtual space determination unit 220 may generate a virtual space by aligning (e.g., moving horizontally, moving vertically, rotating, or converting size) the plurality of images (or pictures) based on at least one of the shooting position and shooting direction of the shared content (e.g., to match at least one of the shooting position and shooting direction of the shared content), generate a virtual space by aligning the plurality of images based on at least one of the shape, size, position, and direction of a theme identified in the shared content (e.g., to match at least one of the shape, size, position, and direction of a theme identified in the shared content), or generate a virtual space by aligning the plurality of images based on a point cloud related to a place related to the shared content (e.g., to match at least one of the shape, size, position, and direction of a point cloud related to a place related to the shared content). In this case, even if the shared content only includes 2D video content related to a subject, a 3D virtual space can be generated by aligning a plurality of 3D videos (or images) related to the shared content based on information related to at least one of the shooting position and shooting direction of the shared content or information related to at least one of the shape, size, position and direction of a theme identified in the shared content. Meanwhile, according to one embodiment of the present invention, the plurality of videos can be obtained from at least one of public videos (or images) and personal videos (or images) related to a real-world place or a virtual place related to the shared content.For example, the virtual space determination unit 220 may acquire a plurality of videos (or images) related to the shared content from CCTV (e.g., CCTV installed at a concert venue or a store), social network service (SNS) videos, YouTube (registered trademark) videos, etc., captured at the same or similar shooting position or shooting direction as the shared content within a predetermined distance from a real-world location or virtual location related to the shared content (such a plurality of videos are acquired in real time through a streaming method, etc.). Meanwhile, the virtual space determination unit 220 may acquire all or a portion of the plurality of videos (or images) related to the shared content as space information related to the shared content.

[0050] As another example, the virtual space determination unit 220 may identify a plurality of point clouds related to a location related to the corresponding shared content by referring to spatial information related to the corresponding shared content, and generate the virtual space based on the identified plurality of point clouds (e.g., such that at least one of the shape, size, position, boundary, and direction of the identified plurality of point clouds match each other). More specifically, the virtual space determination unit 220 may generate a virtual space (see FIG. 16) by matching (e.g., moving horizontally, moving vertically, rotating, and converting size) a plurality of point clouds related to an object scanned at a real-world location where the shared content is captured.

[0051] As another example, when the shared content is a 360-degree image, the virtual space determination unit 220 may immediately generate a virtual space that is the same as or similar to a real-world location or a virtual location related to the shared content by referring to spatial information related to the shared content without using the multiple images. More specifically, the virtual space determination unit 220 may immediately generate a virtual space related to the shared content by referring to spatial information related to the shared content without separately identifying (or acquiring) multiple images related to a location related to the shared content (i.e., the shared content corresponding to a 360-degree image) (although a lookup table or database related to the virtual space corresponding to the spatial information may also be used). In this case, the virtual space determination unit 220 may generate an expanded virtual space by aligning (e.g., moving horizontally, moving vertically, rotating, and converting the size) multiple images related to the expanded space in order to further expand the virtual space by a predetermined range.

[0052] Furthermore, according to one embodiment of the present invention, the virtual space determination unit 220 can correct the virtual space generated as described above by referring to time information related to the user (e.g., information related to a specific time or period).

[0053] For example, according to an embodiment of the present invention, the time information related to the user may correspond to time information at the time when the shared content is provided to the user. Here, according to an embodiment of the present invention, the time information at the time when the shared content is provided to the user may be different from time information at the time when the virtual space is generated (e.g., the time information at the time when the shared content is provided to the user may be later than the time information at the time when the virtual space is generated; that is, the shared content may be provided to the user after a certain time has passed since the virtual space was generated). According to an embodiment of the present invention, the corresponding virtual space may be generated based on the time information at the time when the virtual space was generated. However, if the time information at the time when the shared content is provided to the user according to an embodiment of the present invention is different from the time information at the time when the virtual space was generated, the corresponding virtual space may be corrected to reflect a temporal change between the time information at the time when the shared content is provided to the user and the time information at the time when the virtual space was generated (e.g., seasonal changes of natural objects, creation of natural or artificial objects, erosion or deterioration, etc.). That is, the virtual space determination unit 220 according to an embodiment of the present invention may correct at least one component constituting the virtual space (e.g., the geography, location, object, theme, etc. of the virtual space) so as to match with time information at the time when the shared content is provided to the user. For this purpose, information on events by time period (e.g., the Eiffel Tower may not have existed before 1887 because it was a period before the Eiffel Tower was built), change patterns (e.g., the change patterns in which the leaves turn red, yellow, etc. in autumn and the leaves fall and snow accumulates on the branches in winter) related to natural or man-made objects may be used.

[0054] As another example, according to an embodiment of the present invention, the time information related to the user may correspond to time information specified (or searched) by the user. Here, according to an embodiment of the present invention, the time information specified by the user may be different from the time information at the time when the virtual space was generated (e.g., the time information specified by the user may precede or follow the time information at the time when the virtual space was generated; that is, the time information may be specified by the user before or after the generation of the virtual space). According to an embodiment of the present invention, the corresponding virtual space may be generated based on the time information at the time when the virtual space was generated. However, if the time information specified by the user according to an embodiment of the present invention is different from the time information at the time when the virtual space was generated, the corresponding virtual space may be corrected to reflect the temporal change between the time information specified by the user and the time information at the time when the virtual space was generated (e.g., seasonal change of natural objects, generation, erosion or deterioration of natural objects or artificial objects, etc.). That is, the virtual space determination unit 220 according to an embodiment of the present invention may correct at least one component constituting the virtual space (e.g., geography, location, object, theme, etc. of the virtual space) to match the time information specified by the user. For this purpose, information on seasonal events related to natural or artificial objects (for example, the Eiffel Tower may not have existed before 1887 because this was before the Eiffel Tower was built), change patterns (for example, the change patterns in which tree leaves turn red, yellow, etc. in autumn and then fall and snow accumulates on the branches in winter), etc., can be used.

[0055] Meanwhile, the virtual space determination unit 220 according to an embodiment of the present invention may change the virtual space chronologically by referring to time information related to a user. For example, the virtual space determination unit 220 according to an embodiment of the present invention may change the virtual space continuously according to the flow of time between first time information specified by a user and second time information (or third time information specified by a user (i.e., time information different from the first time information)) at the time when the shared content is provided to the user. That is, the virtual space determination unit 220 may reflect the time change between the time information (e.g., the first time information and the second time information) in the geography, location, object, theme, etc. of the virtual space in a chronological order.

[0056] Meanwhile, the virtual space determination unit 220 according to an embodiment of the present invention may correct the virtual space by referring to the output result of the virtual space correction model. For example, the virtual space correction model according to an embodiment of the present invention may be a model that is specified by performing learning based on artificial intelligence (AI) including machine learning or deep learning, and may be a model that performs learning (e.g., learning a pattern of at least one component constituting the virtual space changing by referring to changes in natural objects or artificial objects due to changes in time (or seasonal changes) (e.g., a pattern in which flowers bloom on a tree object constituting the virtual space in spring, and flowers scatter on the tree object and snow accumulates in winter)) by referring to information on changes over time (specifically, the flow of time) of at least one component constituting the virtual space or at least one environmental element related to at least one component (e.g., nature, technology, architecture, industry, climate, weather, events (e.g., incidents, accidents), etc.), and outputs at least one component constituting the virtual space to be matched with the time information related to the user by referring to the learning result when time information related to a user and at least one component constituting the virtual space are input. The virtual space determination unit 220 according to one embodiment of the present invention may refer to the output result of the virtual space correction model and correct at least one component constituting the virtual space (e.g., the geography, location, object, theme, etc. of the virtual space) so as to match with time information related to the user.

[0057] Meanwhile, in the above, it has been described that the virtual space determination unit 220 generates a virtual space by referring to spatial information related to the shared content, and then corrects the virtual space by referring to time information related to the user. However, it should be made clear that the virtual space determination unit 220 may regenerate (or additionally generate) the virtual space by referring to time information related to the user after generating a virtual space by referring to spatial information related to the shared content (specifically, by referring to both time information related to the user and spatial information related to the shared content).

[0058] Next, the content providing unit 230 according to an embodiment of the present invention may perform a function of providing shared content to a user in the virtual space determined by the virtual space determining unit 220 .

[0059] For example, the content providing unit 230 according to an embodiment of the present invention may display a spatial connection object for providing a user experience in which a location where a user exists and the virtual space are spatially connected, augmented with an image of the location where the user exists. According to an embodiment of the present invention, the location where the user exists may be a real world location, a virtual reality (VR) location, or a mixed reality (MR) location in which the real world and virtual reality are mixed.

[0060] More specifically, the content providing unit 230 may augment and display a spatially connected object having a two-dimensional shape such as a circle, ellipse, or rectangle, or a three-dimensional shape such as a sphere, hemisphere, rectangular hexahedron, or regular cube, against the background of an image of a real-world place or a virtual reality place where the user is present (or an object composed of a character, a volumetric image, a point cloud, or a polygon, against the background of an image of a real-world place or a virtual reality place where the user is present). In this case, according to an embodiment of the present invention, at least a part of the virtual space may be projected or overlapped onto the spatially connected object and displayed. That is, by projecting or overlapping at least a part of the virtual space, the user experience of spatially connecting the place where the user is present and the virtual space may be improved.

[0061] Meanwhile, when it is determined that the user is present in a location that meets a predetermined trigger condition (e.g., a real world location, a virtual reality (VR) location, or a mixed reality (MR) location where the real world and virtual reality are mixed), the content providing unit 230 may display the spatial connection object in an augmented manner against the background of an image of the location where the user is present. According to one embodiment of the present invention, the location that meets the predetermined trigger condition may refer to a location that has a predetermined condition where the virtual space can be displayed to the user.

[0062] For example, when a real-world location where three surfaces exist within a predetermined distance from a user (e.g., a user device) is a location that meets a predetermined trigger condition, the content providing unit 230 may determine that the trigger condition is satisfied when the user is located in a corner of a room, and may augment and display the spatially connected object against the background of an image of the real-world location where the user is located.

[0063] Next, the sharing management unit 240 according to one embodiment of the present invention can perform a function of allowing a user to share authored content based on a location that matches the location where the authored content is created in the virtual space when the authored content is created by a user in the virtual space determined by the virtual space determination unit 220.

[0064] For example, when a user generates (e.g., tags) at least one of text, image, video, and sound content (or content including a character, volumetric video, point cloud, or polygon-based object against a background of a video of a virtual reality location) as authored content for a given location in the virtual space, the shared management unit 240 can display the tagged authored content in the same manner on other users (specifically, other user devices) who are present at a real-world location (or at a different time in the virtual space) that matches the location where the authored content is generated in the virtual space. According to an embodiment of the present invention, the matching real-world location (or a different time in the virtual space) can be specified based on absolute coordinates (e.g., geographical coordinates such as GPS coordinates) or relative coordinates of the location where the authored content is tagged in the virtual space. Meanwhile, in order to more accurately specify the location where the authored content is generated in the virtual space and the matching location, at least one public or private database regarding a 3D map or 3D space regarding the location can be further referenced.

[0065] More specifically, when the location where the copyrighted content is generated by the user in the virtual space is the wall of a specific building, the sharing management unit 240 can augment and display the same content as the copyrighted content in real time on the wall of the corresponding building in the real world.

[0066] Meanwhile, the sharing management unit 240 according to an embodiment of the present invention may allow the authored content generated in a virtual space determined based on the specified time information to be shared with other users when a user specifies a predetermined time information. Also, the sharing management unit 240 according to an embodiment of the present invention may allow the authored content generated at each time point to be shared sequentially in the virtual space according to the flow of time when the virtual space determined by referring to the time information related to the user changes in a chronological manner. For example, the sharing management unit 240 according to an embodiment of the present invention may allow the authored content to be shared sequentially in the chronological manner (e.g., generated according to the time point at which each authored content was generated) in the virtual space that changes in a chronological manner between the first time information specified by the user and the second time information at the time when the shared content is provided to the user (or the third time information specified by the user (i.e., the third time information different from the first time information)).

[0067] Next, the communication unit 250 according to one embodiment of the present invention can perform a function that enables data transmission / reception to / from the spatial information acquisition unit 210, the virtual space determination unit 220, the content provision unit 230, and the sharing management unit 240.

[0068] Finally, the control unit 260 according to an embodiment of the present invention can perform a function of controlling data flow between the spatial information acquisition unit 210, the virtual space determination unit 220, the content providing unit 230, the sharing management unit 240, and the communication unit 250. That is, the control unit 260 according to the present invention can control the spatial information acquisition unit 210, the virtual space determination unit 220, the content providing unit 230, the sharing management unit 240, and the communication unit 250 to perform their respective unique functions by controlling the data flow from / to the outside of the support system 200 or the data flow between each component of the support system 200.

[0069] Hereinafter, a situation in which content is shared according to an embodiment of the present invention will be described in detail with reference to FIGS. 3 to 17B.

[0070] 3 to 17B are diagrams illustrating an example of a situation in which experience sharing is performed between users according to an embodiment of the present invention.

[0071] First Example

[0072] Referring to FIG. 3, a situation can be assumed in which shared content is provided by a first user (specifically, a first user device 310) and shared with a second user (specifically, a second user device 320) and a third user (specifically, a third user device 330).

[0073] First, spatial information regarding a plurality of shared contents including the shared content provided by the first user is acquired, and each virtual space can be generated by referring to the spatial information regarding the plurality of shared contents.

[0074] For example, referring to Figures 4A, 4B, 4C and 4D, spatial information for each of a plurality of shared contents, including shared content relating to major landmarks or major tourist destinations (e.g., "Wembley Stadium", "Paris Stade de France" and "Mercedes-Benz Arena Berlin"), can be obtained, and a plurality of virtual spaces for the plurality of shared contents can be generated by referring to the spatial information for each of the plurality of shared contents.

[0075] Next, a plurality of spatial connection objects (e.g., portals) that spatially connect the plurality of virtual spaces to a virtual location where a second or third user is present may be displayed in an augmented manner against the background of the virtual location where the second or third user is present.

[0076] For example, if the virtual location where the second or third user exists is space, a plurality of spatially connected objects 410, 420, 430 that spatially connect the space with the plurality of virtual spaces may be augmented and displayed against the background of the space.

[0077] As another example, when the virtual location where the second or third user is present is a virtual personal space (e.g., a virtual house, a virtual island, etc.), a plurality of spatial connection objects that spatially connect the virtual personal space and the plurality of virtual spaces may be displayed in an augmented manner against the background of the virtual personal space.

[0078] Meanwhile, a plurality of spatially connected objects 440, 450, 460, and 470 that spatially connect the plurality of virtual spaces and the virtual location where the second or third user is present may be augmented and displayed against the background of a predetermined map or list.

[0079] Second Example

[0080] According to an embodiment of the present invention, it is assumed that shared content, which is a subject of experience sharing, is shared by users in an office or a shopping mall.

[0081] First, spatial information about the shared content that is the subject of the shared experience can be obtained.

[0082] For example, spatial information regarding shared content generated based on Gwanghwamun Street among real-world locations may be acquired, and spatial information regarding shared content generated based on the Sahara Desert among virtual locations may be acquired.

[0083] A virtual space may then be generated with reference to spatial information about the shared content.

[0084] For example, a virtual space for the shared content may be generated by referring to spatial information for the shared content generated based on the Gwanghwamun Street. Also, a virtual space for the shared content may be generated by referring to spatial information for the shared content generated based on the Sahara Desert.

[0085] The shared content can then be provided to users in the generated virtual space.

[0086] For example, referring to Fig. 5A, a spatial connection object 510 for providing a user experience of spatially connecting the virtual space and an office 501 where the user exists may be augmented and displayed against a background of a real world image of the office 501 where the user exists. Referring to Fig. 6A, a spatial connection object 610 for providing a user experience of spatially connecting the virtual space and a shopping mall 601 where the user exists may be augmented and displayed against a background of a real world image of the shopping mall 601 where the user exists. Referring to Fig. 6B, when the user approaches or zooms in on the spatial connection object 610, a user experience of the user entering the virtual space through the spatial connection object 610 may be provided.

[0087] Next, when the authored content is created by the user in the virtual space, the authored content can be shared based on a location that matches the location where the authored content is created in the virtual space.

[0088] For example, referring to Fig. 5B, when the user creates copyrighted content 521-526 related to text and images on Gwanghwamun Street, the copyrighted content may be shared based on the actual Gwanghwamun Street, which is a location that matches the location where the copyrighted content is created. More specifically, the same content as the copyrighted content 521-526 may be augmented and displayed in real time on other users (specifically, other users' devices) who are on the actual Gwanghwamun Street.

[0089] As another example, referring to Figures 12A to 15B, when the user creates copyrighted content 1301 related to text and images in the virtual space of Gwanghwamun Street (see Figures 12A and 12B) (see Figures 13A, 13B, 13C, 13D and 13E), the copyrighted content can be shared based on the virtual Gwanghwamun Street that matches the location where the copyrighted content is created (see Figures 14A, 14B and 14C). In addition, the copyrighted content can include tags 1501 and 1502 that include information about the user who created the content (e.g., ID, creation time, creation location, etc.).

[0090] Meanwhile, when it is determined that the user is present in a real-world location or a virtual location that meets a predetermined trigger condition, the spatially connected object 510, 610 may be displayed in an augmented manner against the background of a real-world image of the office 501 or shopping mall 601 where the user is present.

[0091] Third Example

[0092] According to an embodiment of the present invention, it can be assumed that a first user (specifically, the first user device 310) provides shared content, and the shared content is shared in real time with a second user (specifically, the second user device 320) and a third user (specifically, the third user device 330). Specifically, the first user, the second user, and the third user may chat in a virtual space generated based on space information related to the shared content provided by the first user.

[0093] First, referring to FIG. 7 and FIG. 8, a first user (specifically, a first user device 310) may generate shared content with the Eiffel Tower in Paris, France as a background.

[0094] In addition, a virtual space can be generated by referring to the spatial information 710, 720 regarding the shared content.

[0095] For example, a point cloud relating to a location related to the shared content may be identified based on spatial information 710, 720 related to the shared content, and the virtual space may be generated based on the identified point cloud (e.g., to match at least one of the shape, size, position, boundary, and orientation of multiple point clouds including the identified point cloud).

[0096] Next, referring to Figures 10 and 11, spatial connection objects 1010, 1110 for providing a user experience of spatially connecting the virtual space with the locations where the second and third users (specifically, the second user device 320 and the third user device 330) are present may be displayed in an augmented manner against the background of the locations where the second and third users are present, respectively (1000(a), 1100(a)).

[0097] Meanwhile, the spatial connection objects 1010 and 1110 may be generated only when it is determined that the second and third users (specifically, the second user device 320 and the third user device 330) are present in a location that meets a predetermined trigger condition.

[0098] For example, referring to FIG. 9, when a second user (specifically, a second user device 320) is present in a corner 910 of a room that can face six directions within a predetermined distance, such a spatially connected object may be generated.

[0099] Then, when the second and third users approach or zoom in on the spatial connection objects 1010 and 1110, the virtual space can be displayed to the second and third users, respectively (1000(b) and 1100(b)).

[0100] Next, when the first authored content 1020, 1030 is generated by the second user in the virtual space (1000(c)), the first authored content 1020, 1030 can be shared based on a location that matches the location where the first authored content 1020, 1030 is generated in the virtual space (1100(c), 800(b)).

[0101] Specifically, the first authored contents 1020, 1030 may be shared based on a position in the virtual space of the third user that matches the position where the first authored contents 1020, 1030 are generated in the virtual space (1100(c)). Also, the first authored contents 1020, 1030 may be shared based on a real-world position that matches the position where the first authored contents 1020, 1030 are generated in the virtual space (800(b)).

[0102] Next, when a second-authored content 1120 is generated by a third user in the virtual space (1100(d)), the second-authored content 1120 can be shared based on a location that matches the location where the second-authored content 1120 is generated in the virtual space (1000(d), 800(c)).

[0103] Specifically, the second authored content 1120 may be shared based on a position in the virtual space of the second user that matches with a position where the second authored content 1120 is generated in the virtual space (1000(d)). Also, the second authored content 1120 may be shared based on a real-world position that matches with a position where the second authored content 1120 is generated in the virtual space (800(c)).

[0104] Fourth Example

[0105] According to one embodiment of the present invention, a situation can be assumed in which shared content is provided by a first user (specifically, a first user device 310) and the shared content is shared (i.e., the shared content is provided to the second user) with a second user (specifically, a second user device 320).

[0106] 17A, a shared content is provided from a first user (specifically, the first user device 310) with the Eiffel Tower in Paris, France in the summer of 2018 as a background (e.g., a real-world space where the first user was present), and a virtual space (such a virtual space may include various natural objects and man-made objects in the vicinity including the Eiffel Tower) may be generated by referring to spatial information related to the shared content. In order to generate such a virtual space, data photographed by the first user (specifically, the first user device 310) or point cloud data of the corresponding space (e.g., point cloud data of the corresponding space at the corresponding time) identified based on the position of the first user device 310 may be used.

[0107] Next, referring to FIG. 17B, when the shared content (or virtual space) is provided to a second user (specifically, a second user device 320) who is present in the bedroom as of winter 2020, the virtual space related to the shared content may be corrected (or regenerated) by referring to the time information at the time the shared content is provided to the second user.

[0108] For example, the time information when the virtual space related to the shared content was generated is summer 2018, so there are leaves on the trees constituting the virtual space related to the shared content, but the time information when the shared content is provided to the second user is winter 2020, so the trees constituting the virtual space related to the shared content may be corrected so that there are no leaves on the trees. Meanwhile, it should be clarified that the subject of such correction is not necessarily limited to only the components constituting the virtual space, and may also be applied to copyrighted content generated within the virtual space within the scope of achieving the object of the present invention.

[0109] Meanwhile, according to one embodiment of the present invention, when a second user specifies (or searches) predetermined time information, a virtual space related to the shared content may be corrected (or regenerated) with reference to the time information specified by the second user, and the shared content may be provided to the second user in the corrected (or regenerated) virtual space. Here, the time information specified by the second user may include a past or future time based on the time information at the time the virtual space was generated (i.e., a virtual time capsule, virtual time log, etc. related to the shared content).

[0110] According to an embodiment of the present invention, the shared content may be provided to the second user in a virtual space that changes with time based on time information specified by the second user. For example, if the time information specified by the second user is spring 2019, the shared content may be provided to the second user in a virtual space that changes with time from spring 2019.

[0111] Meanwhile, according to one embodiment of the present invention, a spatial connection object (e.g., a portal) that spatially connects the virtual space and the bedroom where the second user is present (i.e., the real-world space) may be augmented and displayed against the background of the bedroom where the second user is present.

[0112] Meanwhile, according to an embodiment of the present invention, when a second user specifies (or searches) a certain time information, at least one authored content (e.g., a dialogue message; such a dialogue message may be generated in a virtual space corresponding to the specified time information) may be shared with the second user in a virtual space determined based on the specified time information. Here, when the virtual space changes chronologically based on the time information specified by the second user, the authored content (e.g., a dialogue message) generated at each time point may be shared sequentially in the virtual space according to the flow of time. Also, when an additional authored content (e.g., a dialogue message) is generated by the second user in the corresponding virtual space, it may be shared with other users based on a location matching a location where the additional authored content is generated in the corresponding virtual space and the time information of the corresponding virtual space (here, the additional authored content may not exist in a virtual space corresponding to time information preceding the time information where the additional authored content is generated. That is, the authored content may be selectively or dynamically shared with reference to the time information).

[0113] The above-described embodiments of the present invention may be embodied in the form of program instructions that can be executed by various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the computer-readable recording medium may be those specially designed and constructed for the present invention, or those that are known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions, such as ROMs, RAMs, flash memories, and the like. Examples of program instructions include not only machine language codes, such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, and the like. The hardware devices may be replaced by one or more software modules to perform the processes of the present invention, and vice versa.

[0114] Although the present invention has been described above using specific details such as specific components and limited examples and drawings, this is merely provided to assist in a more general understanding of the present invention, and the present invention is not limited to the above examples. Those skilled in the art to which the present invention pertains may make various modifications and changes from such descriptions.

[0115] Therefore, the concept of the present invention should not be limited to the above-described embodiments, and all scopes equivalent to or equivalently modified from the scope of the claims described below, as well as the scope of the invention, fall within the scope of the concept of the present invention. [Explanation of symbols]

[0116] 100:Communication network 200: Support system 210: Spatial information acquisition section 220: Virtual space determination unit 230: Contents Provider 240:Shared management department 250: Communications Department 260: Control unit 300:Device

Claims

1. A method for supporting sharing of experiences between users, comprising: The computer acquiring spatial information about a shared content that is the subject of a shared experience; determining a virtual space for providing the shared content to the user by referring to spatial information related to the shared content and time information related to the user; When the authored content is generated by the user in the virtual space, the authored content is shared based on a location that matches a location in the virtual space where the authored content is generated; Run The authored content may be shared with another user present in the virtual space at a time different from the time associated with the user; A method in which, when the virtual space changes chronologically with reference to time information related to the user, each copyrighted content can be sequentially shared with the other users in the virtual space according to the time at which each copyrighted content was generated by the user.

2. The method of claim 1 , wherein the virtual space is generated with reference to spatial information related to the shared content and is corrected with reference to time information related to the user.

3. The method according to claim 2 , wherein at least one component constituting the virtual space is corrected by referring to time information related to the user.

4. The method of claim 1 , wherein the time information associated with the user corresponds to time information at the time the shared content is provided to the user or time information specified by the user.

5. The method of claim 1 , wherein the authored content is selectively shared with reference to point-in-time information associated with the user.

6. A non-transitory computer-readable recording medium having a computer program recorded thereon for carrying out the method according to claim 1.

7. A system for supporting sharing of experiences between users, comprising: a spatial information acquisition unit that acquires spatial information regarding shared content that is a target of sharing an experience; a virtual space determination unit that determines a virtual space for providing the shared content to the user by referring to spatial information on the shared content and time information related to the user; a sharing management unit for allowing the authored content to be shared based on a location that matches a location where the authored content is generated in the virtual space when the authored content is generated by the user in the virtual space; Equipped with The authored content may be shared with another user present in the virtual space at a time different from the time associated with the user; A system in which, when the virtual space changes chronologically with reference to time information related to the user, each piece of authored content can be sequentially shared with the other users in the virtual space according to the time at which each piece of authored content was generated by the user.

8. The system of claim 7 , wherein the virtual space is generated with reference to spatial information related to the shared content and is corrected with reference to time information related to the user.

9. The system according to claim 8 , wherein at least one component constituting the virtual space is corrected by referring to time information related to the user.

10. The system of claim 7 , wherein the time information associated with the user corresponds to time information at the time the shared content is provided to the user or time information specified by the user.

11. The system of claim 7 , wherein the authored content is selectively shared with reference to point-in-time information associated with the user.

Images