Method and apparatus for controlling a user terminal based on the user terminal being located in a predefined custom area

Abstract

A method for controlling a user terminal is provided, which determines whether the user terminal is located in a preset custom area in a space based on the position of the user terminal, and controls the user terminal based on a service element when it is determined that the user terminal is located in the custom area.

Description

[Technical Field]

[0001] The following description relates to a method and apparatus for controlling a user terminal based on the location of the user terminal, including defining a custom area associated with a service element on a spatial map and providing content to the user terminal based on the user terminal being located in the custom area. [Background technology]

[0002] Augmented reality (AR) is a technology that complements the real world by blending virtual objects and information generated by computer technology. In other words, AR is a technology that displays virtual content in an extended manner in the real environment, allowing users to view the augmented content in response to the real environment through an electronic device.

[0003] The user terminal, which is a user's electronic device, is provided with content related to objects located within a certain distance or radius based on the location of the user terminal, which is augmented to the user terminal's video. If augmented content is provided based solely on the physical distance from the user terminal, more content than necessary may be augmented, increasing the user's cognitive load and the load on the user terminal. Furthermore, content that does not match the service intended by the user may be augmented to the user terminal.

[0004] As an example of technology related to various services that utilize augmented reality technology, Korean Patent Publication No. 10-2014-0065963 (published on May 30, 2014) discloses an augmented reality navigator that is installed in a vehicle, displays an image of the driving route captured by a camera on a display, and maps virtual display information that guides the driver along the driving route onto the image displayed on the display.

[0005] The information above is intended merely to aid in the understanding of the present invention and may contain material that does not form part of the prior art, or that the prior art would not present to one of ordinary skill in the art. Summary of the Invention [Problem to be solved by the invention]

[0006] A method for controlling a user terminal can be provided that determines whether the user terminal is located in a pre-set custom area in a space, and controls the user terminal, such as displaying AR content, providing a user interface (UI), and controlling settings of the user terminal, based on service elements associated with the custom area.

[0007] A custom area can be defined as a two-dimensional or three-dimensional space on a spatial map, and a service element can be associated with the custom area to provide services or information associated with the space based on the service element, or a scenario condition can be defined to control the settings of the user terminal, thereby generating a custom area for controlling the user terminal according to the scenario condition.

[0008] A method can be provided for controlling a user terminal that enters or exits a custom area, such that different content is provided to the user terminal, different UI is provided to the user terminal, or different setting control of the user terminal is provided, according to the scenario conditions defined in the custom area. [Means for solving the problem]

[0009] According to one aspect, there is provided a method for controlling a user terminal, which is executed by a computer system that is a server or a user terminal, and includes a step of determining whether the user terminal is located in a pre-set custom area in a space based on the location of the user terminal (the custom area is associated with at least one service element), and a step of controlling the user terminal based on the service element upon determining that the user terminal is located in the custom area.

[0010] The custom area may be defined on the spatial map to include at least one of a two-dimensional space and a three-dimensional space, and the step of controlling the user terminal may include controlling the user terminal based on the service element when the user terminal is located in the custom area on the map.

[0011] The custom area may include a three-dimensional space having a two-dimensional area and a set height range, and the step of controlling the user terminal may include determining that the user terminal is located in the custom area when the user terminal is located in the two-dimensional area and the height of the user terminal falls within the set height range, and controlling the user terminal based on the service element.

[0012] The custom area defines 3DoF (Degree Of Freedom) or 6DoF attitude conditions for the user terminal, and the step of controlling the user terminal may include controlling the user terminal based on the service elements when it is determined that the user terminal is located in the custom area and the attitude of the user terminal satisfies the attitude conditions.

[0013] The custom area has at least one condition defined therein: a first condition indicating a time condition, a second condition indicating a movement of the user terminal, and a third condition indicating an operation on the user terminal; and the step of controlling the user terminal may include controlling the user terminal based on the service element when it is determined that the user terminal is located in the custom area and the user terminal satisfies the defined condition.

[0014] The service element may include data for displaying AR (augmented reality) content related to a service associated with the space, and the step of controlling the user terminal may include a step of displaying the AR content through an AR view including an image captured by a camera of the user terminal based on the service element when the user terminal is located in the custom area.

[0015] The AR content may be associated with a first object located in the custom area and displayed in the AR view corresponding to the custom area, or may be associated with a second object located in a remote area relative to the custom area and displayed in the AR view corresponding to the remote area.

[0016] The AR content may include a plurality of sequences, and the step of displaying the AR content may include, when it is determined that the user terminal is located in the custom area, displaying different sequences of the AR content through the AR view depending on the position of the user terminal within the custom area.

[0017] The step of displaying the AR content may include a step of changing and displaying the AR content displayed through the AR view or outputting predetermined additional content based on at least one of the position of the user terminal within the custom area, the number of times the user terminal has entered the custom area, and the entry / exit status of the user terminal in the custom area.

[0018] When the user terminal enters the custom area and when the user terminal leaves the custom area, different AR contents may be displayed as the AR content through the AR view.

[0019] The service element may include data for displaying map element content related to a service associated with the space, and the step of controlling the user terminal may include a step of displaying the map element content through a map view including a map including the location of the user terminal based on the service element when the user terminal is located in the custom area.

[0020] The map element content may be associated with a first object located in the custom area and displayed in an area of ​​the map view corresponding to the custom area, or may be associated with a second object located in a remote area relative to the custom area and displayed in an area of ​​the map view corresponding to the remote area.

[0021] The service element may include data for displaying a user interface (UI) related to a service associated with the space, and the step of controlling the user terminal may include a step of displaying the UI on the user terminal based on the service element when the user terminal enters the custom area or when the user terminal leaves the custom area.

[0022] The service element includes data for controlling settings of the user terminal to provide the user terminal with a service associated with the space, and the step of controlling the user terminal may include controlling settings of the user terminal based on the service element by the user terminal entering the custom area or the user terminal leaving the custom area.

[0023] Controlling the settings of the user terminal may include running or stopping a module for recognizing objects in the space or the custom region.

[0024] The custom area defines scenario conditions for providing the user terminal with services or information associated with the space based on the service elements or for controlling the settings of the user terminal, and the step of controlling the user terminal may include controlling the terminal in accordance with the scenario conditions so that the user terminal is provided with services or information associated with the space based on the service elements or the settings of the user terminal are controlled.

[0025] A plurality of custom areas are set in the space, and in the custom area, the scenario conditions may be defined such that, depending on whether at least one of i) a first scenario condition indicating whether the user terminal has passed through a predetermined custom area other than the custom area, and ii) a second scenario condition indicating whether the number of user terminals located in the custom area falls within a predetermined range, the user terminal is provided with different services or information associated with the space based on the service elements, or the settings of the user terminal are controlled differently.

[0026] There may be a plurality of service elements associated with the custom area, each of the plurality of service elements being associated with a different point within the custom area, and within the custom area, the scenario conditions may be defined such that services or information associated with the space are provided differently depending on the order in which the user terminal passes through the points associated with each of the service elements.

[0027] The method for controlling the user terminal further includes a step of receiving, at the user terminal, a request for route guidance to a destination, and a step of providing route guidance to the user terminal according to the route guidance request, and the step of determining and the step of controlling the user terminal may be performed when the user terminal moves to the destination according to the route guidance.

[0028] In another aspect, a computer system for implementing a user terminal or a server is provided, the computer system including at least one processor configured to execute computer-readable instructions contained in a memory, the at least one processor determining, based on a position of the user terminal, whether the user terminal is located in a pre-defined custom area in a space (the custom area being associated with at least one service element), and controlling the terminal based on the service element upon determining that the user terminal is located in the custom area. [Effects of the Invention]

[0029] A custom area can be defined as a two-dimensional or three-dimensional space on a spatial map, and a service element can be associated with the custom area to provide a service or information associated with the space based on the service element, or a scenario condition can be defined to control the settings of the user terminal, thereby generating a custom area for controlling the user terminal according to the scenario condition.

[0030] By setting a custom area while taking into consideration the occlusion area of ​​the user terminal in the space and displaying AR content on the user terminal depending on whether the user enters or exits the custom area, an AR view for displaying AR content with better performance on the user terminal can be realized.

[0031] By associating individual service elements with custom regions, even if custom regions that overlap each other in space are defined, these custom regions can still correctly control the user terminal.

[0032] By defining the custom area as a three-dimensional space, scenario conditions that take into account the height and posture of the user device can be utilized when determining whether the user device is entering or exiting the custom area and determining control of the user device.

[0033] For user terminals entering or exiting a custom area, the content provided to the user terminal, the UI provided to the user terminal, or the setting control of the user terminal can be different depending on the scenario conditions defined in the custom area (e.g., conditions based on the relationship with other custom areas and / or the number of user terminals located in the custom area). [Brief explanation of the drawings]

[0034] [Figure 1] FIG. 10 illustrates a method for controlling a user terminal based on the user terminal being located in a pre-defined custom area in one embodiment. [Figure 2] FIG. 1 illustrates a computer system for executing a method for controlling a user terminal in one embodiment. [Figure 3] 10 is a flowchart illustrating a method for determining whether a user terminal is located in a pre-defined custom area and controlling the user terminal in accordance with an embodiment; [Figure 4]1 is a flowchart illustrating a method for controlling a user terminal based on service elements associated with a custom region in one example. [Figure 5] 1 is a flowchart illustrating a method for creating a custom region in one example. [Figure 6] FIG. 1 illustrates a map with custom regions defined, in one example. [Figure 7] FIG. 10 is a diagram illustrating a map on which a custom region is defined and an AR view screen of a user terminal in one example. [Figure 8a] FIG. 8a illustrates a custom region defined in three-dimensional space in one example. [Figure 8b] FIG. 8b is a diagram illustrating the degree of freedom (DoF) of a user terminal in one example. [Figure 9] FIG. 10 illustrates a method for displaying AR content or map element content associated with an object in a remote region on a user terminal based on the user terminal being located in a pre-defined custom region in one example. [Figure 10] FIG. 1 illustrates a method for displaying a user interface (UI) on a user terminal based on the user terminal being located in a pre-defined custom region, in one example. [Figure 11] FIG. 1 illustrates a method for displaying a user interface (UI) on a user terminal based on the user terminal being located in a pre-defined custom region, in one example. [Figure 12] FIG. 1 illustrates a method for displaying route guidance information in a map view of a user terminal in one example. [Figure 13] FIG. 1 illustrates a method for displaying route guidance information in a map view of a user terminal in one example. [Figure 14] FIG. 10 illustrates a custom region including multiple points in one example. [Figure 15] FIG. 10 illustrates a method for controlling user terminal settings based on the user terminal's location in a pre-defined custom area, in one example. [Figure 16] FIG. 10 is a diagram illustrating a method in which multiple customizations are set in a space and scenario conditions for control of a user terminal are set for each custom area in one example. [Figure 17a] 1 illustrates a method for controlling user terminals located in a custom area according to various scenario conditions defined for the custom area, in one example. [Figure 17b] 1 illustrates a method for controlling user terminals located in a custom area according to various scenario conditions defined for the custom area, in one example. [Figure 17c] 1 illustrates a method for controlling user terminals located in a custom area according to various scenario conditions defined for the custom area, in one example. [Figure 17d] 1 illustrates a method for controlling user terminals located in a custom area according to various scenario conditions defined for the custom area, in one example. [Figure 18a] 1 illustrates a method for controlling user terminals located in a custom area according to various scenario conditions defined for the custom area, in one example. [Figure 18b] 1 illustrates a method for controlling user terminals located in a custom area according to various scenario conditions defined for the custom area, in one example. [Figure 18c] 1 illustrates a method for controlling user terminals located in a custom area according to various scenario conditions defined for the custom area, in one example. [Figure 19a] 1 illustrates a method for controlling user terminals located in a custom area according to various scenario conditions defined for the custom area, in one example. [Figure 19b] 1 illustrates a method for controlling user terminals located in a custom area according to various scenario conditions defined for the custom area, in one example. [Figure 19c]1 illustrates a method for controlling user terminals located in a custom area according to various scenario conditions defined for the custom area, in one example. [Figure 20] 1 illustrates a method for controlling user terminals located in a custom area according to various scenario conditions defined for the custom area, in one example. [Figure 21a] 1 illustrates a method for controlling user terminals located in a custom area according to various scenario conditions defined for the custom area, in one example. [Figure 21b] 1 illustrates a method for controlling user terminals located in a custom area according to various scenario conditions defined for the custom area, in one example. [Figure 21c] 1 illustrates a method for controlling user terminals located in a custom area according to various scenario conditions defined for the custom area, in one example. DETAILED DESCRIPTION OF THE INVENTION

[0035] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings.

[0036] FIG. 1 illustrates a method for controlling a user terminal based on the user terminal being located in a pre-defined custom area in one embodiment.

[0037] In the embodiments described below, "control" of the user terminal 100 may include, for example, as shown in FIG. 1 , at least one of using the AR view 10 of the user terminal 100 to augment and display content (e.g., AR content 30) indicating services or information associated with the space 5 in the image of the AR view 10; displaying a user interface (UI) related to services associated with the space 5 on the user terminal 100; displaying map element content 35 related to services associated with the space 5 on a map (or map view 20) of the user terminal 100; and controlling settings of the user terminal 100 to provide the user terminal 100 with services associated with the space 5. However, "control" of the user terminal in the embodiments of the present disclosure should not be limited to the above-described examples, and may include controlling the functions and / or operations of the user terminal.

[0038] In the embodiments, the "space 5" associated with the provision of augmented content may refer to an indoor space and / or an outdoor space. In other words, the space 5 described below may be defined as a space including an indoor location or an outdoor location, or may be defined as a composite space that combines an indoor space and an outdoor space. The space 5 may be at least a part of a service target area for providing services including the provision of content.

[0039] In the detailed description below, for convenience of explanation, the location of a user who possesses the user terminal 100 may be described as the location of the user terminal 100. Also, for convenience of explanation, the terms "user" and "user terminal 100" possessed by the user may be used interchangeably.

[0040] In the detailed description below, "extending" content and / or information (e.g., indicators, etc.) onto a video may be interpreted to encompass overlaying the content and / or information onto the video / screen of the AR view, depending on the embodiment.

[0041] As shown in the figure, a user terminal 100 such as a smartphone may capture images of the surroundings using a camera and display AR content 30 through an AR view 10 including the image captured by the camera. The AR content 30 may be associated with an object present in a space 5. Such AR content 30 may be mapped to the object (or a node representing the object).

[0042] At least one custom area may be defined on the map of the space 5 shown in the figure (or other data representing the space 5). In the example shown in the figure, two areas, a first custom area 60 and a second custom area 70, are defined. At least one service element may be associated with each of the custom areas 60 and 70. Furthermore, a scenario condition may be defined for each of the custom areas 60 and 70 to provide a service or information associated with the space 5 or to control the settings of the user terminal 100 based on the corresponding service element. The service or information may include information for advertising or marketing.

[0043] The "custom areas 60, 70" may be preset on the server 200. For example, at least a part of the space 5 or a partitioned partial area of ​​the space 5 may be set as the "custom areas 60, 70" on the server 200.

[0044] The method of defining each custom area 60, 70 and associating service elements and scenario conditions with the custom area 60, 70 to configure (create) the custom area 60, 70 will be described in more detail with reference to FIG.

[0045] In the embodiment, it may be determined whether the user terminal 100 is located in a first custom area 60 within the space 5 based on the location of the user terminal 100. If it is determined that the user terminal 100 is located in the first custom area 60, AR content 30 may be displayed in the AR view 10 of the user terminal 100 based on a service element associated with the custom area 60. For example, when the user terminal 100 enters the first custom area 60, AR content 30 associated with an object (object) "metal type" may be displayed in the AR view 10. At this time, a current location 50 of the user terminal 100 may be displayed in the map view 20. The current location 50 may correspond to a position 50' in the space 5. In the illustrated embodiment, when the user terminal 100 enters the first custom area 60, an AR content 30 associated with a “metal type” that is an object existing outside the first custom area 60 is displayed. However, depending on the service element associated with the first custom area 60, when the user terminal 100 enters the first custom area 60, other AR content associated with an object existing inside the first custom area 60, such as the “Epitaph of Princess Suyeong,” may be displayed.

[0046] On the other hand, in other embodiments, based on the location of the user terminal 100, other types of content may be displayed in the AR view 10, an appropriate UI may be provided, appropriate map element content may be displayed, or appropriate settings of the user terminal 100 may be changed according to scenario conditions defined in the first custom area 60, such as when the user terminal 100 leaves the first custom area 60.

[0047] The map view 20 displayed together with the AR view 10 on the screen of the user terminal 100 may include a map that matches the image of the AR view 10. The map view 20 may include a current location 50 of the user terminal 100. The map view 20 may include a two-dimensional map or a three-dimensional map. The map view 20 may include objects in the space 5 through which the user terminal 100 moves. The map view 20 may display map element content 35 corresponding to the AR content 30. The map element content 35 may be displayed when the AR content 30 is displayed in the AR view 10 or when an object associated with the AR content 30 is included in the map view 20.

[0048] The user terminal 100 may communicate with the server 200 for control upon entering / exiting the custom areas 60, 70. For example, the server 200 may cause the user terminal 100 to be controlled based on a service element associated with the custom area 60, 70 in which the user terminal 100 is located.

[0049] The server 200 may be a computer system that manages the custom areas 60, 70 defined for the space 5, and may perform management such as associating service elements with the custom areas 60, 70 and defining scenario conditions. The server 200 may also be a map server that provides digital maps (3D maps and / or 2D maps).

[0050] Specific methods for controlling the user terminal 100 by the user terminal 100 and / or the server 200 according to the user terminal 100's position in the custom areas 60, 70 and / or entry / exit into / exit from the custom areas 60, 70, and specific methods for generating (setting) the custom areas 60, 70 will be described in further detail with reference to Figures 2 to 16.

[0051] FIG. 2 illustrates a computer system that executes a method for controlling a user terminal in one embodiment.

[0052] The user terminal 100 according to the embodiment described with reference to Fig. 1 may be realized by the computer system 2 shown in the figure. For example, a computer program for realizing the method according to the embodiment may be installed and executed in the computer system 2, and such computer system 2 may execute the enhanced content providing method according to the embodiment under the control of the executed computer program.

[0053] The method for controlling the user terminal 100 according to the embodiment may be executed through a PC-based program or a dedicated application for a mobile terminal. For example, the method for controlling the user terminal 100 according to the embodiment may be implemented in the form of an independently operating program, or may be implemented in the form of an in-app for a specific application so as to be operable on the specific application. For example, such a specific application may be installed on the computer system 2 and provide AR content 30 through the AR view 10 to execute the method for controlling the user terminal 100 according to the embodiment.

[0054] The computer system 2 is an electronic device corresponding to the above-mentioned user terminal 100, and may be a smartphone or similar device capable of installing and executing an application or program, as shown in Fig. 1. The computer system 2 may also be a personal computer (PC), a laptop computer, a tablet, an Internet of Things (IoT) device, a wearable computer, or the like.

[0055] As shown in the figure, the computer system 2 may include a memory 210, a processor 220, a communication interface 230, and an input / output interface 240 as components for executing the method for controlling the user terminal 100.

[0056] Memory 210 is a computer-readable recording medium and may include random access memory (RAM), read-only memory (ROM), and a persistent mass storage device such as a disk drive. Here, a persistent mass storage device such as a ROM or a disk drive may be included in computer system 2 as a separate persistent storage device distinct from memory 210. Memory 210 may also store an operating system and at least one program code. Such software components may be loaded into memory 210 from a computer-readable recording medium separate from memory 210. Such separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, or a memory card. In other embodiments, software components may be loaded into memory 210 through communication interface 230, which is not a computer-readable recording medium. For example, software components may be loaded into memory 210 of computer system 2 based on a computer program installed by a file received via network 170.

[0057] Processor 220 may be configured to process computer program instructions by performing basic arithmetic, logic, and input / output operations. The instructions may be provided to processor 220 by memory 210 or by communication interface 230. For example, processor 220 may be configured to execute instructions received according to program code stored in a storage device such as memory 210.

[0058] That is, the processor 220 can manage the components of the computer system 2 and execute programs or applications used by the computer system 2. For example, the processor 220 may be configured to execute an application for executing the method for controlling the user terminal 100 of the embodiment and process data received from the server 200 to control the user terminal 100 (e.g., data received from the server 200 to control the AR content 30 through the AR view 10). The processor 220 may also process calculations required for executing programs or applications and processing data, and may be at least one processor of the computer system 2 or at least one core within a processor.

[0059] The communication interface 230 may provide a function for the computer system 2 to communicate with other computer systems (not shown) via the network 170. For example, requests, instructions, data, files, etc. generated by the processor 220 of the computer system 2 in accordance with program code stored in a storage device such as the memory 210 may be transmitted to the other computer systems via the network 170 under the control of the communication interface 230. Conversely, signals, instructions, data, files, etc. from the other computer systems may be received by the computer system 2 via the communication interface 230 of the computer system 2 via the network 170. The signals, instructions, data, etc. received via the communication interface 230 may be transmitted to the processor 220 or the memory 210, and files, etc. may be recorded on a storage device (the above-mentioned permanent storage device) that the computer system 2 may further include. For example, the communication interface 230 may be a hardware module such as a network interface card, a network interface chip, or a networking interface port of the computer system 2, or a software module such as a network device driver or a networking program.

[0060] The input / output interface 240 may be a means for interfacing with the input / output device 250. For example, the input device may include a device such as a microphone, keyboard, or mouse, and the output device may include a device such as a display or speaker. As another example, the input / output interface 240 may be a means for interfacing with a device that integrates input and output functions into one, such as a touch screen. The input / output device 250 may be configured as a single device together with the computer system 2.

[0061] Also, in other embodiments, computer system 2 may include more components than those shown in FIG. 2 . However, it is not necessary to explicitly illustrate most of the conventional components. For example, computer system 2 may be implemented to include at least some of the input / output devices connected to input / output interface 240 described above, and may further include other components such as a transceiver, a GPS (Global Positioning System) module, a camera, various sensors, a database, etc. As a more specific example, if computer system 2 is implemented in the form of a mobile device such as a smartphone, computer system 2 may further include various components typically found in mobile devices, such as a camera, an acceleration sensor, a gyro sensor, various physical buttons, buttons using a touch panel, input / output ports, and a vibrator for vibration.

[0062] For example, the computer system 2 corresponding to the user terminal 100 may include a camera that captures the surroundings to execute the AR view 10. The computer system 2 may display the image captured by the camera as the AR view 10, and may display the AR content 30 by extending the image of the AR view 10. For example, the computer system 2 may Whether the user terminal 100 is located in a pre-set custom area 60, 70 may be determined by positioning, and if the user terminal 100 is located in the custom area 60, 70, AR content 30 based on a service element associated with the custom area 60, 70 may be displayed in the AR view 10. A method using visual localization or a Visual Positioning System (VPS) may be used to position the user terminal 100. Alternatively, a method using marker recognition, a method using GPS, a Wi-Fi Positioning System (WPS), a Bluetooth module, an odometry, or the like may be used to position the user terminal 100.

[0063] On the other hand, the actual operation of determining whether computer system 2 is located in custom area 60, 70 may be performed by server 200 rather than computer system 2. Server 200 may be an electronic device that communicates with computer system 2 and provides data for controlling user terminal 100 to computer system 2, thereby controlling user terminal 100. Server 200 is a device that manages (registers, modifies, deletes, saves, maintains, etc.) the data and content provided to computer system 2 and custom areas 60, 70, and may include a database or communicate with such a database. Server 200 may also be or include a map server that provides digital maps (3D maps and / or 2D maps).

[0064] The server 200 may include at least one computer system. The computer system included in the server 200 may have a configuration similar to that of the computer system 2 described above, and a duplicated description thereof will be omitted.

[0065] In an embodiment, the user terminal 100, which is the computer system 2, may communicate with the server 200 and be controlled based on data and / or information provided by the server 200. In this case, the server 200 may receive the location of the user terminal 100, and may determine whether the user terminal 100 is located within a custom area 60, 70 based on the received location. If the user terminal 100 is located within the custom area 60, 70, the server 200 may control the user terminal 100 based on the service elements associated with the custom area 60, 70.

[0066] In the detailed description below, for convenience of explanation, the embodiment will be described focusing on the computer system 2 corresponding to the user terminal 100, and explanations of communication with the server 200 and operations on the server 200 side may be simplified or omitted. Also, for convenience of explanation, operations (steps) performed by the components (e.g., processor, etc.) of the computer system 2 (or the server 200) may be described as being performed by the computer system 2 (or the server 200).

[0067] The explanation of the technical features given above with reference to FIG. 1 can be applied to FIG. 2 as well, so duplicate explanations will be omitted.

[0068] FIG. 3 is a flowchart illustrating a method for determining whether a user terminal is located in a pre-defined custom area and controlling the user terminal in accordance with the determined method in one embodiment.

[0069] 3, a method for controlling a user terminal 100 executed by a computer system 2 will be described. The computer system 2 may be the user terminal 100 described with reference to FIG. 1, but in the following detailed description, the embodiment will be described using the terms user terminal 100 and server 200 instead of computer system 2.

[0070] Furthermore, steps 310 to 340 described below and at least some of the steps described with reference to Fig. 4 may be configured to be executed by the server 200 rather than the user terminal 100. Below, the steps executed mainly by the user terminal 100 will be described, and redundant descriptions of the server 200 may be omitted.

[0071] In step 330, the user terminal 100 or the server 200 may determine, based on the location of the user terminal 100, whether the user terminal 100 is located in a pre-defined custom area 60, 70 within the space 5. The custom area 60, 70 may be associated with at least one service element. The user terminal 100 may determine the location of the user terminal 100, for example, by a method based on at least one of GPS, a Visual Positioning System (VPS), and a Wi-Fi Positioning System (WPS), and may determine, based on the determined location, whether the user terminal 100 is located in the custom area 60, 70, or has entered or left the custom area 60, 70. Alternatively, the server 200 may receive the determined location of the user terminal 100 and, based on the received location, determine whether the user terminal 100 is located in the custom area 60, 70, or has entered or left the custom area 60, 70.

[0072] In step 340, the user terminal 100 or the server 200 may determine that the user terminal 100 is located in the custom area 60, 70 (including the user terminal 100 leaving the custom area 60, 70) and thereby control the user terminal based on the service elements associated with the custom area 60, 70. That is, the user terminal 100 may be controlled to perform a specific operation based on the data included in the service elements associated with the custom area 60, 70.

[0073] "Controlling" the user terminal 100 may include, for example, as shown in FIG. 1, at least one of using the AR view 10 of the user terminal 100 to augment and display content (e.g., AR content 30) indicating services or information associated with the space 5 in the image of the AR view 10; displaying a user interface (UI) related to services associated with the space 5 on the user terminal 100; displaying map element content 35 related to services associated with the space 5 on the map (or map view 20) of the user terminal 100; and controlling the settings of the user terminal 100 to provide the services associated with the space 5 to the user terminal 100.

[0074] In an embodiment, the custom areas 60, 70 may have scenario conditions defined therein for providing the user terminal 100 with services or information associated with the space 5 or for controlling the settings of the user terminal 100 based on the service elements associated with the custom areas 60, 70. The scenario conditions may include scenario conditions for providing services or information associated with the space 5 or for controlling the settings of the user terminal 100.

[0075] In step 340 described above, the user terminal 100 or the server 200 may control the user terminal 100 so that services or information associated with the space 5 are provided to the user terminal 100 based on the service elements associated with the custom areas 60, 70 in accordance with the scenario conditions defined for such custom areas 60, 70.

[0076] A specific control method of the user terminal 100 based on the service elements associated with the custom areas 60 and 70 will be described in more detail with reference to FIGS.

[0077] Meanwhile, the method for controlling the user terminal 100 according to the embodiment may be executed in combination with a route guidance service. For example, the method for controlling the user terminal 100 according to the embodiment may be executed in combination with AR navigation that provides route guidance indoors or outdoors.

[0078] In step 310, the user terminal 100 may receive a request for route guidance to a destination. For example, a user of the user terminal 100 may request route guidance from the user terminal 100 by executing an application or program that provides an AR navigation service or another route guidance service on the user terminal 100 and inputting a destination. The route guidance request may be transmitted to the server 200 that provides the route guidance service.

[0079] In step 320, the user terminal 100 may provide route guidance in accordance with the route guidance request received in step 310. For example, the user terminal 100 may receive data for route guidance to a destination from the server 200 that provides a route guidance service, and may display information for route guidance to a destination (e.g., guide information) based on the received data.

[0080] The user of the user terminal 100 may move to the destination according to the route guidance. In this case, the above-described steps (including step 310 and step 320) may be executed when the user terminal 100 moves to the destination according to the route guidance.

[0081] For example, when the user terminal 100 moves to a destination according to route guidance, it may be determined whether the user terminal 100 is located in a custom area 60, 70, and based on the determination result, the user terminal 100 may be controlled based on the service elements associated with the custom area 60, 70.

[0082] As a result, the user terminal 100 can be provided with not only information for route guidance to the destination, but also additional information or services regarding the space or custom area 60, 70 upon entering the pre-set custom area 60, 70.

[0083] In this regard, FIGS. 12 and 13 illustrate a method for displaying route guidance information in a map view of a user terminal in one example.

[0084] 13 shows an example in which information for route guidance is displayed on the map view 20 of the user terminal 100. The information for route guidance may include a current location 1310 of the user terminal 100, a destination location 1330, and a route 1320 along which the user terminal 100 should travel toward the destination. The destination location 1330 may be displayed only if the destination is included in the map view 20.

[0085] When the user terminal 100 moves to a destination according to information for route guidance and the user terminal 100 comes to be located in a custom area 60, 70, the user terminal 100 may be controlled based on the service elements associated with the custom area 60, 70. For example, when the user terminal 100 enters the custom area 60, 70, AR content 30 may be displayed in the AR view 10 based on the service elements associated with the custom area 60, 70, or predetermined map element content may be displayed in the map view 20 based on the service elements.

[0086] On the other hand, FIG. 12 shows an example in which information for route guidance is displayed on the AR view 10 of the user terminal 100.

[0087] 12 shows an example in which guide information 1200 is displayed as information for route guidance on the AR view 10. The guide information 1200 may include information to be referred to in order to reach the destination, and may include, for example, as shown in the figure, information regarding a location (third floor) to which the user should move in order to reach the destination from the user's current location (first floor).

[0088] On the other hand, the guide information 1200 may be an example of content that is displayed on the user terminal 100 based on service elements associated with the predetermined custom areas 60 and 70 .

[0089] In other words, when the user terminal 100 receiving route guidance to the destination enters the custom area 60, 70, which is the area near the elevator, as shown in the figure, the user terminal 100 may display guide information 1200 for traveling to the destination as information (or content) about boarding the elevator based on the service elements associated with that custom area.

[0090] Meanwhile, the content displayed on the user terminal 100 based on the service elements when the terminal 100 enters the custom area 60, 70 may differ depending on the scenario conditions defined in the custom area 60, 70. For example, when the user terminal 100 enters the custom area 60, 70, if the user terminal 100 has requested route guidance and is moving to a destination, guide information 1200 for moving to the destination may be displayed on the user terminal 100. However, if the user terminal 100 has not requested route guidance, general information related to the space or the service provided in the custom area 60, 70 may be displayed on the user terminal 100. As in the example shown in the figure, if the custom area 60, 70 is an area near an elevator, the general information may include explanatory information describing the elevator or information such as "Please board the elevator."

[0091] Specific examples of how the user terminal 100 is controlled based on the service elements associated with the custom areas 60, 70 and the defined scenario conditions are described in more detail below with reference to the drawings.

[0092] The explanation of the technical features given above with reference to FIGS. 1 and 2 can be applied as is to FIGS. 3, 12, and 13, and therefore duplicate explanations will be omitted.

[0093] FIG. 4 is a flow chart illustrating a method for controlling a user terminal based on service elements associated with a custom region, in one example.

[0094] Referring to steps 410 and 412, a method for providing AR content to the user terminal 100 will be described as an example of controlling the user terminal 100 by positioning the user terminal 100 in the custom areas 60, 70.

[0095] The service elements associated with the custom regions 60, 70 may include data for displaying AR content related to the services associated with the space 5.

[0096] In step 410, when the user terminal 100 is positioned in the custom area 60, 70, the user terminal may display AR content through the AR view 10 based on the service element. The AR content may be, for example, three-dimensional content, as described with reference to FIG. 1 . The AR content may be displayed at or around the location of a real object present in the space 5 to provide information about the real object. For example, if the space 5 is a museum, the real object may be an exhibit displayed in the museum. Alternatively, the real object may be a means of transportation located in the space 5, such as stairs, escalators, elevators, or toilets, signs, labels, information, or the like.

[0097] The AR content displayed on the user terminal 100 may be associated with a first object located in the custom area 60, 70 and displayed in the AR view 10 corresponding to the custom area 60, 70. For example, referring to FIG. 1, when the user terminal 100 enters the first custom area 60, AR content associated with the first object (e.g., "Epitaph of Princess Suyeong") included in the first custom area 60 may be displayed in the AR view 10.

[0098] Alternatively, the AR content displayed on the user terminal 100 may be associated with a second object located in a remote area relative to the custom areas 60 and 70, and may be displayed in the AR view 10 corresponding to the remote area. The remote area may be a custom area different from the custom areas 60 and 70. For example, referring to FIG. 1 , when the user terminal 100 enters a first custom area 60, the AR content 30 associated with a second object ("metal type") included in a remote second custom area 70, rather than the first custom area 60, may be displayed in the AR view 10. When the user terminal 100 enters the first custom area 60, the AR content 30 associated with the second object may be displayed in the direction of the second custom area 70 in the AR view 10. Therefore, in such an embodiment, when the user terminal 100 enters a specific custom area, not only information about objects in the custom area but also information about objects present in the remote area may be displayed as AR content in the AR view 10.

[0099] The AR content displayed on the user terminal 100 may include an animation or multiple sequences.

[0100] If the AR content includes multiple sequences, when the user terminal 100 displays the AR content based on the service element, when it is determined that the user terminal 100 is located in the custom area 60, 70, the user terminal 100 may display the sequence of the AR content through the AR view 10 depending on the position of the user terminal 100 within the custom area.

[0101] If the AR content includes animation, when the user terminal 100 displays the AR content based on the service element, when it is determined that the user terminal 100 is located in the custom area 60, 70, the user terminal 100 may display different scenes of the animation through the AR view 10 depending on the position of the user terminal 100 within the custom area.

[0102] As an example, the user terminal 100 may display different scenes or different sequences of AR content in the AR view 10 by being positioned at a point indicating a pre-set entrance, a point indicating an exit, or a specific point within the custom area 60, 70, and the custom area 60, 70 may configure scenario conditions so that such different scenes or different sequences are provided to the user terminal 100.

[0103] A scene or sequence may represent a sub-animation of an animation or a sub-sequence of AR content.

[0104] In step 412, the user terminal 100 may reflect and display AR content in the AR view 10 or output additional content according to scenario conditions predefined for the custom areas 60, 70. For example, the user terminal 100 may change and display the AR content displayed through the AR view 10 or output predetermined additional content based on at least one of the position of the user terminal 100 within the custom areas 60, 70, the number of times the user terminal 100 has entered the custom areas 60, 70, and the entry / exit status of the user terminal 100 for the custom areas 60, 70.

[0105] For example, when the user terminal 100 is located at a point indicating a preset entrance, a point indicating an exit, or a specific point within the custom areas 60 and 70, the user terminal 100 may reflect and display AR content at each point or output additional content. The custom areas 60 and 70 may configure scenario conditions for providing the user terminal 100 with the changed AR content or outputting the additional content.

[0106] Furthermore, the user terminal 100 may change and display AR content or output additional content by distinguishing between the first time the user terminal 100 enters the custom area 60, 70 and the nth time the user terminal 100 enters the custom area 60, 70 (n is a natural number greater than or equal to 2). The custom area 60, 70 may configure a scenario condition to provide the changed AR content to the user terminal 100 or output additional content. In this case, the number of times the user terminal 100 enters the custom area 60, 70 may be counted within a predetermined time period (for example, one hour or one day).

[0107] In addition, the user terminal 100 may display different AR content through the AR view 10 or output additional content by distinguishing between when the user terminal 100 enters the custom area 60, 70 and when it leaves the custom area 100. The custom area 60, 70 may provide the changed AR content to the terminal 100 or configure a scenario condition for outputting the additional content. The additional content is content in a form that can be output by the user terminal 100, and may be, for example, audio, text, image, video, etc.

[0108] In this regard, an example of a custom region is shown in Figure 14. Figure 14 shows a custom region that includes multiple points in one example.

[0109] The illustrated custom area 1400 may be an area defined on a map of the space 5. The custom area 1400 includes a point indicating an entrance / exit A and a point indicating an entrance / exit B. Furthermore, the custom area 1400 includes a point indicating an object A and a point indicating an object B. The user terminal 100 or the server 200 may determine whether the user terminal 100 has entered or exited the custom area 1400 by tracking the location of the user terminal 100. For example, the user terminal 100 may be determined to have entered or exited the custom area 60, 70 based on the current location of the user terminal 100 and the route the user terminal 100 has traveled or is traveling. Based on the route, the user terminal 100 or the server 200 may determine what nodes the user terminal 100 has passed through or will pass through, and accordingly determine whether the user terminal 100 has entered or exited the custom area in which it is currently located.

[0110] A plurality of service elements may be associated with the custom area 1400. In this case, each of the plurality of service elements may be associated with a different point within the custom area 1400. A scenario or scenario condition defined in the custom area may be defined such that services or information associated with the space 5 are provided differently depending on the order in which the user terminal 100 passes through the points associated with each service element. The services or information provided differently may be AR content, or may be content in other forms, such as audio, text, images, or video, that can be output by the user terminal 100.

[0111] As an example, a scenario condition may be defined in the custom area 1400 such that different types of content are output from the user terminal 100 when the user terminal 100 enters or exits the custom area 1400.

[0112] Alternatively, scenario conditions may be defined in the custom area 1400 to output different types of content to the user terminal 100, such as when the user terminal 100 enters through a specific point within the custom area 1400 (e.g., at least one of the points of Entrance A, Entrance B, Object A, and Object B shown in the figure) or when the user terminal 100 enters the custom area 1400 after passing through a specific point outside the custom area 1400. For example, when the user terminal 100 enters the custom area 1400 through a point indicating Entrance A, general welcome content may be output to the user terminal 100, and when the user terminal 100 enters the custom area 1400 through a point indicating Entrance B, other content, such as coupon content, may be output to the user terminal 100. The coupon content may include a discount coupon or a special offer that can be used within the space 5.

[0113] Alternatively, a scenario condition may be defined in the custom area 1400 that requests the user terminal 100 to perform an additional action when the user terminal 100 enters the custom area 1400, such as requesting the user terminal 100 to move along the edge of the custom area 1400, and that outputs content in a different form to the user terminal 100 when the action is performed. Such a request for an additional action may also be provided to the user terminal 100 in the form of content or a UI.

[0114] Alternatively, a scenario condition may be defined in the custom area 1400 that counts the number of times the user terminal 100 enters the custom area 1400 and outputs specific content to the user terminal 100 if the user terminal 100 enters the custom area 1400 more than n times (n is a natural number greater than or equal to 2), or that outputs specific content to the user terminal 100 if the user terminal 100 stays in the custom area 1400 for a predetermined time or longer.

[0115] Meanwhile, when the user terminal 100 leaves (i.e., exits) the custom area 1400, all content displayed on the user terminal 100 may be hidden, i.e., the display may be canceled. When the user terminal 100 leaves the custom area 1400, a predetermined notification sound may be output to the user terminal 100. Alternatively, when the user terminal 100 leaves the custom area 1400, AR content may be displayed on the AR view 10 to notify the user that the user terminal 100 has left the custom area 1400.

[0116] In this way, in the embodiment, it is possible to determine whether the user terminal 100 is entering or exiting the custom area 1400, and depending on the determination result, the display of the AR content can be changed (for example, the entry guide AR content can be expanded when entering, and the exit guide AR content can be expanded when exiting), the UI display can be changed, the display of the map element content can be changed, or the setting control of the user terminal 100 can be changed.

[0117] Various scenario conditions defined in the custom area 1400 in relation to the display of AR content will be described in more detail with reference to FIGS.

[0118] Next, referring to step 420, a method for providing map element content to the user terminal 100 will be described as an example of controlling the user terminal 100 depending on the user terminal 100 being located in the custom area 60, 70.

[0119] The service elements associated with the custom regions 60, 70 may include data for displaying map element content related to the services associated with the space 5.

[0120] In step 420, when the user terminal 100 is located in the custom area 60, 70, map element content may be displayed through the map view 20, including a map including the location of the user terminal, based on the service elements associated with the custom area 60, 70. The map element content may be content displayed through the map view 20 or the map displayed on the user terminal 100. The map element content may include text, images, three-dimensional content, etc. For example, the map element content may be content that describes an object included in the map view 20 or the map, content that highlights the object, or an effect that is applied to the object.

[0121] The map element content displayed on the user terminal 100 may be associated with a first object located in the custom area 60, 70 and may be displayed in an area of ​​the map view 20 corresponding to the custom area 60, 70. For example, referring to FIG. 1, when the user terminal 100 enters the first custom area 60, map element content associated with the first object (e.g., "Epitaph of Princess Suyeong") included in the first custom area 60 may be displayed in the map view 20 (not shown in FIG. 1).

[0122] Alternatively, the map content displayed on the user terminal 100 may be associated with a second object located in a remote area relative to the custom areas 60 and 70, and may be displayed in an area corresponding to the remote area of ​​the map view 20. The remote area may be an area different from the custom areas 60 and 70. For example, referring to FIG. 1 , when the user terminal 100 enters the first custom area 60, map element content 35 associated with a second object (e.g., "metal type") included in the remote second custom area 70, rather than the first custom area 60, may be displayed in the map view 20. When the user terminal 100 enters the first custom area 60, map element content 35 associated with the second object may be displayed in the second custom area 70 in the map view 20, for example, at the position of a specific object associated with the map element content 35 or AR content of the second custom area 70. Therefore, in such an embodiment, when the user terminal 100 enters a particular custom area, the map view 20 can display, as map element content, information about objects in the custom area as well as information about objects in remote areas.

[0123] Hereinafter, a method for displaying AR content and map element content on the user terminal 100 will be described in more detail with reference to FIG.

[0124] FIG. 9 is a diagram illustrating a method for displaying AR content or map element content associated with an object in a remote region on a user terminal based on the user terminal being located in a pre-defined custom region in one example.

[0125] A third custom area 910 and a fourth custom area 920 are defined on the map of the space 900 shown in the figure. Figure 9 shows a case where the user terminal 100 moves from a position 905 to the third custom area 910.

[0126] The fourth custom area 920 indicates the area of ​​the museum where the "gold crown" is placed, and the third custom area 910 indicates the area near the entrance to the gold crown room where the gold crown is displayed.

[0127] In the embodiment, when the user terminal 100 enters the third custom region 910 from the position 905, the AR view 10 of the user terminal 100 may display AR content 930 showing a gold crown in the direction of the gold crown, which is an object in the fourth custom region 920 ( <1> ) Furthermore, the map view 20 of the user terminal 100 may display that the user terminal 100 is at a position 905' in the third custom area 910, and may also display map element content 940 showing a gold crown at the position of the gold crown object in the fourth custom area 920 ( <2> ).

[0128] In this manner, in an embodiment, remote objects or content that do not exist in a particular custom area on the coordinate system can be associated with the custom area, and when the user terminal 100 enters the custom area, information about the remote objects or content can be displayed on the user terminal 100 as AR content and / or map element content.

[0129] Displaying the AR content 930 and / or the map element content 940 may involve switching the corresponding content that was hidden to visible, or adding additional effects, such as highlighting effects or animation effects, to content that is already displayed.

[0130] In this way, in the embodiment, the convenience of providing content in the AR view 10 and the map view 20 can be improved.

[0131] Next, referring to step 430, a method for providing a user interface (UI) to the user terminal 100 will be described as an example of controlling the user terminal 100 by positioning the user terminal 100 in the custom area 60, 70.

[0132] The service elements associated with the custom regions 60, 70 may include data for displaying a user interface (UI) associated with the services associated with the space 5.

[0133] In step 430, when the user terminal 100 enters the custom area 60, 70 or when the user terminal 100 leaves the custom area, the user terminal 100 may display a UI on the user terminal 100 based on the service elements associated with the custom area 60, 70. The UI may be a GUI displayed on the user terminal 100 to provide additional information related to the services provided in the space 5. The user terminal 100 may obtain additional information related to the services provided in the space 5 by interacting with the displayed UI.

[0134] In this regard, FIGS. 10 and 11 are diagrams illustrating a method for displaying a user interface (UI) on a user terminal based on the user terminal being located in a pre-defined area, in one example.

[0135] FIG. 10 shows a UI 1000 displayed through the AR view 10 of the user terminal 100 when the user terminal 100 enters a specific custom area.

[0136] The UI 1000 may be displayed together with content associated with the "Iron Buddha" object in the corresponding custom area, such as AR content, or after the AR content is displayed. The UI 1000 may be displayed on the user terminal 100 when the user terminal 100 enters the corresponding custom area according to a scenario condition defined for the custom area. The UI 1000 may be displayed semi-transparently relative to the image of the AR view 10.

[0137] In the example shown in the figure, when "Cancel" is selected by the user terminal 100, the display of the UI 1000 may be terminated. On the other hand, when "Directions" is selected by the user terminal 100, route guidance to another location or another object indicated by the UI 1000 may be provided to the user terminal 100. When "Directions" is selected by the user terminal 100, a route to another location or another object that is a destination may be displayed in the map view 20. Alternatively, when "Directions" is selected by the user terminal 100, another UI may be further displayed on the user terminal prior to the display of the route.

[0138] In relation to this, FIG. 11 is a diagram illustrating an additional UI 1100 displayed through the AR view 10 of the user terminal 100 when the destination is located on another floor during route guidance.

[0139] The additional UI 1100 may include a UI for selecting a means of transportation between floors. The user terminal 100 may provide route guidance using the selected means and display the resulting route on the map view 20. The UI 1100 may be displayed semi-transparently relative to the image of the AR view 10.

[0140] The user can travel to the destination while checking the route provided through the user terminal 100. When the user enters a custom area 60, 70 near an elevator, as shown in FIG. 12 , the user terminal 100 may display guide information 1200 for traveling to the destination as information (or content) related to elevator boarding based on a service element associated with the custom area. As described above, the guide information 1200 may be displayed when the user terminal 100 requests route guidance. The guide information 1200 may be displayed semi-transparently with respect to the image of the AR view 10.

[0141] Various scenario conditions defined in the custom areas 60 and 70 in relation to the UI display will be described in more detail with reference to FIGS.

[0142] Next, referring to step 440, a method for controlling the settings of the user terminal 100 will be described as an example of controlling the user terminal 100 depending on the user terminal 100 being located in the custom area 60, 70.

[0143] The service elements associated with the custom areas 60 , 70 may include data for controlling the configuration of the user terminal 100 to provide the services associated with the space 5 .

[0144] In step 440, the user terminal 100 may enter the custom area 60, 70 or the user terminal 100 may leave the custom area 60, 70, thereby controlling the settings of the user terminal 100 based on the service elements associated with the custom area 60, 70.

[0145] Controlling the settings of the user terminal 100 may include controlling the functions and / or operations of the user terminal 100 in relation to the services received by the user terminal 100 within the space 5. Controlling the functions and / or operations may include turning on / off or activating / deactivating the relevant functions and / or operations.

[0146] For example, controlling the settings of the user terminal 100 may include executing or stopping a module for recognizing objects within the space 5 or the custom area 60, 70 into which the user terminal 100 has entered. As an example, when the user terminal 100 enters the custom area 60, 70, a module for object recognition of the user terminal 100 may be turned on or initiated, thereby enabling the user terminal 100 to recognize objects within the space 5 or the custom area 60, 70. When the user terminal 100 leaves the custom area 60, 70, the module for object recognition of the user terminal 100 may be turned off or deactivated again. However, depending on the embodiment, the controlled settings may be maintained even when the user terminal 100 leaves the custom area 60, 70.

[0147] According to this embodiment, the user terminal 100 can recognize and detect objects only in a specific area or only after entering a specific custom area, rather than indiscriminately recognizing and detecting objects in all areas within the space 5. Therefore, services associated with the space 5 can be efficiently provided to the user terminal 100 while taking into consideration the user's movement line within the space 5 and the field of view (in that the custom area is a three-dimensional space having a height range).

[0148] The module for object recognition may be, for example, a module for Vision Object Tracking (VOT). Alternatively, the module for object recognition may be a configuration used for object recognition using a smart lens of the user terminal 100. Such a module may be a part of the processor 220 or may be implemented separately within the user terminal 100. Furthermore, such a module may be a software module executed by the processor 220.

[0149] In this regard, FIG. 15 illustrates a method for controlling user terminal settings based on the user terminal's location in a pre-defined custom area, in one example.

[0150] As shown in the figure, a custom area 1510 is set in space 1500, and when user terminal 100 enters custom area 1510 from position 1505, the settings of the user terminal may be controlled to turn on or initiate a module for object recognition in user terminal 100.

[0151] Such an embodiment can eliminate the need for an excessive amount of content to be extended through the AR view 10, which can complicate the user interface, increase the cognitive load on the user, and increase the load on the user terminal 100.

[0152] Various scenario conditions defined in the custom areas 60 and 70 in relation to the setting control of the user terminal 100 will be described in more detail with reference to FIGS.

[0153] The explanation of the technical features described above with reference to Figures 1 to 3, 12, and 13 can also be applied directly to Figures 4, 9 to 11, 14, and 15, so duplicate explanations will be omitted.

[0154] Furthermore, the scope of the embodiments should not be limited to the specific examples described with reference to the drawings.

[0155] FIG. 5 is a flow chart illustrating a method for creating a custom region, in one example.

[0156] Referring to Figure 5, we will explain how to define the above-mentioned custom areas 60, 70 on data representing the space 5 (e.g., a map of the space 5), associate service elements with the custom areas 60, 70, and provide services or information associated with the space 5 to the user terminal 100, or how to define scenario conditions for controlling the settings of the user terminal 100.

[0157] The custom areas 60 and 70 may be generated by an administrator terminal (described below) through a custom area generation tool. The custom area generation tool may be a program or part of a program installed on the administrator terminal, or a program or part of a program executed on the server 200. For example, the custom area generation tool is a creation tool for generating the custom areas 60 and 70 for the space 5 to provide a location-based AR service. The custom area generation tool may set areas corresponding to the custom areas 60 and 70, set predetermined scenario conditions (i.e., spatial conditions, etc.) for the areas corresponding to the custom areas 60 and 70, and associate specific service elements with the areas, thereby generating and augmenting three-dimensional content such as AR content on the user terminal 100 and displaying it. That is, the custom area generation tool may have a function of setting two-dimensional and / or three-dimensional spaces as areas corresponding to the custom areas 60 and 70, and associating the set areas with a service for outputting three-dimensional content. The custom area generation tool may manage each custom area 60, 70 using a unique ID, and may associate and manage the area corresponding to each custom area 60, 70 with the scenario conditions through the ID of each custom area 60, 70.

[0158] Alternatively, the custom region generation tool may include or form part of an indoor map generation tool.

[0159] Steps 510-530 described below may be performed through a custom region generation tool.

[0160] In step 510, the terminal or server 200 may define custom regions 60, 70 including at least one of a two-dimensional space and a three-dimensional space on data representing the space 5 (e.g., a map of the space 5). The server 200 may be a device that manages (registers, changes, deletes, saves, maintains, etc.) the custom regions 60, 70, and may be or include a map server that provides digital maps (3D maps and / or 2D maps).

[0161] Areas corresponding to the custom areas 60, 70 may be set in the space 5 using an absolute coordinate system. Areas corresponding to the custom areas 60, 70 may be set by at least one of inputting coordinate values ​​corresponding to the area, selecting a point, and selecting the area by dragging.

[0162] The data representing the space 5 is data that is rendered on the UI of the custom area generation tool, and may include at least one of a map of the space 5, point cloud data (PCD) collected for the space 5, and an image captured of the space 5. At least two of these may be displayed overlaid on the UI of the custom area generation tool.

[0163] The terminal may be a terminal of a service provider used to define custom regions 60, 70 on a map of the space 5. The terminal may be an administrator terminal that is separate from the user terminal 100 that receives the above-described services and content, i.e., displays content and information. The structure of the administrator terminal is similar to that of the computer system 2 described with reference to FIG. 2, and therefore a redundant description will be omitted. Through the administrator terminal, the service provider may define custom regions 60, 70 on data representing the space 5, such as a map. Once the custom regions 60, 70 are defined through the administrator terminal, they may be registered in the server 200.

[0164] However, depending on the embodiment, the above-described user terminal 100 may be used to define custom regions 60, 70 on data representing the space 5 (e.g., a map of the space 5). That is, the user terminal 100 that receives the service for the space 5 may also define the custom regions 60, 70 for providing the service for the space 5. In this case, the service for the space 5 may be based on crowdsourcing of the user terminals 100.

[0165] Furthermore, at least one location or area within the space 5 for generating the custom area 60, 70 within the space 5 may be sold to the user terminal 100. The user terminal 100 may purchase a location or area for generating the custom area 60, 70 through the server 200 and generate the custom area 60, 70 in the purchased location or area. The user terminal 100 may generate its own custom area 60, 70 in the location or area it purchased, thereby displaying desired AR content through the AR view of other user terminals located in the custom area 60, 70, or controlling other user terminals according to the scenario conditions of the custom area 60, 70. In such an embodiment, the locations or areas available for purchase by the user terminal 100 may belong to a portion of the space 5 permitted by the service provider, rather than the entire range of the space 5. In other words, the user terminal 100 may personalize the generation of the custom area 60, 70 by purchasing a location or area that belongs to a portion of the space 5 permitted by the service provider (i.e., a location or area permitted for sale).

[0166] The terminal may define a custom area by selecting an area to be designated as a custom area 60, 70 on data representing the space 5 (e.g., a map of the space 5), entering coordinates corresponding to the area, or dividing the area.

[0167] The defined custom regions 60, 70 may be displayed on a map of the space 5 on the administrator terminal.

[0168] In step 520, the terminal or server 200 may associate at least one service element with the custom area 60, 70 defined in step 510. Each defined custom area 60, 70 may be assigned an identifier (ID), and a service element may be associated with the identifier of each such custom area 60, 70.

[0169] In an embodiment, multiple custom areas may be defined on a map of the space 5, and each may be assigned an identifier. At least one service element may be associated with the identifier of each such custom area. By individually matching each service element with the identifier of the custom area, the user terminal 100 can be correctly controlled by these custom areas, even if some of the custom areas defined in the space 5 overlap each other.

[0170] The service element may include data for performing at least one of the following for a user terminal that enters the configured custom area 60, 70: displaying content in the AR view 10 of the user terminal 100; displaying a UI related to the service on the user terminal 100; displaying map element content on a map on the user terminal 100; and controlling settings of the user terminal 100 to provide the service to the user terminal 100.

[0171] In step 530, the terminal or server 200 may define (set) scenario conditions for the defined custom area 60, 70 to provide the user terminal 100 with services or information associated with the space 5 based on the service elements, or to control the settings of the user terminal 100.

[0172] Such scenario conditions are conditions for controlling the user terminal 100, and may include conditions for providing services or information associated with the space 5 or for controlling the settings of the user terminal 100.

[0173] In steps 510 to 530, the custom areas 60, 70 can be set (generated) on the map of the space 5 by associating service elements with the custom areas 60, 70 and defining scenario conditions.

[0174] The generated custom areas 60, 70 may control the user terminal 100 based on the associated service elements when the user terminal 100 is located in the custom area 60, 70 on the map (or when it is determined that the user terminal 100 has left the custom area 60, 70). Furthermore, the generated custom areas 60, 70 may provide the user terminal 100 with services or information associated with the space 5 based on the service elements, or may control the settings of the user terminal 100, according to the scenario conditions defined in the custom areas 60, 70.

[0175] In embodiments, scenarios or scenario conditions may be defined for each service element and may be part of the service element, i.e., a service element associated with a custom region 60, 70 may include a scenario or scenario condition.

[0176] As described above, steps 510 to 530 can be performed by a program or application including a custom area generation tool for generating (setting) custom areas 60, 70, which is executed on the administrator terminal or user terminal 100 described above.

[0177] Meanwhile, the custom areas 60 and 70 may be generated by an administrator possessing a specific measurement terminal directly scanning the space 5. The measurement terminal may be included in the administrator terminal or may be a separate device. The measurement terminal may be a device capable of more precise positioning. In this case, a certain range of area may be defined as the custom area 60 and 70 based on the positioning of the measurement terminal within the space 5. The positioning of the measurement terminal may be performed using a method such as visual localization, a Visual Positioning System (VPS), a marker recognition method, a GPS method, a Wi-Fi Positioning System (WPS), a Bluetooth module, or an odometry method. The custom areas 60 and 70 may be defined on a map of the space 5 based on the area and / or object recognized by the positioning of the measurement terminal. In other words, the custom areas 60 and 70 may be generated by directly scanning the space 5 using the administrator terminal, or this method may be used in combination with the method using the custom area generation tool described above. The custom region generation tool may be used to modify and adjust the automatically defined regions and set scenario conditions.

[0178] Meanwhile, the custom areas 60, 70 that have been created may be displayed not only on the map of the administrator terminal, but also on the map view 20 of the user terminal 100. This allows the user of the user terminal 100 to easily identify the custom areas 60, 70 through the map view 20.

[0179] The technical features described above with reference to FIGS. 1 to 4 and 9 to 14 can also be applied to FIG. 5, so duplicated explanations will be omitted.

[0180] Furthermore, the scope of the embodiments should not be limited to the specific examples described with reference to the drawings.

[0181] FIG. 6 illustrates a map with custom regions defined, in one example.

[0182] The generated custom regions 60, 70 are described in more detail with reference to FIG.

[0183] In the illustrated example, a first custom region 60 and a second custom region 70 are defined within the space 5 .

[0184] The custom regions 60 and 70 shown in the figure may be set in consideration of the occlusion region for the user terminal 100 in the space 5.

[0185] For example, in an embodiment, referring to FIG. 1 , AR content may be displayed at position 625 of the object "metal type" and / or position 620 of the object "epitaph of Princess Suyeong" in the AR view 10 only when the user terminal 100 enters the first custom area 60. That is, when the user terminal 100 is at position 630, AR content may be displayed at positions 620 and / or 625 of the object in the AR view 10. However, when the user terminal 100 is at position 640, the AR content in the AR view 10 is not displayed.

[0186] When the user terminal 100 is at the position 640, the structure 610 may be an occlusion region for the user terminal 100. If the user terminal 100 is at the position 640 but AR content is displayed at the object positions 620 and / or 625 in the AR view 10, the AR content would be displayed in an occlusion region, which is inappropriate.

[0187] According to an embodiment, custom areas 60 and 70 can be set taking into consideration occlusion areas caused by structures in the space 5, and AR is displayed in the AR view 10 only when the user terminal 100 is positioned in such custom areas 60 and 70, thereby improving the quality of AR content provided through the AR view 10.

[0188] Therefore, compared to a "rendering range"-based method that displays AR content based on the distance or radius from the object position 620 and / or 625, or a "semantic range"-based method that divides the space 5 into semantic unit areas and displays AR content when the user terminal 100 enters the unit space, the embodiment utilizes a "custom range" method that associates service elements to display AR content and uses areas that define scenario conditions, thereby making it possible to display AR content on the user terminal 100 in an easy-to-use and natural manner.

[0189] If a "rendering range" based method or a "semantic range" based method is used, even when the user terminal 100 is at position 640, the AR content would be displayed at object positions 620 and / or 625 in the AR view 10, which is inappropriate because the AR content would therefore be displayed in the occlusion area.

[0190] The technical features described above with reference to FIGS. 1 to 5 and 9 to 15 can also be applied to FIG. 6, so duplicated explanations will be omitted.

[0191] FIG. 7 is a diagram illustrating an example of a map on which a custom region is defined and an AR view screen of a user terminal.

[0192] The generated custom regions 710 and 730 are described in more detail with reference to Figure 7. In the illustrated example, a fifth custom region 710 and a sixth custom region 730 are defined within the space.

[0193] In the illustrated example, the user terminal 100 moves from a position 720 to a fifth custom area 710. Before the user terminal 100 enters the fifth custom area 710, predetermined guide information 740 may be displayed on the AR view 10. The guide information 740 may include information about actions that the user terminal 100 should perform to receive AR navigation services or AR content through the AR view 10.

[0194] When the user terminal 100 enters the fifth custom area 710, AR content may be displayed in the AR view 10 based on the service element associated with the fifth custom area 710. Such AR content may be, for example, welcome content that is displayed in the AR view 10 when the user first enters a space. Such a space may be, for example, a building.

[0195] In an embodiment, when the user terminal 100 is near the entrance before passing through the entrance or when the user terminal 100 is located just after passing through the entrance, the fifth custom region 710 may be set at a position at least a certain distance from the entrance so that the welcome content is not displayed in the AR view 10. This prevents the welcome content from being displayed incorrectly or from obstructing the view of the AR view 10. However, when using the above-described "rendering range" based method or "semantic range" based method, it is not possible to solve this problem.

[0196] The sixth custom area 730 may be set as an area around the elevator.

[0197] When the user terminal 100 enters a sixth custom area 730, which is an area near an elevator as shown in the figure, the user terminal 100 may be provided with information (or content) about riding the elevator based on a service element associated with the sixth custom area 730. At this time, if route guidance to the destination is provided to the user terminal 100 through the AR view 10, guide information for traveling to the destination (e.g., guide information 1200 in FIG. 12 ) may be displayed on the AR view 10.

[0198] The technical features described above with reference to FIGS. 1 to 6 and 9 to 15 can also be applied to FIG. 7, so duplicated explanations will be omitted.

[0199] Furthermore, the scope of the embodiments should not be limited to the specific examples described with reference to the drawings.

[0200] FIG. 8a illustrates a custom region defined in three-dimensional space in one example.

[0201] FIG. 8a provides further details about a custom region 800 defined as a three-dimensional space and the scenario conditions defined therein.

[0202] The above-mentioned custom regions 60, 70 may be defined on the map of the space 5 to include at least one of two-dimensional space and three-dimensional space.

[0203] The user terminal 100 or the server 200 may control the user terminal based on the service elements associated with the custom area 60, 70 when the user terminal 100 is located in the custom area 60, 70 on the map (or when it is determined that the user terminal 100 has left the custom area 60, 70).

[0204] The custom region 800 may be defined to include a two-dimensional area and a three-dimensional space with a set height range. Alternatively, the custom region 800 may be defined as a multi-dimensional space including two-dimensional and three-dimensional spaces. Defining such a custom region 800 may be performed by step 510 described above.

[0205] The two-dimensional region that makes up custom region 800 may include a two-dimensional coordinate space (x, z). The height range that makes up custom region 800 may include a y-axis coordinate space. The height range may be a range with an upper and lower limit (H), as shown in the figure, or it may be a range without an upper or lower limit, as shown in the figure.

[0206] If the user terminal 100 is located in a two-dimensional area that constitutes the custom area 800 and the height of the user terminal 100 falls within a set height range that constitutes the custom area 800, the user terminal 100 or the server 200 may determine that the user terminal 100 is located in the custom area 800 and control the user terminal 100 based on the service elements associated with the custom area 800.

[0207] The height of the user terminal 100 may be acquired by a sensor included in the user terminal 100. The height of the user terminal 100 may indicate, for example, the height of the camera of the user terminal 100.

[0208] Although one height range (H) is illustrated in FIG. 8a, multiple height ranges may be defined in the custom area 800, and service elements may be associated with each height range to define scenario conditions.

[0209] In this way, by setting the custom area 800 as a stereoscopic three-dimensional space, even if the user terminal 100 is located in the same two-dimensional space, the user terminal 100 can be controlled according to different scenario conditions depending on the height of the user terminal 100.

[0210] The scenario conditions defined for the custom regions 60, 70 are described in more detail below with reference to FIG. 8b.

[0211] FIG. 8b is a diagram illustrating the degree of freedom (DoF) of a user terminal in one example.

[0212] The custom regions 60, 70 may define 3DoF or 6DoF attitude conditions for the user terminal 100. 6DoF may refer to movement along the x-, y-, and z-axes, and rotation along the x-, y-, and z-axes (roll, yaw, and pitch).

[0213] Such pose conditions may be scenario conditions defined for the custom region 60, 70 and may be included in the service elements associated with the custom region 60, 70, as described above.

[0214] When the user terminal 100 is determined to be located in the custom area 60, 70, if the attitude of the user terminal 100 satisfies the attitude conditions defined in the custom area 60, 70, the user terminal 100 or the server 200 may control the user terminal based on the service elements associated with the custom area 60. The attitude of the user terminal 100 may be acquired from a sensor included in the user terminal 100, for example, a gyro sensor.

[0215] By defining such posture conditions in the custom regions 60, 70, for example, the user terminal 100 may be controlled based on the service element only when the user terminal 100 enters the custom region 60, 70 and the AR view 10 maintains a specific angle or a specific angle range. As an example, AR content that should be displayed in a specific region within the custom region 60, 70, for example, at a position above or below a certain height, may be displayed in the AR view 10 of the user terminal 100 only when the user terminal 100 enters the custom region 60, 70 and the AR view 10 maintains an angle or angle range that reflects the specific region above or below a certain height.

[0216] Therefore, in an embodiment, the AR content can be displayed at a more appropriate position and viewpoint of the AR view 10.

[0217] Further examples of scenario conditions defined for custom regions 60, 70 are described in more detail below.

[0218] The custom areas 60, 70 may have defined therein at least one of a first condition indicating a time condition, a second condition indicating a movement of the user terminal 100, and a third condition indicating an operation on the user terminal 100.

[0219] The first condition may be, for example, a condition requiring the user terminal 100 to stay in the custom area 60, 70 for a certain period of time or more (e.g., three seconds or more), or for a certain period of time or less. Alternatively, the first condition may be a condition requiring the user terminal 100 to enter the custom area 60, 70 during a certain time period, or to enter the custom area 60, 70 at a time other than the certain time period. The second condition may be, for example, a condition indicating a physical operation of the user terminal 100, such as a condition requiring the user terminal 100 to be shaken a certain number of times or with a certain strength or more. Alternatively, the second condition may be a condition requiring the user terminal 100 to be moved in a certain shape (e.g., a figure eight). The third condition may be an act of electronically operating the user terminal 100, such as a condition requiring the user to press a certain button or key, input a certain gesture, input a certain voice or audio, or photograph a certain object.

[0220] When the user terminal 100 is determined to be located in the custom area 60, 70, if the user terminal 100 satisfies at least one of the first to third scenario conditions, the user terminal 100 or the server 200 may control the user terminal 100 based on the service elements associated with the custom area 60, 70. For example, the user terminal 100 may display AR content differently, display a UI differently, display map element content differently, or control the settings of the user terminal 100 differently, depending on whether the scenario condition is satisfied.

[0221] The explanation of the technical features given above with reference to FIGS. 1 to 7 and 9 to 15 can also be applied to FIGS. 8a and 8b, and therefore a duplicate explanation will be omitted.

[0222] Furthermore, the scope of the embodiments should not be limited to the specific examples described with reference to the drawings.

[0223] FIG. 16 is a diagram showing a method in which multiple custom areas are set in a space and scenario conditions for controlling user terminals for the custom areas are set in one example.

[0224] Other examples of scenario conditions defined for each of the multiple areas 1610 to 1640 set in the space 1600 will be described in more detail below.

[0225] The above-described custom areas 60 and 70 may be any one of the custom areas 1610 to 1640.

[0226] At least one of the following scenario conditions may be defined for any one of the custom areas 60, 70 among the custom areas 1610-1640: i) a first scenario condition indicating whether the user terminal 100 has passed through a predetermined custom area other than the custom area 60, 70; and ii) a second scenario condition indicating whether the number of user terminals located in the custom area 60, 70 falls within a predetermined range. Depending on whether the scenario condition is satisfied, the user terminal 100 may be provided with different services or information associated with the space 5, 1600 based on the service elements associated with the custom area 60, 70, or the settings of the user terminal 100 may be controlled differently.

[0227] The first scenario condition may be a condition that requires the user terminal 100, having entered the custom area 60, 70, to enter another specific area before entering the custom area 60, 70. For example, the first scenario may be satisfied if the user terminal 100 enters the custom area 1610 (within a predetermined time) after entering a specific area among the areas 1620 to 1640. Alternatively, the first scenario may be satisfied if the user terminal 100 enters the custom area 1610 (within a predetermined time) after entering at least some of the areas 1610 to 1640 in a specific order. Depending on whether the first scenario condition is satisfied, the user terminal 100 may display AR content differently, display a UI differently, display map element content differently, or perform different setting control of the user terminal 100.

[0228] The second scenario condition may be a condition that requires the number of user terminals located in the custom areas 60, 70, or the cumulative number of user terminals that have entered the custom areas 60, 70 (within a predetermined time period) to be greater than or less than a certain number. Alternatively, the second scenario condition may be a condition that requires the number of user terminals located in other areas other than the custom area 60, 70 into which the user terminal 100 has entered, or the cumulative number of user terminals that have entered other areas (within a predetermined time period) to be greater than or less than a certain number. Alternatively, the second scenario condition may be a condition that requires the number of user terminals located in both the custom areas 60, 70 and other areas, or the cumulative number of user terminals that have entered two custom areas (within a predetermined time period) to be greater than or less than a certain number. Depending on whether the second scenario condition is satisfied, the user terminal 100 may display different AR content, different UI display, different map element content, or different setting control of the user terminal 100.

[0229] The first and second scenario conditions described above may be used in combination. For example, when the user terminal 100 enters the custom area 60, 70, the user terminal 100 or the server 200 may determine whether the first and second scenario conditions are met, and may control the user terminal 100 based on the service elements only if the scenario conditions are met. Alternatively, the control of the user terminal 100 based on the service elements may differ depending on whether the first or second scenario condition is met.

[0230] The technical features described above with reference to FIGS. 1 to 15 can be applied to FIG. 16 as well, so duplicated explanations will be omitted.

[0231] Furthermore, the scope of the embodiments should not be limited to the specific examples described with reference to the drawings.

[0232] 17 to 21 are diagrams illustrating a method for controlling user terminals located in a custom area according to various scenario conditions defined for the custom area, in one example.

[0233] A method for controlling the user terminal 100 using a custom area generated by the above-described method for generating a custom area will be described in more detail with reference to FIGS.

[0234] As described above, the service elements associated with the generated custom area may include data for displaying augmented reality (AR) content related to the service associated with the space. When the user terminal is located in the custom area, the AR content may be displayed through the AR view 10 according to the scenario conditions set for the custom area. Thus, in the examples of FIGS. 17 to 21, a method for displaying AR content on the user terminal 100 as a method for controlling the user terminal 100 that has entered the custom area will be described in more detail.

[0235] FIG. 17a illustrates an AR view 10 according to an embodiment in a state where a custom region does not exist, and the position of the user terminal 100 at that time. As illustrated in the figure, the AR view 10 may display both AR content displayed on the first floor of the space (e.g., AR content associated with an object located on the first floor) and AR content (A2) displayed on the second floor (e.g., AR content associated with an object located on the second floor). An example of the AR content displayed at this time may be an image and / or text corresponding to a sign. That is, in FIG. 17a, even though the user terminal 100 is located on the first floor, the AR content (A2) for the second floor is also displayed on the AR view 10. This complicates the implementation of the AR view 10 in the user terminal 100, and increases the burden on the user terminal 100 for implementing such an AR view 10.

[0236] 17b-17d show the same space as shown in FIG. 17a, but with two custom regions (CR_1 and CR_2) created. Each of the custom regions (CR_1 and CR_2) may have been created using the custom region tool 102 described above. Custom region (CR_1) represents the first floor of the space, and custom region (CR_2) represents the second floor of the space. Each of the regions (CR_1 and CR_2) represents a three-dimensional space and may correspond to a rectangular parallelepiped polygon as shown in the figures.

[0237] 17c, when the user terminal 100 is located on the first floor but in the custom area (CR_1), only the AR content (A1) for the first floor may be displayed on the AR view 10 of the user terminal 100. In other words, when the user terminal 100 is located in the custom area (CR_1), the AR content (A2) for the second floor is not displayed.

[0238] On the other hand, as shown in FIG. 17d, when the user terminal 100 is located on the second floor but in the custom area (CR_2), only the AR content (A2) for the second floor may be displayed in the AR view 10 of the user terminal 100. In other words, when the user terminal 100 is located in the custom area (CR_2), the AR content for the first floor is not displayed. In some embodiments, the AR content for the floor may be displayed in the AR view 10 by positioning the user terminal 100 so as to look down on it (for example, by maintaining the AR view 10 in a position that looks down on the first floor).

[0239] As shown in Figures 17c and 17d, by properly defining custom areas (CR_1, CR_2) for each floor, the user terminal 100 can display only the AR content of the custom area in which the user terminal 100 is located, depending on the location of the user terminal 100.

[0240] Meanwhile, in the generated custom area, a scenario condition may be set to distinguish between when the user terminal 100 enters the custom area and when the user terminal 100 leaves the custom area, and different AR content may be displayed through the AR view 10. In other words, even if the user terminal 100 is located within the same custom area, different AR content may be displayed through the AR view 10 when the user terminal 100 enters the custom area and when the user terminal 100 leaves the custom area.

[0241] In this regard, FIGS. 18a to 18c show the case where six custom regions (CR_1 to CR_6) are generated in the same space as that shown in FIG.

[0242] As shown in the figure, in addition to the regions (CR_1 to CR_6), multiple nodes may be defined within the space. Lines connecting the nodes may be called "links." The defined nodes and links may be used to provide route guidance to a predetermined destination for the user terminal 100. Multiple nodes may be defined within each of the custom regions (CR_1 to CR_6).

[0243] 18a to 18c show a route (R) along which the user terminal 100 travels in accordance with such route guidance. The route (R) may be provided to the user terminal 100 as route guidance information.

[0244] The service elements associated with each custom area may be associated with at least one node within the corresponding custom area.

[0245] The scenario conditions of the custom area may be set so that different AR contents are displayed through the AR view 10 when the user terminal 100 moves according to route guidance to the destination, i.e., moves along the route (R) presented through the route guidance, and when the user terminal 100 enters the custom area and when the user terminal 100 leaves the custom area.

[0246] At this time, whether the user terminal 100 enters a custom area or whether the user terminal 100 leaves a custom area may be determined based on the route (R) presented to the user terminal in accordance with the route guidance. In other words, the user terminal 100 or the server 200 may determine, based on the route (R), what nodes the user terminal 100 has passed through or what nodes it will pass through, and may thereby determine whether the user terminal 100 has entered or exited the custom area in which it is currently located.

[0247] For example, as shown in FIG. 18b, it is assumed that the user terminal 100 has entered a custom area (CR1) based on the current location of the user terminal 100 and the route (R) that the user terminal 100 has traveled or is traveling along. At this time, AR content including an arrow pointing toward the entrance / exit direction along with the text "Entering the building" may be displayed on the AR view 10 of the user terminal 100. Such AR content may be displayed in association with a specific node (Gate node) within the custom area (CR1). As shown in the figure, the custom area (CR1) may be outdoors, and the custom area (CR2) may be indoors.

[0248] When the user terminal 100 enters the custom area (CR2), information related to the indoor space including the custom area (CR2) may be displayed as AR content in the AR view 10. Alternatively, welcome content such as "Welcome" may be displayed through the AR view 10. Meanwhile, as shown in FIG. 18c, it is assumed that the user terminal 100 has left the custom area (CR2) based on the current location of the user terminal 100 and the route (R) along which the user terminal 100 has moved or is traveling. At this time, AR content including an arrow pointing toward the entrance / exit direction along with text such as "Move outside the building" may be displayed on the AR view 10 of the user terminal 100. Such AR content may be displayed in association with a specific node (Gate node) within the custom area (CR2).

[0249] Thus, in the embodiment, according to the travel route provided to the user terminal 100, the user terminal 100 can determine whether to enter / exit a particular custom area, and accordingly display appropriate AR content in the AR view 10.

[0250] Figure 19 provides a more detailed explanation of the case where multiple custom regions are defined for one space (exhibition room). As shown in the figure, four CRs may be generated for one exhibition room. Assume that there are three objects in the exhibition room: work A, work B, and work C.

[0251] The scenario condition set for the long-distance extended CR may be to display AR content including first information of works A, B, and C on the AR view 10 of the user terminal 100 located in the long-distance extended CR. The first information may include only summary information such as the title of the work.

[0252] Meanwhile, a scenario condition set for the CR for work A may be to display AR content including second information of work A in the AR view 10, and to deactivate all AR content associated with other works. The second information may include more abundant and detailed information about the first work than the first information. That is, the user terminal 100 that has entered the CR for work A can concentrate on checking only the detailed information about work A associated with the CR for work A. A similar explanation may be applied to the CR for work B and the CR for work C.

[0253] In this manner, in an embodiment, multiple custom areas can be generated for a single space that output AR content with different amounts of information, thereby improving the user experience of receiving the AR content.

[0254] Figure 20 illustrates a case where two custom areas are created within the space described with reference to Figure 7, and a specific point or sub-area is defined in each custom area. For example, a point or sub-area (Point B) may be defined in a custom area (CR_1) corresponding to the first floor, and a point or sub-area (Point A) may be defined in a custom area (CR_2) corresponding to the second floor.

[0255] The scenario conditions set for the custom area (CR_1) may be set so that the AR content is displayed differently in each of the following cases: when entering the custom area (CR_1) via a specific point or sub-area (Point B) versus other cases; when exiting the custom area (CR_1) via a specific point or sub-area (Point B) versus other cases; when entering the custom area (CR_1) via another custom area (CR_2) versus other cases; and when entering the custom area (CR_1) via a specific point or sub-area (Point A) in another custom area (CR_2) versus other cases.

[0256] In other words, the scenario conditions may be set so that when the user terminal 100 is located in a custom area, AR content determined based on other custom areas that the user terminal 100 entered / passed through before entering the custom area is displayed through the AR view 10. Alternatively, the scenario conditions may be set so that when the user terminal 100 is located in a custom area, content determined based on points that the user terminal 100 passed through is displayed through the AR view 10.

[0257] The above-mentioned "determined AR content" may be existing AR content displayed in a different form, with other animations or effects applied, or may be AR content with other content displayed, or existing AR content with additional content (audio, text, images, video, etc.) added.

[0258] FIG. 21 illustrates a method for changing the AR content displayed through the AR view 10 in consideration of the direction of the user terminal 100.

[0259] In Figure 21a, the user terminal 100 enters the custom area (CR_A) via the custom area (CR_A_1). In Figure 21b, the user terminal 100 enters the custom area (CR_A) via the custom area (CR_A_2).

[0260] In FIGS. 21a and 21b, the user terminal 100 is located in the same custom area (CR_A), but the content displayed through the AR view 10 may be different in each case. For example, when the user terminal 100 passes through the custom area (CR_A_1), it is determined that the user terminal 100 is entering a specific space (restaurant), and AR content that provides information about the entrance to the specific space or additional services (e.g., recommended menu items at the restaurant) may be displayed on the AR view 10. On the other hand, when the user terminal 100 passes through the custom area (CR_A_2), it is determined that the user terminal 100 is leaving the specific space (restaurant), and AR content that provides information about the outdoors, a parking lot, or additional services (e.g., information about other dessert cafes) may be displayed on the AR view 10. Figure 21 c).

[0261] As another example, if the user terminal 100 passes through a custom area indicating a parking lot and enters a specific space (such as a shopping mall), the user terminal 100 or the server 200 may determine that the user visited the specific space using a private car as a means of transportation. Alternatively, if the user terminal 100 passes through a custom area indicating an entrance to a subway station and enters a specific space (such as a shopping mall), the user terminal 100 or the server 200 may determine that the user visited the specific space using public transportation as a means of transportation. In this case, when the user terminal 100 enters a custom area within the specific space or leaves the specific space, content customized for the means of transportation used by the user may be output in the AR view 10 of the user terminal 100. For example, a user who uses a private car may be provided with AR content including information about how to get to the parking lot or about car accessories, and a user who uses public transportation may be provided with AR content including information about how to get to the subway station.

[0262] The explanation of the technical features given above with reference to FIGS. 1 to 16 can be applied to FIGS. 17 to 21 as well, so duplicate explanations will be omitted.

[0263] Furthermore, the scope of the embodiments should not be limited to the specific examples described with reference to the drawings.

[0264] The above-described devices may be implemented using hardware components, software components, and / or a combination of hardware and software components. For example, the devices and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or various devices capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the OS. The processing device may also access, record, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, a single processing device may be described. However, those skilled in the art will understand that a processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing device may include multiple processors or one processor and one controller. Other processing configurations, such as parallel processors, are also possible.

[0265] Software may include computer programs, codes, instructions, or a combination of one or more of these, which may configure a processing device to operate as desired or may independently or collectively instruct the processing device. The software and / or data may be embodied in any type of machine, component, physical device, computer storage medium, or device to be interpreted by the processing device or to provide instructions or data to the processing device. The software may be distributed and stored and executed in a distributed manner on computer systems connected by a network. The software and data may be stored on one or more computer-readable storage media.

[0266] Methods according to embodiments may be implemented in the form of program instructions executable by various computer means and recorded on a computer-readable medium. In this case, the medium may continuously record a computer-executable program or may temporarily record the program for execution or download. The medium may be various recording or storage means in the form of a single piece of hardware or multiple pieces of hardware combined together. It may be a medium directly connected to a computer system or distributed over a network. Examples of media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and media configured to record program instructions, such as ROMs, RAMs, and flash memories. Other examples of media include recording media or storage media managed by app stores that distribute applications, or by websites or servers that provide or distribute various software.

[0267] Although the embodiments have been described above based on limited examples and drawings, those skilled in the art will appreciate that various modifications and variations may be made from the above description. For example, the described techniques may be performed in an order different from that described, and / or the described system, structure, device, circuit, or other element may be coupled or combined in a manner different from that described, or may be substituted or replaced by other elements or equivalents, and still achieve suitable results.

[0268] Therefore, different embodiments are within the scope of the appended claims, provided that they are equivalent to the claims.

Claims

1. A method for controlling a user terminal executed by a computer system that is a server or a user terminal, comprising: determining whether the user terminal is located in a preset custom area in space based on the location of the user terminal, the custom area being associated with at least one service element; and controlling the user terminal based on the service element when the user terminal is determined to be located in the custom area; A method for controlling a user terminal, comprising:

2. the custom region is defined on the map of the space to include at least one of a two-dimensional space and a three-dimensional space; The step of controlling the user terminal includes: When the position of the user terminal is located in the custom area on the map, control the user terminal based on the service element. A method for controlling a user terminal according to claim 1.

3. the custom region includes a two-dimensional area and a three-dimensional space having a set height range; The step of controlling the user terminal includes: When the user terminal is located in the two-dimensional area and the height of the user terminal is within the set height range, determine that the user terminal is located in the custom area, and control the user terminal based on the service element. A method for controlling a user terminal according to claim 1.

4. The custom area defines a 3DoF (Degree Of Freedom) or 6DoF posture condition of the user terminal, The step of controlling the user terminal includes: When it is determined that the user terminal is located in the custom area, if the posture of the user terminal satisfies the posture condition, control the user terminal based on the service element. A method for controlling a user terminal according to claim 1.

5. At least one condition is defined in the custom area: a first condition indicating a time condition, a second condition indicating a movement of the user terminal, and a third condition indicating an operation on the user terminal; The step of controlling the user terminal includes: When it is determined that the user terminal is located in the custom area, if the user terminal satisfies the defined condition, control the user terminal based on the service element. A method for controlling a user terminal according to claim 1.

6. The service element includes data for displaying augmented reality (AR) content related to a service associated with the space; The step of controlling the user terminal includes: and displaying the AR content through an AR view including an image captured by a camera of the user terminal based on the service element when the user terminal is located in the custom area.

2. A method for controlling a user terminal according to claim 1, comprising:

7. The AR content is The custom area is associated with a first object located in the custom area and is displayed in the AR view corresponding to the custom area; The custom region is associated with a second object located in a remote region relative to the custom region, and is displayed within the AR view corresponding to the remote region. A method for controlling a user terminal according to claim 6.

8. the AR content includes a plurality of sequences; The step of displaying the AR content includes: When it is determined that the user terminal is located in the custom area, displaying different sequences of the AR content through an AR view according to a position of the user terminal within the custom area. A method for controlling a user terminal according to claim 6.

9. The step of displaying the AR content includes: and changing and displaying the AR content displayed through the AR view or outputting predetermined additional content based on at least one of the position of the user terminal within the custom area, the number of times the user terminal has entered the custom area, and the state of the user terminal entering / exiting the custom area.

7. A method for controlling a user terminal according to claim 6, comprising:

10. When the user terminal enters the custom area and when the user terminal leaves the custom area, different AR contents are displayed as the AR content through the AR view. A method for controlling a user terminal according to claim 9.

11. the service element includes data for displaying map element content related to a service associated with the space; The step of controlling the user terminal includes: displaying the map element content through a map view including a map including the location of the user terminal based on the service element when the user terminal is located in the custom area; 2. A method for controlling a user terminal according to claim 1, comprising:

12. The map element content includes: The map view is associated with a first object located in the custom area and is displayed in an area of ​​the map view that corresponds to the custom area; a second object associated with the second object located in a remote area relative to the custom area, the second object being displayed in an area of ​​the map view corresponding to the remote area; A method for controlling a user terminal according to claim 11.

13. the service element includes data for displaying a user interface (UI) associated with a service associated with the space; The step of controlling the user terminal includes: Displaying the UI on the user terminal based on the service element when the user terminal enters the custom area or when the user terminal leaves the custom area.

2. A method for controlling a user terminal according to claim 1, comprising:

14. the service element includes data for controlling a setting of the user terminal to provide the service associated with the space to the user terminal; The step of controlling the user terminal includes: Controlling the configuration of the user terminal based on the service element when the user terminal enters the custom area or when the user terminal leaves the custom area; A method for controlling a user terminal according to claim 1.

15. Controlling the settings of the user terminal includes executing or stopping a module for recognizing an object in the space or the custom area. A method for controlling a user terminal according to claim 14.

16. The custom area defines a scenario condition for providing the user terminal with a service or information associated with a space based on the service element or for controlling a setting of the user terminal; The step of controlling the user terminal includes: controlling the user terminal such that a service or information associated with the space is provided to the user terminal based on the service element, or a setting of the user terminal is controlled, according to the scenario condition; A method for controlling a user terminal according to claim 1.

17. A plurality of custom areas are set in the space, In the custom area, the scenario conditions are defined such that, depending on whether at least one of i) a first scenario condition indicating whether the user terminal has passed through a predetermined custom area other than the custom area, and ii) a second scenario condition indicating whether the number of user terminals located in the custom area falls within a predetermined range, different services or information associated with the space based on the service elements are provided to the user terminal, or the settings of the user terminal are controlled differently. A method for controlling a user terminal according to claim 16.

18. the service elements associated with the custom area are multiple; each service element of the plurality of service elements associated with a different point within the custom region; In the custom area, the scenario conditions are defined so as to provide services or information associated with the space differently according to the order in which the user terminal passes through the points associated with each of the service elements. A method for controlling a user terminal according to claim 16.

19. receiving a route guidance request to a destination from the user terminal; and providing route guidance to the user terminal in accordance with the route guidance request; further comprising the determining step and the controlling step of the user terminal are executed when the user terminal moves to the destination according to the route guidance. A method for controlling a user terminal according to claim 1.

20. A computer system realizing a user terminal or a server, at least one processor configured to execute computer-readable instructions contained in the memory; Including, The at least one processor Determine, based on the location of the user terminal, whether the user terminal is located in a preset custom area in a space, the custom area being associated with at least one service element; and control the user terminal based on the service element when it is determined that the user terminal is located in the custom area; Computer system.

Images