Method and system for providing service based on three-dimensional map to which image-based VPS technology is applied

Abstract

A method and a system for providing a service based on a three-dimensional map to which image-based VPS technology is applied are disclosed. According to one embodiment, the method for providing a three-dimensional map-based service may comprise the steps of: specifying, on the basis of an image acquired through a portable terminal of a user located in a target space, the location of a user on a three-dimensional map generated to correspond to the target space; and providing a three-dimensional map-based service on the basis of the position of the user specified on the three-dimensional map.

Description

Method and system for providing 3D map-based services using image-based VPS technology

[0001] The following examples relate to a method and system for providing a 3D map-based service, and relate to a method and system for providing a 3D map-based service using image-based VPS (Visual positioning system) technology.

[0002] With the spread of smartphones, location information services used in PC or laptop environments have evolved to be used in mobile terminal environments.

[0003] Microsoft has released a service called Photosynth that searches and collects photos shared on the Internet, calculates the shooting location, direction, and 3D coordinates of key features of each video, and provides the collected photos as if they were actual navigation, rather than a simple thumbnail view. Google is also providing a 3D map service by uploading 3D models modeled using 3D modeling tools such as Google Sketchup to Google Earth.

[0004] The 3D model or 3D map generated in this way can be provided as a virtual / augmented reality service by augmenting a responsive object that can interact with the user, or as a navigation service by augmenting a display object that guides the user along a path.

[0005] However, conventional techniques for augmenting objects on 3D maps have the disadvantages of high complexity and difficulty, as they presuppose that skilled designers use professional computer graphics software such as Maya and 3D Max.

[0006] Accordingly, there is a need to propose an object augmentation technique that reduces complexity and difficulty.

[0007] Meanwhile, the generated 3D map is applied with VPS (Visual Positioning System) technology to be used in the process of providing virtual / augmented reality services or navigation services.

[0008] Existing VPS technology generally identifies the user's location and pose on a 3D map based on sensing values ​​sensed by the sensors of a device providing a 3D map, and thus has the limitation that virtual / augmented reality services or navigation services are only possible on devices equipped with various sensors.

[0009] Therefore, there is a need to propose a technology that enables virtual / augmented reality services or navigation services to be provided through portable terminals carried by non-experts.

[0010] The technical challenge that one embodiment seeks to achieve is to enable virtual / augmented reality services or navigation services to be provided through portable terminals carried by non-experts.

[0011] To this end, one embodiment proposes a method and system for providing a 3D map-based service using image-based VPS technology.

[0012] In addition, one embodiment proposes a method and system for augmenting objects in a 3D map through a service platform on a portable terminal carried by a non-expert, in order to reduce the complexity and difficulty of object augmentation.

[0013] The technical problems to be achieved by the embodiments are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the embodiments belong from the description below.

[0014]

[0015] In order to achieve the above technical task, a method for providing a 3D map-based service performed by a computer device according to one embodiment may include a step of specifying a location of a user on a 3D map generated corresponding to a target space based on an image acquired through a portable terminal of a user located in the target space; and a step of providing a 3D map-based service based on the location of the user specified on the 3D map.

[0016] According to one aspect, the step of specifying may include a step of analyzing and matching at least one feature point of the image and at least one feature point of the three-dimensional map to specify the location of the user defined in the coordinate system of the three-dimensional map.

[0017] According to another aspect, the step of specifying may further include a step of analyzing and matching at least one feature point of the image and at least one feature point of the three-dimensional map to specify a pose of the user defined by roll, pitch, and yaw on the three-dimensional map.

[0018] According to another aspect, the step of specifying may be characterized by including a step of selecting at least one feature point to be analyzed and matched based on the reliability of each of the feature points included in the image; and a step of selecting at least one feature point to be analyzed and matched based on the reliability of each of the feature points included in the 3D map.

[0019] According to another aspect, the reliability of each feature point included in the image may be calculated based on the generation accuracy of each feature point included in the image, and the reliability of each feature point included in the 3D map may be calculated based on the generation accuracy of each feature point included in the 3D map.

[0020] According to another aspect, the step of providing may be characterized by including a step of augmenting at least one content object on the 3D map based on a location of a user specified on the 3D map.

[0021] According to another aspect, the at least one content object may be characterized by including at least one of a responsive object that can interact with the user on the 3D map or a display object that provides information to the user on the 3D map.

[0022] According to another aspect, the augmenting step may further include a step of adjusting the augmented position and pose of the at least one content object on the three-dimensional map in response to a user input generated through a service platform provided on a portable terminal.

[0023] According to one embodiment, a computer-readable recording medium having recorded thereon a computer program for executing a method for providing a 3D map-based service on a computer device, wherein the method for providing a 3D map-based service may include a step of specifying a location of a user on a 3D map generated corresponding to a target space based on an image acquired through a portable terminal of a user located in the target space; and a step of providing a 3D map-based service based on the location of the user specified on the 3D map.

[0024] According to one embodiment, a computer device for performing a method for providing a 3D map-based service may include at least one processor configured to execute computer-readable commands, wherein the at least one processor may include a specific unit for specifying a location of a user on a 3D map generated corresponding to a target space based on an image acquired through a portable terminal of a user located in the target space; and a providing unit for providing a 3D map-based service based on the location of the user specified on the 3D map.

[0025] According to one aspect, the specific portion may be characterized by analyzing and matching at least one feature point of the image and at least one feature point of the three-dimensional map to specify the location of the user defined in the coordinate system of the three-dimensional map.

[0026] According to another aspect, the specific portion may be characterized by analyzing and matching at least one feature point of the image and at least one feature point of the three-dimensional map to specify the user's pose defined by roll, pitch, and yaw on the three-dimensional map.

[0027] According to another aspect, the providing unit may be characterized by augmenting at least one content object on the 3D map based on a location of a user specified on the 3D map.

[0028] One embodiment proposes a method and system for providing a 3D map-based service using image-based VPS technology, thereby enabling virtual / augmented reality services or navigation services to be provided through portable terminals carried by non-experts.

[0029] In addition, one embodiment can reduce the complexity and difficulty of object augmentation by proposing a method and system for augmenting objects on a 3D map through a service platform on a portable terminal carried by a non-expert.

[0030] It should be understood that the described technical effects are not limited to the effects described above, but include all effects that can be inferred from the composition of the invention described in the detailed description below or the claims.

[0031] Figure 1 is a diagram illustrating an example of a service environment according to one embodiment.

[0032] FIG. 2 is a block diagram illustrating an example of a computer device according to one embodiment.

[0033] FIG. 3 is a block diagram illustrating an example of components that the processor illustrated in FIG. 2 may include.

[0034] FIG. 4 is a flowchart illustrating a method for providing a 3D map-based service that can be performed by the computer device illustrated in FIG. 2.

[0035] Figures 5 to 7 are drawings for explaining how to specify a user's location in the 3D map-based service providing method illustrated in Figure 4.

[0036] Hereinafter, the present invention will be described with reference to the attached drawings. However, the present invention can be implemented in various different forms and is therefore not limited to the embodiments described herein. In the drawings, irrelevant parts have been omitted for clarity of description, and similar parts have been designated with similar reference numerals throughout the specification.

[0037] Throughout the specification, when a part is said to be "connected (connected, contacted, or coupled)" to another part, this includes not only cases where it is "directly connected," but also cases where it is "indirectly connected" with another part in between. Furthermore, when a part is said to "include" a component, this does not exclude other components, but rather implies that it may include other components, unless otherwise specifically stated.

[0038] The terminology used herein is merely used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this specification, it should be understood that the terms "comprises" or "has" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0039]

[0040] In the following examples, a method and system for providing a 3D map-based service that enables virtual / augmented reality services or navigation services to be provided through a portable terminal carried by a non-expert by applying image-based VPS technology are described.

[0041] In addition, in the following embodiments, a method and system for augmenting objects on a 3D map through a service platform on a portable terminal carried by a non-expert are described in order to reduce the complexity and difficulty of object augmentation.

[0042] The service platform is implemented in the form of a dedicated program, application or web page that runs on a portable terminal, which is a mobile terminal carried by a non-expert user, and can be used as an interface in the process of performing a 3D map-based service provision method.

[0043] A method for providing a 3D map-based service can be performed by a 3D map-based service providing system implemented in a server or a portable terminal comprising at least one computer device including a processor, and the 3D map-based service providing system can be operated under the control of a computer program. The above-described computer program can be stored in a computer-readable recording medium so as to be combined with a computer device and execute the 3D map-based service providing method on the computer device. The computer program described herein may have the form of an independent program package, or may have the form of an independent program package pre-installed on a computer device and linked with an operating system or other program packages.

[0044]

[0045] FIG. 1 is a diagram illustrating an example of a service environment according to one embodiment. The service environment of FIG. 1 represents an example including a plurality of electronic devices (110, 120, 130, 140), a plurality of servers (150, 160), and a network (170).

[0046] This drawing 1 is an example for explaining the invention, and the number of electronic devices or servers is not limited to that of drawing 1. In addition, the service environment of drawing 1 merely illustrates one example of environments applicable to the present embodiments, and the environments applicable to the present embodiments are not limited to the service environment of drawing 1.

[0047] The plurality of electronic devices (110, 120, 130, 140) may be mobile terminals or portable terminals implemented as computer devices.

[0048] In particular, each of the plurality of electronic devices (110, 120, 130, 140) may mean one of various physical computer devices that can communicate with a server (150, 160) via a network (170) using a wireless or wired communication method, assuming that the plurality of electronic devices includes a camera so that the target space can be photographed and an image can be obtained while being held by a user. For example, examples of the plurality of electronic devices (110, 120, 130, 140) include a smart phone, a mobile phone, a navigation system, a laptop, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), a tablet PC, etc.

[0049] That is, the electronic devices (110, 120, 130, 140) are portable terminals carried by users that capture images of a target space to apply VPS technology, and may refer to terminals that provide a 3D map-based service described below.

[0050] The communication method between the electronic devices (110, 120, 130, 140) and the servers (150, 160) is not limited, and may include not only a communication method utilizing a communication network (e.g., a mobile communication network, a wired Internet, a wireless Internet, a broadcasting network) that the network (170) may include, but also short-range wireless communication between the devices. For example, the network (170) may include any one or more of a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), the Internet, and the like. In addition, the network (170) may include any one or more of a network topology including, but not limited to, a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree, or a hierarchical network.

[0051] Each of the servers (150, 160) may be implemented as a computer device or multiple computer devices that communicate with multiple electronic devices (110, 120, 130, 140) via a network (170) to provide commands, codes, files, content, services, etc. For example, the server (150) may be a system that implements a method for providing a 3D map-based service based on a service platform operated by multiple electronic devices (110, 120, 130, 140) connected via a network (170).

[0052]

[0053] FIG. 2 is a block diagram illustrating an example of a computer device according to one embodiment. Each of the plurality of electronic devices (110, 120, 130, 140) or servers (150, 160) described above may be implemented by the computer device (200) illustrated in FIG. 2.

[0054] The computer device (200) may include a memory (210), a processor (220), a communication interface (230), and an input / output interface (240), as illustrated in FIG. 2. The memory (210) may be a computer-readable recording medium, and may include a non-permanent mass storage device such as a random access memory (RAM), a read only memory (ROM), and a disk drive. Here, the non-permanent mass storage device such as a ROM and a disk drive may be included in the computer device (200) as a separate permanent storage device distinct from the memory (210). In addition, the memory (210) may store an operating system and at least one program code. These software components may be loaded into the memory (210) from a computer-readable recording medium separate from the memory (210). The separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, a memory card, etc. In another embodiment, the software components may be loaded into the memory (210) via a communication interface (230) other than a computer-readable recording medium. For example, the software components may be loaded into the memory (210) of the computer device (200) based on a computer program installed by files received over a network (170).

[0055] The processor (220) may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to the processor (220) via the memory (210) or the communication interface (230). For example, the processor (220) may be configured to execute instructions received according to program code stored in a storage device such as the memory (210).

[0056] The communication interface (230) may provide a function for the computer device (200) to communicate with other devices (e.g., the storage devices described above) via the network (170). For example, requests, commands, data, files, etc. generated by the processor (220) of the computer device (200) according to program codes stored in a recording device such as the memory (210) may be transmitted to other devices via the network (170) under the control of the communication interface (230). Conversely, signals, commands, data, files, etc. from other devices may be received by the computer device (200) via the communication interface (230) of the computer device (200) via the network (170). Signals, commands, data, etc. received via the communication interface (230) may be transmitted to the processor (220) or the memory (210), and files, etc. may be stored in a storage medium that the computer device (200) may further include.

[0057] The input / output interface (240) may be a means for interfacing with an input / output device (250). For example, the input device may include a device such as a microphone, keyboard, or mouse, and the output device may include a device such as a display or speaker. As another example, the input / output interface (240) may be a means for interfacing with a device that integrates input and output functions, such as a touchscreen. The input / output device (250) may also be configured as a single device with the computer device (200).

[0058] Furthermore, in other embodiments, the computer device (200) may include fewer or more components than those illustrated in FIG. 2. However, it is not necessary to explicitly illustrate most conventional components. For example, the computer device (200) may further include components such as a lidar, a scanner, a camera, and an IMU sensor to scan a target space and obtain scanned space information when implementing each of the plurality of electronic devices (110, 120, 130, and 140).

[0059] Below, specific embodiments of a 3D map-based service provision method and system, and a service platform are described.

[0060]

[0061] FIG. 3 is a block diagram illustrating an example of components that the processor illustrated in FIG. 2 may include, FIG. 4 is a flow chart illustrating a method for providing a 3D map-based service that the computer device illustrated in FIG. 2 may perform, and FIGS. 5 to 7 are drawings for explaining specifying a user's location in the 3D map-based service providing method illustrated in FIG. 4.

[0062] In embodiments of the present invention, a computer device (200) can execute a 3D map-based service provision method that enables virtual / augmented reality services or navigation services to be provided through a portable terminal carried by a non-expert by applying image-based VPS technology. Hereinafter, a 3D map refers to a virtual space that simulates a real target space.

[0063] In addition, the computer device (200) can execute a method for providing a 3D map-based service based on a 3D map-based service platform that can be used even by non-experts.

[0064] To this end, a 3D map-based service providing system, which is a subject performing a 3D map-based service providing method, may be configured in the computer device (200). For example, the 3D map-based service providing system may be implemented in the form of an independently operating program, or may be implemented in the form of an in-app of a dedicated application so as to be operable on the dedicated application.

[0065] The processor (220) of the computer device (200) may be implemented as a component for performing the 3D map-based service providing method according to FIG. 4. For example, the processor (220) may include a specific unit (310) and a provision unit (320) as illustrated in FIG. 3 so as to perform steps (S410 to S420) illustrated in FIG. 4. Depending on the embodiment, the components of the processor (220) may be selectively included in or excluded from the processor (220). Furthermore, depending on the embodiment, the components of the processor (220) may be separated or merged to express the function of the processor (220).

[0066] These processors (220) and components of the processor (220) can control the computer device (200) to perform steps (S410 to S420) included in the 3D map-based service providing method of FIG. 4. For example, the processor (220) and components of the processor (220) can be implemented to execute instructions according to the code of the operating system included in the memory (210) and the code of at least one program.

[0067] Here, the components of the processor (220) may be representations of different functions performed by the processor (220) according to commands provided by program codes stored in the computer device (200). For example, the provision unit (320) may be used as a functional representation of the processor (220) that controls the computer device (200) to provide a 3D map-based service based on a specific user's location on a 3D map.

[0068] The processor (220) can read necessary commands from the memory (210) loaded with commands related to the control of the computer device (200). In this case, the read commands may include commands for controlling the processor (220) to execute steps (S410 to S420) to be described later.

[0069] Before step (S410), the processor (220) (more precisely, a specific unit (310) included in the processor (220)) can capture an image (500) of a target space using a portable terminal to obtain an image (500).

[0070] At this time, the processor (220) (more precisely, a specific unit (310) included in the processor (220)) provides a guide and template to assist a non-expert user in taking an image through a portable terminal through a service platform on the portable terminal, thereby ensuring that an image (500) having a minimum quality for application of image-based VPS technology is obtained.

[0071] In addition, before step (S410), the processor (220) (more precisely, a specific unit (310) included in the processor (220)) can obtain point cloud information for the image (500) from the image (500) as illustrated in FIG. 5 and generate a 3D point cloud model corresponding to the image.

[0072] The 3D map (600) may include point cloud information, as shown in FIG. 6, which has already been created and produced before the 3D map-based service provision method is performed.

[0073] In step (S410), the processor (220) (more precisely, a specific unit (310) included in the processor (220)) can specify the location of the user on a 3D map generated corresponding to the target space based on an image (500) acquired through a portable terminal of the user located in the target space.

[0074] In more detail, in step (S410), the processor (220) (more precisely, a specific unit (310) included in the processor (220)) can analyze and match at least one feature point of the image (500) with at least one feature point of the three-dimensional map (600) to determine the location of the user defined in the coordinate system of the three-dimensional map (600). That is, the processor (220) (more precisely, a specific unit (310) included in the processor (220)) can analyze and match at least one feature point of the image (500) with at least one feature point of the three-dimensional map (600) to determine which area of ​​the three-dimensional map (600) the space corresponding to the image (500) matches, thereby determining the location of the user on the three-dimensional map (600).

[0075] For example, the processor (220) (more precisely, a specific unit (310) included in the processor (220)) can analyze and match at least one feature point of the image (500) with at least one feature point of the three-dimensional map (600) to confirm that the space corresponding to the image (500) matches a specific area (700) on the three-dimensional map (600) as illustrated in FIG. 7. Accordingly, the processor (220) (more precisely, a specific unit (310) included in the processor (220)) can determine that the user exists at a location where the specific area (700) on the three-dimensional map (600) is in the user's field of view.

[0076] The matching accuracy of the space corresponding to the image (500) on the 3D map (600) is affected by the reliability of each of at least one feature point of the image (500) that is analyzed and matched with each other and at least one feature point of the 3D map (600).

[0077] Accordingly, in step (S410), the processor (220) (more precisely, a specific unit (310) included in the processor (220)) may select at least one feature point (at least one feature point of the image (500)) to be analyzed and matched based on the reliability of each of the feature points included in the image (500) and at least one feature point (at least one feature point of the 3D map (600)) to be analyzed and matched based on the reliability of each of the feature points included in the 3D map (600). For example, the processor (220) (more precisely, a specific unit (310) included in the processor (220)) may select at least one feature point (at least one feature point of the image (500)) having a preset reference value (a minimum reference value that ensures quality that can be analyzed and matched) or higher based on the reliability of each of the feature points included in the unknown, and may select at least one feature point (at least one feature point of the 3D map (600)) having a preset reference value (a minimum reference value that ensures quality that can be analyzed and matched) or higher based on the reliability of each of the feature points included in the 3D map (600).

[0078] The reliability of each of the feature points described above can be calculated based on the generation accuracy of each of the feature points. For example, the higher the generation accuracy of each feature point, the higher the reliability can be calculated. For a more specific example, the reliability of each of the feature points included in the image (500) can be calculated based on the generation accuracy of each of the feature points included in the image (500), and the reliability of each of the feature points included in the 3D map (600) can be calculated based on the generation accuracy of each of the feature points included in the 3D map (600).

[0079] The generation accuracy of each feature point can be evaluated by analyzing the correlation between the actual shape of the target space and the 3D map or the photographed image generated for the target space. That is, in step (S410), the processor (220) (more precisely, a specific unit (310) included in the processor (220)) analyzes the correlation between a plurality of actual spaces and the 3D map or the photographed image generated for the actual spaces, and uses the learned generation accuracy artificial intelligence model to calculate the reliability of each of the feature points of the 3D map and the feature points of the image based on the generation accuracy of each of the feature points of the 3D map generated for the target space and the feature points of the photographed image, thereby selecting at least one feature point to be analyzed and matched from each of the 3D map and the image.

[0080] In addition, in step (S410), the processor (220) (more precisely, a specific unit (310) included in the processor (220)) not only specifies the user's location on the 3D map (600), but also analyzes and matches at least one feature point of the image (500) with at least one feature point of the 3D map (600), thereby specifying the user's pose defined by roll, pitch, and yaw on the 3D map (600). That is, the processor (220) (more precisely, a specific unit (310) included in the processor (220)) analyzes and matches at least one feature point of the image (500) with at least one feature point of the 3D map (600), thereby confirming which area on the 3D map (600) the space corresponding to the image (500) matches, thereby specifying the user's pose on the 3D map (600).

[0081] For example, the processor (220) (more precisely, a specific unit (310) included in the processor (220)) can analyze and match at least one feature point of the image (500) with at least one feature point of the three-dimensional map (600) to confirm that the space corresponding to the image (500) matches a specific area (700) on the three-dimensional map (600) as illustrated in FIG. 7. Accordingly, the processor (220) (more precisely, a specific unit (310) included in the processor (220)) can determine that the user is taking a pose defined by roll, pitch, and yaw in a direction in which the specific area (700) on the three-dimensional map (600) is in the field of view.

[0082] In step (S420), the processor (220) (more precisely, the provision unit (320) included in the processor (220)) can provide a 3D map-based service based on the location of a user specified on a 3D map (600).

[0083] More specifically, the processor (220) (more precisely, the provision unit (320) included in the processor (220)) can generate and provide a 3D map-based service by augmenting at least one content object on the 3D map (600) based on the location of a user specified on the 3D map (600).

[0084] Hereinafter, the 3D map-based service may include a virtual / augmented reality service in which a responsive object capable of interaction with a user is augmented on a 3D map (600) or a navigation service in which a display object (a display object that guides a user to a route) that provides information to a user is augmented on a 3D map (600).

[0085] Accordingly, at least one content object augmented on the 3D map (600) may include at least one of a responsive object that can interact with a user on the 3D map (600) or a display object that provides information to a user on the 3D map (600).

[0086] At this time, when augmenting at least one content object on the 3D map (600), a user input generated through a service platform provided on a portable terminal may be reflected. For example, the processor (220) (more precisely, the provision unit (320) included in the processor (220)) may adjust the augmented position and pose of at least one content object on the 3D map (600) in response to a user input generated through the service platform provided on the portable terminal.

[0087]

[0088] As described above, by applying image-based VPS technology without using sensors and specifying the user's location and pose on a 3D map, a technical effect can be achieved that enables virtual / augmented reality services or navigation services to be provided through portable terminals carried by non-experts.

[0089] In addition, a technical effect of reducing the complexity and difficulty of object augmentation can be achieved by augmenting objects in a 3D map through a service platform on a portable terminal carried by a non-expert user without the use of specialized computer graphics software by a skilled expert.

[0090]

[0091] The devices described above may be implemented as hardware components, software components, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be implemented using one or more general-purpose computers or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone; however, one of ordinary skill in the art will recognize that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing unit may include multiple processors, or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

[0092] Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing device to perform a desired operation or may independently or collectively command the processing device. The software and / or data may be embodied in any type of machine, component, physical device, computer storage medium, or device for interpretation by the processing device or for providing instructions or data to the processing device. The software may also be distributed over networked computer systems and stored or executed in a distributed manner. The software and data may be stored on one or more computer-readable recording media.

[0093] The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. In this case, the medium may be one that continuously stores a computer-executable program or one that temporarily stores it for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or multiple hardware combinations, and is not limited to a medium directly connected to a computer system, but may also be distributed over a network. Examples of the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and those configured to store program commands, including ROM, RAM, and flash memory. In addition, examples of other media may include recording media or storage media managed by app stores that distribute applications, sites that supply or distribute various software, servers, etc.

[0094] In one embodiment of the present invention, a processor (220) executes a deep learning algorithm to automatically extract feature points from an image, and AI corrects the matching results. The input data for AI-based feature point extraction may include visual data collected from an image sensor. The AI ​​model analyzes this image to derive meaningful feature points, thereby reducing errors in location identification and mapping, thereby providing a sophisticated 3D map.

[0095] In one embodiment of the present invention, the processor (220) collects and processes data from IoT sensors to perform real-time map updates and location tracking, and the specific unit (310) determines specific environmental conditions or user locations based on data collected from the IoT sensors and dynamically updates a 3D map (600) in real-time according to the IoT sensor data. This map may be characterized by being sensitive to real-time changes and reflecting more accurate location information and environmental data.

[0096] In one embodiment of the present invention, the processor (220) processes location information of multiple users in real time and transmits this data to other users for collaboration, while the provider (320) can provide users with information on the locations and actions of other users in real time. A 3D map (600) shared by all users can be updated in real time, reflecting each user's location and interaction information.

[0097] The area (700) corresponding to the space of the 3D map image, which is a virtual space where each user can interact, can perform collaboration through real-time updated information. It can include features that allow multiple users to manipulate and share virtual objects in the same space in an AR / VR environment.

[0098] In one embodiment of the present invention, a method for providing a 3D map-based service performed by a computer device may include a step of specifying a location of a user on a 3D map generated corresponding to a target space based on an image acquired through a portable terminal of a user located in the target space, and a step of providing a 3D map-based service based on the location of the user specified on the 3D map.

[0099] In one embodiment of the present invention, the step of specifying may be a method for providing a 3D map-based service, characterized in that it includes a step of analyzing and matching at least one feature point of the image and at least one feature point of the 3D map to specify the location of the user defined in the coordinate system of the 3D map.

[0100] In one embodiment of the present invention, the step of specifying may further include a step of analyzing and matching at least one feature point of the image and at least one feature point of the 3D map to specify the pose of the user defined by roll, pitch, and yaw on the 3D map. The method may be a 3D map-based service providing method.

[0101] In one embodiment of the present invention, the step of specifying may be a method for providing a 3D map-based service, characterized in that it includes a step of selecting at least one feature point to be analyzed and matched based on the reliability of each of the feature points included in the image, and a step of selecting at least one feature point to be analyzed and matched based on the reliability of each of the feature points included in the 3D map.

[0102] In one embodiment of the present invention, the reliability of each of the feature points included in the image is calculated based on the generation accuracy of each of the feature points included in the image, and the reliability of each of the feature points included in the 3D map is calculated based on the generation accuracy of each of the feature points included in the 3D map. This may be a 3D map-based service providing method.

[0103] In one embodiment of the present invention, the providing step may be a method for providing a 3D map-based service, characterized in that it includes a step of augmenting at least one content object on the 3D map based on a location of a user specified on the 3D map.

[0104] In one embodiment of the present invention, the at least one content object may be a 3D map-based service providing method characterized in that it includes at least one of a responsive object that can interact with the user on the 3D map or a display object that provides information to the user on the 3D map.

[0105] In one embodiment of the present invention, the augmenting step may be a method for providing a 3D map-based service, characterized in that it further includes a step of adjusting the augmented position and pose of the at least one content object on the 3D map in response to a user input generated through a service platform provided on a portable terminal.

[0106] In one embodiment of the present invention, a computer-readable recording medium having recorded thereon a computer program for executing a method for providing a 3D map-based service on a computer device, wherein the method for providing a 3D map-based service may include a step of specifying a location of a user on a 3D map generated corresponding to a target space based on an image acquired through a portable terminal of a user located in the target space, and a step of providing a 3D map-based service based on the location of the user specified on the 3D map.

[0107] In one embodiment of the present invention, a computer device for performing a method for providing a 3D map-based service may include at least one processor configured to execute computer-readable commands, wherein the at least one processor may be a computer device including a specific unit for specifying a location of a user on a 3D map generated corresponding to a target space based on an image acquired through a portable terminal of a user located in a target space, and a providing unit for providing a 3D map-based service based on the location of the user specified on the 3D map.

[0108] In one embodiment of the present invention, the specific unit may be a computer device characterized in that it analyzes and matches at least one feature point of the image and at least one feature point of the three-dimensional map to specify the location of the user defined in the coordinate system of the three-dimensional map.

[0109] In one embodiment of the present invention, the specific unit may be a computer device characterized in that it analyzes and matches at least one feature point of the image and at least one feature point of the three-dimensional map to specify the pose of the user defined by roll, pitch, and yaw on the three-dimensional map.

[0110] In one embodiment of the present invention, the providing unit may be a computer device characterized in that it augments at least one content object on the 3D map based on a location of a user specified on the 3D map.

[0111] In another embodiment of the present invention, a method for providing a 3D map-based service performed by a computer device comprises: a step of analyzing and matching at least one feature point of an image acquired through a portable terminal of a user located in a target space with at least one feature point of the 3D map generated in correspondence to the target space, thereby specifying a location of the user defined in a coordinate system of the 3D map on the 3D map; and a step of providing a 3D map-based service based on the location of the user specified on the 3D map, wherein the step of specifying the location of the user comprises: a step of selecting the at least one feature point to be analyzed and matched based on a reliability of each of the feature points included in the image calculated based on a generation accuracy of each of the feature points included in the image; and a step of selecting the at least one feature point to be analyzed and matched based on a reliability of each of the feature points included in the 3D map calculated based on the generation accuracy of each of the feature points included in the 3D map, wherein the step of specifying the location of the user comprises: a step of acquiring an image by taking an image of the target space using a portable terminal carried by a non-expert; and not stopping at specifying the location of the user on the 3D map. And, by analyzing and matching at least one feature point of the image and at least one feature point of the 3D map, the user's pose defined by Roll, Pitch and Yaw on the 3D map is specified as the user's location, and by using an artificial intelligence model of generation accuracy learned by analyzing the correlation between a plurality of real spaces and the 3D map or the photographed images generated for the real spaces, the reliability of each of the feature points of the 3D map and the feature points of the image is calculated based on the generation accuracy of each of the feature points of the 3D map generated for the target space and the feature points of the photographed image, thereby selecting at least one feature point to be analyzed and matched from each of the 3D map and the image,The step of specifying the location of the user further comprises selecting at least one feature point having a preset reference value (a minimum reference value that ensures quality that can be analyzed and matched) or higher based on the reliability of each of the feature points included in the image, and selecting at least one feature point having a preset reference value (a minimum reference value that ensures quality that can be analyzed and matched) or higher based on the reliability of each of the feature points included in the 3D map, and the step of acquiring an image comprises providing a guide and a template to assist a non-expert user in taking an image using a portable terminal through a service platform on a portable terminal, thereby ensuring a minimum quality for application of image-based VPS technology, and the image comprises obtaining point cloud information for the image from the image to create a 3D point cloud model corresponding to the image, and the 3D map comprises a 3D map including the point cloud information that is already created and produced before the 3D map-based service providing method is performed, wherein the step of taking an image of a target space using a portable terminal carried by the non-expert and acquiring the image comprises providing a sensor and computer graphics through the service platform on the portable terminal. A method for providing a 3D map-based service, characterized in that it includes a feature capable of augmenting an object without software, and the step of providing a 3D map-based service based on a location of a user specified on the 3D map further includes a step of augmenting at least one content object on the 3D map based on the location of the user specified on the 3D map.

[0112] In another embodiment of the present invention, the at least one content object may be a 3D map-based service providing method characterized in that it includes at least one of a responsive object that can interact with the user on the 3D map or a display object that provides information to the user on the 3D map.

[0113] In another embodiment of the present invention, the augmenting step may be a method for providing a 3D map-based service, characterized in that it further includes a step of adjusting the augmented position and pose of the at least one content object on the 3D map in response to a user input generated through a service platform provided on a portable terminal.

[0114] In another embodiment of the present invention, a computer-readable recording medium having recorded thereon a computer program for executing a method for providing a 3D map-based service on a computer device, the method for providing a 3D map-based service includes a step of analyzing and matching at least one feature point of an image acquired through a portable terminal of a user located in a target space with at least one feature point of the 3D map generated in correspondence with the target space, thereby specifying a location of the user defined in a coordinate system of the 3D map on the 3D map, and a step of providing a 3D map-based service based on the location of the user specified on the 3D map, wherein the step of specifying the location of the user includes a step of selecting the at least one feature point to be analyzed and matched based on a reliability of each of the feature points included in the image calculated based on a generation accuracy of each of the feature points included in the image, and a step of selecting the at least one feature point to be analyzed and matched based on a reliability of each of the feature points included in the 3D map calculated based on the generation accuracy of each of the feature points included in the 3D map, and wherein the step of specifying the location of the user includes a step of specifying the location of the user, wherein the step of specifying the location of the user is such that a non-expert can perform the analysis. A step of acquiring an image by taking an image of a target space using a portable terminal in possession, and not only specifying the user's location on the 3D map, but also analyzing and matching at least one feature point of the image with at least one feature point of the 3D map, and specifying the user's location by the user's pose defined by roll, pitch, and yaw on the 3D map, and using an artificial intelligence model with learned generation accuracy by analyzing the correlation between a plurality of real spaces and the 3D map or the taken images generated for the real spaces.The method further comprises: selecting at least one feature point to be analyzed and matched from the 3D map and the image by calculating the reliability of each of the feature points of the 3D map and the feature points of the image based on the generation accuracy of each of the feature points of the 3D map generated for the target space and the feature points of the captured image, wherein the step of specifying the user's location further comprises: selecting at least one feature point having a preset reference value (a minimum reference value that ensures quality that can be analyzed and matched) or higher based on the reliability of each of the feature points included in the image; and selecting at least one feature point having a preset reference value (a minimum reference value that ensures quality that can be analyzed and matched) or higher based on the reliability of each of the feature points included in the 3D map; and the step of acquiring an image comprises: providing a guide and template to assist a non-expert user in taking an image using a portable terminal through a service platform on a portable terminal, thereby ensuring a minimum quality for applying an image-based VPS technology; and the image comprises: obtaining point cloud information for the image from the image to generate a 3D point cloud model corresponding to the image; and the 3D The map is characterized in that the 3D map including the point cloud information is already created and produced before the 3D map-based service providing method is performed, and the step of obtaining an image by taking an image of the target space using a portable terminal carried by the non-expert is characterized in that the object can be augmented without a sensor and computer graphics software through a service platform on the portable terminal, and the step of providing the 3D map-based service based on the location of a user specified on the 3D map is,A computer-readable recording medium may further include a step of augmenting at least one content object on the 3D map based on a location of a user specified on the 3D map.

[0115] In another embodiment of the present invention, a computer device for performing a method for providing a 3D map-based service comprises at least one processor configured to execute computer-readable commands, wherein the at least one processor comprises a specific unit for analyzing and matching at least one feature point of an image acquired through a portable terminal of a user located in a target space with at least one feature point of the 3D map generated in correspondence to the target space, thereby specifying a location of the user defined in a coordinate system of the 3D map on the 3D map, and a providing unit for providing a 3D map-based service based on the location of the user specified on the 3D map, wherein the specific unit selects the at least one feature point to be analyzed and matched based on the reliability of each of the feature points included in the image calculated based on the generation accuracy of each of the feature points included in the image, and selects the at least one feature point to be analyzed and matched based on the reliability of each of the feature points included in the 3D map calculated based on the generation accuracy of each of the feature points included in the 3D map, and comprises a process of capturing an image of the target space by using a portable terminal carried by a non-expert, wherein The specific part further includes a feature that guarantees a minimum quality for application of image-based VPS technology by providing a guide and template to assist a non-expert user in the process of acquiring the image through a service platform on a portable terminal, and acquires point cloud information from the image to create a 3D point cloud model corresponding to the image, and the 3D map is characterized in that the 3D map including the point cloud information is already created and produced before the 3D map-based service providing method is performed.A computer device characterized in that it includes a step of acquiring an image by taking an image of a target space using a portable terminal carried by the non-expert, and augmenting an object without a sensor and computer graphics software through a service platform on the portable terminal.

[0116] In another embodiment of the present invention, the specific unit may be a computer device characterized in that it analyzes and matches at least one feature point of the image and at least one feature point of the three-dimensional map to specify the pose of the user defined by roll, pitch, and yaw on the three-dimensional map.

[0117] In another embodiment of the present invention, the providing unit may be a computer device characterized in that it augments at least one content object on the 3D map based on a location of a user specified on the 3D map.

[0118] Although the embodiments described above have been described by way of limited examples and drawings, those skilled in the art will appreciate that various modifications and variations can be made based on the above teachings. For example, appropriate results can still be achieved even if the described techniques are performed in a different order than described, and / or components of the described systems, structures, devices, circuits, etc. are combined or combined in a different manner than described, or are replaced or substituted with other components or equivalents.

[0119] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims described below.

[0120] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will readily appreciate that the present invention can be readily modified into other specific forms without altering the technical spirit or essential characteristics of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single entity may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined manner.

[0121] The scope of the present invention is indicated by the claims set forth below, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention.

[0122] The mode for carrying out the invention has been described together with the best mode for carrying out the invention above.

[0123] This relates to a method and system for providing a 3D map-based service, and has various industrial applicability as it relates to a method and system for providing a 3D map-based service using image-based VPS (Visual positioning system) technology.

Claims

1. A method for providing a 3D map-based service performed by a computer device, A step of specifying the location of the user on a three-dimensional map generated corresponding to the target space based on an image acquired through a portable terminal of the user located in the target space; and A step for providing a 3D map-based service based on the location of a specific user on the above 3D map. A method for providing a three-dimensional map-based service including:

2. In paragraph 1, The above specific steps are: A step of analyzing and matching at least one feature point of the image and at least one feature point of the three-dimensional map to specify the location of the user defined in the coordinate system of the three-dimensional map. A method for providing a three-dimensional map-based service, characterized by including:

3. In paragraph 2, The above specific steps are: A step of analyzing and matching at least one feature point of the image and at least one feature point of the 3D map to specify the user's pose defined by roll, pitch and yaw on the 3D map. A method for providing a 3D map-based service, characterized by further including:

4. In either paragraph 2 or paragraph 3, The above specific steps are: A step of selecting at least one feature point to be analyzed and matched based on the reliability of each of the feature points included in the image; and A step of selecting at least one feature point to be analyzed and matched based on the reliability of each feature point included in the above 3D map. A method for providing a three-dimensional map-based service, characterized by including:

5. In paragraph 4, The reliability of each feature point included in the image above is It is calculated based on the generation accuracy of each feature point included in the above image, The reliability of each feature point included in the above 3D map is A method for providing a 3D map-based service, characterized in that the accuracy of generation of each feature point included in the 3D map is calculated based on the accuracy of generation.

6. In paragraph 1, The steps provided above are: A step of augmenting at least one content object on the 3D map based on a location of a specific user on the 3D map. A method for providing a three-dimensional map-based service, characterized by including:

7. In paragraph 6, At least one content object of the above, A method for providing a 3D map-based service, characterized in that it includes at least one of a responsive object capable of interacting with the user on the 3D map or a display object that provides information to the user on the 3D map.

8. In paragraph 6, The above augmenting step is, A step of adjusting the augmented position and pose of at least one content object on the three-dimensional map in response to a user input generated through a service platform provided on a portable terminal. A method for providing a 3D map-based service, characterized by further including:

9. A computer-readable recording medium having recorded thereon a computer program for executing a method for providing a 3D map-based service on a computer device, The above 3D map-based service provision method is: A step of specifying the location of the user on a three-dimensional map generated corresponding to the target space based on an image acquired through a portable terminal of the user located in the target space; and A step for providing a 3D map-based service based on the location of a specific user on the above 3D map. A computer-readable recording medium containing: In a computer device performing a method for providing a 10.3-dimensional map-based service, At least one processor configured to execute computer-readable instructions Including, At least one processor of the above, A specific unit that specifies the location of the user on a three-dimensional map generated corresponding to the target space based on an image acquired through a portable terminal of the user located in the target space; and A service provider that provides 3D map-based services based on the location of a specific user on the above 3D map. A computer device comprising:

11. In paragraph 10, The above specific part, A computer device characterized in that it analyzes and matches at least one feature point of the image and at least one feature point of the three-dimensional map to specify the location of the user defined in the coordinate system of the three-dimensional map.

12. In paragraph 11, The above specific part, A computer device characterized in that it analyzes and matches at least one feature point of the image and at least one feature point of the three-dimensional map to specify the pose of the user defined by roll, pitch and yaw on the three-dimensional map.

13. In paragraph 10, The above-mentioned provision department, A computer device characterized by augmenting at least one content object on a three-dimensional map based on a location of a specific user on the three-dimensional map.

Images