Method and system for generating three-dimensional map for improving accuracy of vps

Abstract

A method and a system for generating a three-dimensional map for improving the accuracy of a visual positioning system (VPS) are disclosed. The method for generating a three-dimensional map according to an embodiment may include the steps of: acquiring consecutive images for a target space; and generating a three-dimensional map corresponding to the space itself, except for at least one object having movement in the target space, on the basis of the consecutive images.

Description

Method and system for generating a 3D map to improve VPS accuracy

[0001] The following examples relate to a method and system for generating a 3D map, and to a technique for generating a 3D map that improves the accuracy of a Visual Positioning System (VPS).

[0002] With the spread of smartphones, location information services used in PC or laptop environments have evolved to be used in mobile terminal environments.

[0003] Microsoft has released a service called Photosynth that searches and collects photos shared on the Internet, calculates the shooting location, direction, and 3D coordinates of key features of each video, and provides the collected photos as if they were actual navigation, rather than a simple thumbnail view. Google is also providing a 3D map service by uploading 3D models modeled using 3D modeling tools such as Google Sketchup to Google Earth.

[0004] Conventional techniques for creating 3D models like this have the disadvantage of high production complexity and difficulty, as they presuppose that skilled designers use specialized computer graphics software such as Maya and 3D Max.

[0005] Accordingly, there is a need to propose a 3D map generation technology that can be used even by non-experts by reducing the complexity and difficulty of production.

[0006] Meanwhile, the generated 3D map is used in a virtual and augmented reality service in which at least one content object is augmented as the user's location on the 3D map is specified based on a Visual positioning system (VPS).

[0007] At this time, if the VPS accuracy is low, the user's location on the 3D map cannot be accurately identified, so at least one content cannot be augmented in the location where it should be, and thus proper virtual and augmented reality services cannot be provided.

[0008] Therefore, there is a need to propose a technique to improve VPS accuracy.

[0009] A technical problem to be achieved by one embodiment is to propose a method and system for improving VPS accuracy by generating a 3D map corresponding to the space itself excluding at least one object having motion, taking into account the characteristic that VPS accuracy deteriorates when a 3D map is generated including a moving object rather than the space itself.

[0010] At this time, one embodiment proposes a method and system for creating a 3D map using a 3D map creation service platform that can be used even by non-experts, in order to reduce the complexity and difficulty of production.

[0011] The technical problems to be achieved by the embodiments are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the embodiments belong from the description below.

[0012] In order to achieve the above technical task, a method for generating a three-dimensional map performed by a computer device according to one embodiment may include the steps of: acquiring consecutive images of a target space; and generating, based on the consecutive images, a three-dimensional map corresponding to the space itself, excluding at least one object moving in the target space.

[0013] According to one aspect, the generating step may be characterized by including a step of detecting and then removing at least one object having the movement from the consecutive images.

[0014] According to another aspect, the removing step may be characterized by including a step of detecting at least one object having a change in position or shape in the continuous images as at least one object having the movement.

[0015] According to another aspect, the step of removing may further include a step of performing inpainting on an area from which the object having the movement has been removed.

[0016] According to another aspect, the performing step may be characterized as a step of performing inpainting using background information from the edge toward the inside of the area where the object having the movement has been removed.

[0017] According to another aspect, the generating step may be characterized by including: a step of collecting scan point cloud information from consecutive images on which the inpainting has been performed; and a step of generating the three-dimensional map using the scan point cloud information.

[0018] According to another aspect, the collecting step may be characterized as a step of collecting the scan point cloud information from the continuous images that only include information corresponding to the space itself, excluding at least one object having the movement.

[0019] According to another aspect, the obtaining step may further include a step of providing a guide and template for scanning the target space to enable obtaining the continuous images using a portable terminal through a service platform on the portable terminal.

[0020] According to another aspect, the generating step may be characterized by including a step of generating the three-dimensional map by reflecting user input generated through a service platform provided on a portable terminal.

[0021] According to another aspect, the generating step may further include a step of including at least one content object on the three-dimensional map in response to a user input generated through a service platform provided on the portable terminal.

[0022] A computer-readable recording medium having recorded thereon a computer program for executing a method for generating a three-dimensional map according to one embodiment of the present invention on a computer device, wherein the three-dimensional map generating method may include: acquiring consecutive images of a target space; and generating, based on the consecutive images, a three-dimensional map corresponding to the space itself, excluding at least one object moving in the target space.

[0023] A computer device for performing a three-dimensional map generation method according to one embodiment may include at least one processor configured to execute computer-readable commands, wherein the at least one processor may include: an acquisition unit for acquiring continuous images of a target space; and a generation unit for generating, based on the continuous images, a three-dimensional map corresponding to the space itself, excluding at least one object moving in the target space.

[0024] According to one aspect, the generating unit is a computer device characterized in that it detects and then removes at least one object having the movement from the continuous images.

[0025] According to another aspect, the generating unit may be characterized in that it performs inpainting on an area from which an object having the movement has been removed.

[0026]

[0027] One embodiment may propose a method and system for improving VPS accuracy by generating a 3D map corresponding to the space itself excluding at least one object with movement, taking into account the characteristic that VPS accuracy deteriorates when a 3D map is generated including a moving object rather than the space itself.

[0028] At this time, one embodiment proposes a method and system for creating a 3D map using a 3D map creation service platform that can be used even by non-experts, thereby reducing the complexity and difficulty of production.

[0029] It should be understood that the described technical effects are not limited to the effects described above, but include all effects that can be inferred from the composition of the invention described in the detailed description below or the claims.

[0030] Figure 1 is a diagram illustrating an example of a service environment according to one embodiment.

[0031] FIG. 2 is a block diagram illustrating an example of a computer device according to one embodiment.

[0032] FIG. 3 is a block diagram illustrating an example of components that the processor illustrated in FIG. 2 may include.

[0033] FIG. 4 is a flow chart illustrating a method for generating a three-dimensional map that can be performed by the computer device illustrated in FIG. 2.

[0034] FIGS. 5 to 8 are drawings for explaining the creation of a three-dimensional map corresponding to the space itself, excluding at least one object having movement, in the three-dimensional map creation method illustrated in FIG. 4.

[0035] Hereinafter, the present invention will be described with reference to the attached drawings. However, the present invention can be implemented in various different forms and is therefore not limited to the embodiments described herein. In the drawings, irrelevant parts have been omitted for clarity of description, and similar parts have been designated with similar reference numerals throughout the specification.

[0036] Throughout the specification, when a part is said to be "connected (connected, contacted, or coupled)" to another part, this includes not only cases where it is "directly connected," but also cases where it is "indirectly connected" with another part in between. Furthermore, when a part is said to "include" a component, this does not exclude other components, but rather implies that it may include other components, unless otherwise specifically stated.

[0037] The terminology used herein is merely used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this specification, it should be understood that the terms "comprises" or "has" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0038]

[0039] In the following embodiments, a method and system for improving VPS accuracy are described by generating a 3D map corresponding to the space itself excluding at least one object having motion, taking into account the characteristic that VPS accuracy deteriorates when a 3D map is generated that includes a moving object rather than the space itself.

[0040] In addition, the following embodiments describe a method and system for creating a 3D map using a 3D map creation service platform (hereinafter, “service platform”) that can be used by non-experts and reduces the complexity and difficulty of creation.

[0041] The service platform is implemented in the form of a dedicated program, application or web page that runs on a portable terminal carried by a user, and can be used as an interface in the process of performing a 3D map creation method.

[0042] The method for generating a 3D map can be performed by a 3D map generation system implemented in a server or a portable terminal comprising at least one computer device including a processor, and the 3D map generation system can be operated under the control of a computer program. The above-described computer program can be stored in a computer-readable recording medium so as to be coupled with a computer device and execute the 3D map generation method on the computer device. The computer program described herein may have the form of an independent program package, or may have the form of an independent program package pre-installed on a computer device and linked with an operating system or other program packages.

[0043]

[0044] FIG. 1 is a diagram illustrating an example of a service environment according to one embodiment. The service environment of FIG. 1 represents an example including a plurality of electronic devices (110, 120, 130, 140), a plurality of servers (150, 160), and a network (170).

[0045] This drawing 1 is an example for explaining the invention, and the number of electronic devices or servers is not limited to that of drawing 1. In addition, the service environment of drawing 1 merely illustrates one example of environments applicable to the present embodiments, and the environments applicable to the present embodiments are not limited to the service environment of drawing 1.

[0046] The plurality of electronic devices (110, 120, 130, 140) may be mobile terminals or portable terminals implemented as computer devices.

[0047] In particular, each of the plurality of electronic devices (110, 120, 130, 140) may mean one of various physical computer devices that can communicate with a server (150, 160) via a network (170) using a wireless or wired communication method, assuming that the plurality of electronic devices includes a lidar, a scanner, a camera, an IMU sensor, etc. so as to scan a target space while being held by a user to obtain scan space information (e.g., continuous images). As an example, examples of the plurality of electronic devices (110, 120, 130, 140) include a smart phone, a mobile phone, a navigation system, a laptop, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), a tablet PC, etc.

[0048] Here, the electronic devices (110, 120, 130, 140) may mean portable terminals that are carried by a user who wants to scan a target space to create a 3D map and receive the created 3D map.

[0049] The communication method between the electronic devices (110, 120, 130, 140) and the servers (150, 160) is not limited, and may include not only a communication method utilizing a communication network (e.g., a mobile communication network, a wired Internet, a wireless Internet, a broadcasting network) that the network (170) may include, but also short-range wireless communication between the devices. For example, the network (170) may include any one or more of a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), the Internet, and the like. In addition, the network (170) may include any one or more of a network topology including, but not limited to, a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree, or a hierarchical network.

[0050] Each of the servers (150, 160) may be implemented as a computer device or multiple computer devices that communicate with multiple electronic devices (110, 120, 130, 140) via a network (170) to provide commands, codes, files, content, services, etc. For example, the server (150) may be a system that implements a three-dimensional map creation method based on a service platform operated by multiple electronic devices (110, 120, 130, 140) connected via a network (170).

[0051]

[0052] FIG. 2 is a block diagram illustrating an example of a computer device according to one embodiment. Each of the plurality of electronic devices (110, 120, 130, 140) or servers (150, 160) described above may be implemented by the computer device (200) illustrated in FIG. 2.

[0053] The computer device (200) may include a memory (210), a processor (220), a communication interface (230), and an input / output interface (240), as illustrated in FIG. 2. The memory (210) may be a computer-readable recording medium, and may include a non-permanent mass storage device such as a random access memory (RAM), a read only memory (ROM), and a disk drive. Here, the non-permanent mass storage device such as a ROM and a disk drive may be included in the computer device (200) as a separate permanent storage device distinct from the memory (210). In addition, the memory (210) may store an operating system and at least one program code. These software components may be loaded into the memory (210) from a computer-readable recording medium separate from the memory (210). The separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, a memory card, etc. In another embodiment, the software components may be loaded into the memory (210) via a communication interface (230) other than a computer-readable recording medium. For example, the software components may be loaded into the memory (210) of the computer device (200) based on a computer program installed by files received over a network (170).

[0054] The processor (220) may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to the processor (220) via the memory (210) or the communication interface (230). For example, the processor (220) may be configured to execute instructions received according to program code stored in a storage device such as the memory (210).

[0055] The communication interface (230) may provide a function for the computer device (200) to communicate with other devices (e.g., the storage devices described above) via the network (170). For example, requests, commands, data, files, etc. generated by the processor (220) of the computer device (200) according to program codes stored in a recording device such as the memory (210) may be transmitted to other devices via the network (170) under the control of the communication interface (230). Conversely, signals, commands, data, files, etc. from other devices may be received by the computer device (200) via the communication interface (230) of the computer device (200) via the network (170). Signals, commands, data, etc. received via the communication interface (230) may be transmitted to the processor (220) or the memory (210), and files, etc. may be stored in a storage medium that the computer device (200) may further include.

[0056] The input / output interface (240) may be a means for interfacing with an input / output device (250). For example, the input device may include a device such as a microphone, keyboard, or mouse, and the output device may include a device such as a display or speaker. As another example, the input / output interface (240) may be a means for interfacing with a device that integrates input and output functions, such as a touchscreen. The input / output device (250) may also be configured as a single device with the computer device (200).

[0057] Furthermore, in other embodiments, the computer device (200) may include fewer or more components than those illustrated in FIG. 2. However, it is not necessary to explicitly illustrate most conventional components. For example, the computer device (200) may further include components such as a lidar, a scanner, a camera, and an IMU sensor to scan a target space and obtain scanned space information when implementing each of the plurality of electronic devices (110, 120, 130, and 140).

[0058] Below, specific embodiments of a 3D map creation method and system, and a service platform are described.

[0059]

[0060] FIG. 3 is a block diagram illustrating an example of components that the processor illustrated in FIG. 2 may include, FIG. 4 is a flow chart illustrating a three-dimensional map generation method that the computer device illustrated in FIG. 2 may perform, and FIGS. 5 to 8 are drawings for explaining generation of a three-dimensional map corresponding to space itself, excluding at least one object having movement, in the three-dimensional map generation method illustrated in FIG. 4.

[0061] In embodiments of the present invention, the computer device (200) can execute a 3D map generation method that improves VPS accuracy by generating a 3D map corresponding to the space itself excluding at least one object (510, 520) that has movement, taking into account the characteristic that VPS accuracy deteriorates when a 3D map is generated that includes a moving object rather than the space itself. Hereinafter, a 3D map means a virtual space that simulates a real target space.

[0062] Additionally, the computer device (200) can execute a 3D map creation method based on a 3D map creation service platform that can be used even by non-experts.

[0063] To this end, a computer device (200) may be configured with a 3D map generation system, which is a subject performing a 3D map generation method. For example, the 3D map generation system may be implemented in the form of an independently operating program, or may be implemented in the form of an in-app of a dedicated application so as to be operable on the dedicated application.

[0064] The processor (220) of the computer device (200) may be implemented as a component for performing the 3D map generation method according to FIG. 4. For example, the processor (220) may include an acquisition unit (310) and a generation unit (320) as illustrated in FIG. 3 so as to perform steps (S410 to S420) illustrated in FIG. 4. Depending on the embodiment, the components of the processor (220) may be selectively included in or excluded from the processor (220). In addition, depending on the embodiment, the components of the processor (220) may be separated or merged to express the function of the processor (220).

[0065] These processors (220) and components of the processor (220) can control the computer device (200) to perform steps (S410 to S420) included in the three-dimensional map generation method of FIG. 4. For example, the processor (220) and components of the processor (220) can be implemented to execute instructions according to the code of the operating system included in the memory (210) and the code of at least one program.

[0066] Here, the components of the processor (220) may be representations of different functions performed by the processor (220) according to commands provided by program code stored in the computer device (200). For example, the acquisition unit (310) may be used as a functional representation of the processor (220) that controls the computer device (200) to acquire consecutive images (500) of the target space.

[0067] The processor (220) can read necessary commands from the memory (210) loaded with commands related to the control of the computer device (200). In this case, the read commands may include commands for controlling the processor (220) to execute steps (S410 to S420) to be described later.

[0068] In step (S410), the processor (220) (more precisely, the acquisition unit (310) included in the processor (220)) can acquire consecutive images (500) of the target space.

[0069] At this time, before step (S410), the processor (220) (more precisely, the acquisition unit (310) included in the processor (220)) may provide a guide and template for scanning the target space to enable acquisition of consecutive images (500) using the portable terminal through a service platform on the portable terminal.

[0070] This configuration takes into account the fact that a portable terminal possessed by a non-professional user is different from a scanning terminal for a professional user. A guide and template for photographing and scanning a target space may be provided through a service platform on the portable terminal so that a non-professional user can photograph a target space through the portable terminal and obtain continuous images (500). For example, a guide guiding the user in the direction of photographing the target space may be provided through the service platform on the portable terminal, or a template indicating a photographing area of ​​the target space may be provided through the service platform on the portable terminal.

[0071] In step (S420), the processor (220) (more precisely, the generation unit (320) included in the processor (220)) can generate a three-dimensional map corresponding to the space itself, excluding at least one object (510, 520) that moves in the target space, based on the consecutive images (500).

[0072] In more detail, in step (S420), the processor (220) (more precisely, the generation unit (320) included in the processor (220)) detects at least one object (510, 520) having a movement in the continuous images (500) as illustrated in FIG. 5, removes at least one object (510, 520) having a movement as illustrated in FIG. 6, and performs inpainting on the area (610, 620) from which at least one object (510, 520) having a movement has been removed as illustrated in FIG. 7, thereby collecting scan point cloud information from the continuous images (700) (continuous images on which inpainting has been performed as illustrated in FIG. 7) that include only information corresponding to the space itself, excluding at least one object (510, 520) having a movement.

[0073] However, in another embodiment according to the present invention, at least one object having the movement can be recognized by various types such as an adult, a child (child), an infant, and / or a pet such as a dog, and the distance moved or the movement state or the movement of a joint (which can be linked to a Kinect Sensor, etc.) for a set period of time is recognized, and the object is automatically classified by type or categorized through a computer device (200) connected to an external database according to the average value of the movement distance for a set period of time for each type or the movement pattern, which has been learned / stored in advance in the computer device (200), and the object is classified, processed (image removed), and stored in the computer device (200), so that it can be used for future learning, etc.

[0074] In addition, according to another embodiment of the present invention, objects such as temporary stands, stalls, or partially-in-progress repair work (e.g., removal and reinstallation of part of floor tiles) installed in a space where an image is being filmed may also be provided to recognize changes or variations over time and remove them based on this, or output them to an input / output interface (240) so that a user can compare and select images before and after installation of the stand or before and after repair work. That is, in the case of an object that is fixedly positioned for a certain period of time or longer, unlike the aforementioned people or pets, the computer device (200) according to the present invention is fixedly positioned within the image for a set period of time, whereas it is not a permanent object like a building, and thus, depending on the user's selection, an object that is not a moving object (such as a person) or a fixed object (such as a building) among the images located within the shooting range is separately classified (recognized based on whether it moves or changes its position within a set period of time or time), and in the case of such temporary installations (such as stands, signboards, objects subject to temporary maintenance work, etc.), it is possible to selectively provide removal or maintenance within the image depending on the user's selection.

[0075] In step (S420), the processor (220) (more precisely, the generation unit (320) included in the processor (220)) forms a 3D point cloud model (800) using scan point cloud information that includes only information corresponding to the space itself, excluding at least one object (510, 520) having movement as illustrated in FIG. 8, and then generates a 3D mesh model based on the 3D point cloud model (800) and then reflects texture information for the target space to complete the generation of a 3D map.

[0076] Here, the scan point cloud information can be collected by extracting at least one feature point from consecutive images (700) on which inpainting has been performed, and forming a scan point cloud from the at least one feature point.

[0077] In relation to detecting at least one object (510, 520) having movement, the processor (220) (more precisely, the generation unit (320) included in the processor (220)) can detect at least one object having a change in position or shape in the consecutive images (500) as at least one object (510, 520) having movement.

[0078] In relation to performing inpainting on consecutive images (500), the processor (220) (more precisely, the generation unit (320) included in the processor (220)) can perform inpainting by using background information (background information located around the area from which the object with movement has been removed) from the edge toward the inside of the area (610, 620) from which the object with movement (510, 520) has been removed.

[0079] In relation to collecting scan point cloud information, the processor (220) (more precisely, the generation unit (320) included in the processor (220)) may minimize the number and density of feature points extracted from consecutive images (700) so as to satisfy conditions for forming a 3D point cloud model, taking into account that a 3D point cloud model (800) to be formed based on the scan point cloud information will be provided through a service platform on a portable terminal.

[0080] In step (S420), the processor (220) (more precisely, the generation unit (320) included in the processor (220)) can generate a 3D map by reflecting user input generated through a service platform provided on a portable terminal. For example, the processor (220) (more precisely, the generation unit (320) included in the processor (220)) can generate a 3D map by adding or deleting a point cloud to a specific portion in response to user input generated to a specific portion of a 3D point cloud model (800) provided on the service platform.

[0081] In addition, in step (S420), the processor (220) (more precisely, the generation unit (320) included in the processor (220)) may not only generate a 3D map, but may also include at least one content object on the 3D map so that a 3D map including at least one content object is used to provide a 3D map-based service.

[0082] The 3D map generated in step (S420) can be provided to a portable terminal carried by the user through a service platform.

[0083] As described above, a technical effect of improving VPS accuracy in the generated 3D map can be achieved by generating a 3D map based on a series of images (700) that contain only information corresponding to the space itself, excluding at least one object (510, 520) having movement.

[0084] In addition, a technical effect of reducing the complexity and difficulty of production can be achieved by photographing and acquiring consecutive images (500) of a target space while providing guides and templates through a service platform on a portable terminal carried by a non-expert user without the use of specialized computer graphics software by a skilled expert.

[0085]

[0086] The devices described above may be implemented as hardware components, software components, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be implemented using one or more general-purpose computers or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone; however, one of ordinary skill in the art will recognize that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing unit may include multiple processors, or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

[0087] Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing device to perform a desired operation or may independently or collectively command the processing device. The software and / or data may be embodied in any type of machine, component, physical device, computer storage medium, or device for interpretation by the processing device or for providing instructions or data to the processing device. The software may also be distributed over networked computer systems and stored or executed in a distributed manner. The software and data may be stored on one or more computer-readable recording media.

[0088] The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. In this case, the medium may be one that continuously stores a computer-executable program or one that temporarily stores it for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or multiple hardware combinations, and is not limited to a medium directly connected to a computer system, but may also be distributed over a network. Examples of the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and those configured to store program commands, including ROM, RAM, and flash memory. In addition, examples of other media may include recording media or storage media managed by app stores that distribute applications, sites that supply or distribute various software, servers, etc.

[0089] In one embodiment of the present invention, the processor (220) automatically filters out specific types of objects (e.g., furniture, vehicles, etc.) when generating a 3D map from consecutive images (500) using a deep learning-based object recognition module (e.g., YOLO, SSD), and the acquisition unit (310) collects initial data required for generating the 3D map, transmits the filtered object data to the processor (220), and processes the data, and the generation unit (320) generates a 3D map from which specific objects have been removed using the data filtered by the processor (220). Through this, the map can be completed with unnecessary objects removed.

[0090] In one embodiment of the present invention, after a user creates a 3D map, the processor (220) processes the user's input to modify specific portions of the 3D map in real time. Tasks such as texture modification, position and height adjustment are performed, and the acquisition unit (310) collects editing commands provided through the user interface and transmits them to the processor (220) to apply the necessary changes, thereby providing a function for modifying or manipulating specific portions of the map in real time.

[0091] This includes changing the texture of the object, adjusting the position and height, etc., and the generation unit receives the modified data from the processor (220), generates a 3D map that is updated in real time, and can reflect it in real time through the user interface.

[0092] In one embodiment of the present invention, the processor (220) is connected to the AR module, processes map data modified by the user in real time, and transmits the same to the AR device, and the acquisition unit (310) receives data on which the user interacts from the AR device and transmits the data to the processor (220), and the generation unit (320) visualizes the result modified by the user in real time through AR technology, and allows the user to check the result of the 3D map modified by the user in real time through the AR device.

[0093] Users can visualize the changed positions of objects or added objects through AR devices, providing useful feedback for furniture arrangements or space design simulations.

[0094] In one embodiment of the present invention, the processor (220) communicates with a cloud server to synchronize data so that multiple users can edit the same 3D map in real time, and the acquisition unit (310) collects data from each user terminal and transmits it to the cloud server to enable real-time collaboration, thereby providing a cloud-based collaboration function that allows multiple users to work simultaneously.

[0095] The generation unit (320) receives updated data from the cloud server, generates the latest map, and reflects it on all users' terminals so that changes can be immediately synchronized.

[0096] In one embodiment of the present invention, the processor (220) executes a CNN-based neural network model to analyze and learn the size, shape, and movement pattern of a specific object, and improves the performance of the system based on the learned results, and the acquisition unit (310) can collect data necessary for analysis and learning and transmit it to the processor (220).

[0097] The learned results are utilized for subsequent analysis, and the generation unit (320) can automatically generate a 3D map reflecting the analyzed object data based on the learned results. This function may include the ability to automatically analyze the location and size of a specific object and suggest actions to be performed in the next step.

[0098] In one embodiment of the present invention, the processor (220) processes various sensor data (e.g., LiDAR, camera, etc.) to predict the movement of at least one object (510, 520) having movement and remove the object based on the result.

[0099] The acquisition unit (310) collects sensor data and transmits it to the processor (220), providing information necessary for movement prediction. The generation unit (320) can then generate a high-precision 3D map based on the fused data and reflect it in the system. This function can be applied to autonomous vehicles, drones, robots, etc., and can be utilized to update maps in real time.

[0100] In one embodiment of the present invention, a database system for managing multiple frames is introduced, so that the processor (220) manages frames within the database, automatically removes unnecessary previous frames, the acquisition unit (310) collects new frame data and transmits it to the processor (220), and replaces it with the latest frame as needed, and the generation unit (320) can generate and update a 3D map with the latest frame data reflected, which can contribute to large-capacity data processing and storage space optimization.

[0101] In one embodiment of the present invention, the processor (220) can track at least one object (510, 520) moving in real time and update a three-dimensional map in real time accordingly, the acquisition unit (310) collects data necessary for object tracking and transmits it to the processor (220), and the generation unit (320) generates an updated three-dimensional map according to the tracked object information, which can be useful for maintaining an up-to-date map by reflecting an environment that changes in real time.

[0102] In one embodiment of the present invention, the processor (220) can merge and synchronize data collected from different devices in real time based on the cloud, and the acquisition unit (310) can receive data from a cloud server and transmit it to the processor (220) to achieve real-time synchronization. The generation unit (320) can generate a 3D map based on the data synchronized in the cloud to provide consistent information to all users.

[0103] In one embodiment of the present invention, the processor (220) may be characterized by analyzing the quality of an input image and automatically correcting brightness, contrast, distortion, etc. of the image using a computer vision algorithm, and the specific unit (310) may be characterized by evaluating the image quality and performing necessary correction operations before extracting feature points in the image correction operation.

[0104] The provision unit (320) displays a corrected image to the user in real time or provides additional guidance information based on the correction result, and the image can be used in the VPS system after being corrected by the processor (220).

[0105] In one embodiment of the present invention, the processor (220) can process real-time traffic conditions, user location, and IoT sensor data to calculate an optimal route and provide it to the user. The specific unit (310) collects data necessary for route guidance, analyzes the location and surrounding environment in real time, and the provision unit (320) provides the calculated route to the user.

[0106] In one embodiment of the present invention, the processor (220) can recognize an object selected by a user using AI-based object recognition technology and automatically suggest a location to place the object on a 3D map, and the specific unit (310) can analyze the size, shape, location, etc. of the object and select the most suitable location on the 3D map.

[0107] The provision unit (320) can visually display the location or placement direction of an object selected by the user, provide additional information related to the placement of the object, thereby defining the three-dimensional space in which the object will be placed, and determine how the placed object interacts with other elements.

[0108] In one embodiment of the present invention, the processor (220) synchronizes data of multiple users, manages a 3D map updated in real time to merge tasks performed by each user to maintain a consistent map, and the specific unit (310) can collect data input from each user, process it, and transmit it to a cloud server.

[0109] The provision unit (320) is characterized by providing an interface that reflects the current working status of each user in real time and shares the working status with other users, thereby enabling smooth collaboration in a multi-user environment.

[0110] In one embodiment of the present invention, the processor (220) enables the system to calculate and provide real-time feedback when a user manipulates a virtual object in an AR environment. For example, the processor (220) analyzes the suitability of object placement, the possibility of collision, etc., and the specific unit (310) collects data on the user's interactions in the AR environment and generates real-time feedback based on this data.

[0111] The provision unit (320) provides real-time feedback to the user in the form of visual or audio, thereby providing an immediate response to object placement or manipulation, and can maintain the position and state of the virtual object with which the user interacts and reflect real-time feedback.

[0112] In one embodiment of the present invention, the processor (220) analyzes surrounding environmental data to adjust elements for optimizing user experience, such as brightness, sound volume, and color of the interface, and the specific unit (310) collects environmental data from IoT sensors or peripheral devices, detects environmental changes, and the provision unit (320) dynamically changes the interface based on the collected environmental data, thereby providing an optimized experience to the user.

[0113] In one embodiment of the present invention, a method for generating a three-dimensional map performed by a computer device may include a step of acquiring continuous images of a target space and a step of generating a three-dimensional map corresponding to the space itself, excluding at least one object moving in the target space, based on the continuous images.

[0114] In one embodiment of the present invention, the generating step may be a three-dimensional map generating method characterized in that it includes a step of detecting and then removing at least one object having the movement from the continuous images.

[0115] In one embodiment of the present invention, the removing step may be a three-dimensional map generation method characterized in that it includes a step of detecting at least one object having a change in position or shape in the continuous images as at least one object having the movement.

[0116] In one embodiment of the present invention, the removing step may be a method for generating a 3D map, characterized in that it further includes a step of performing inpainting on an area from which the object having the movement has been removed.

[0117] In one embodiment of the present invention, the performing step may be a method for generating a 3D map, characterized in that the performing step is a step of performing inpainting using background information from the edge of the area from which the object having the movement has been removed toward the inside.

[0118] In one embodiment of the present invention, the generating step may be a 3D map generating method characterized in that it includes a step of collecting scan point cloud information from consecutive images on which the inpainting has been performed and a step of generating the 3D map using the scan point cloud information.

[0119] In one embodiment of the present invention, the collecting step may be a three-dimensional map generation method characterized in that it is a step of collecting the scan point cloud information from the continuous images that include only information corresponding to the space itself excluding at least one object having the movement.

[0120] In one embodiment of the present invention, the method for creating a 3D map may further include a step of providing a guide and template for scanning the target space to enable acquisition of the continuous images using a portable terminal through a service platform.

[0121] In one embodiment of the present invention, the generating step may be a 3D map generating method characterized in that it includes a step of generating the 3D map by reflecting a user input generated through a service platform provided on a portable terminal.

[0122] In one embodiment of the present invention, the generating step may be a method for generating a 3D map, characterized in that it further includes a step of including at least one content object on the 3D map in response to a user input generated through a service platform provided on the portable terminal.

[0123] In one embodiment of the present invention, a computer-readable recording medium having recorded thereon a computer program for executing a three-dimensional map generation method on a computer device, wherein the three-dimensional map generation method may include a step of acquiring consecutive images of a target space and a step of generating a three-dimensional map corresponding to the space itself, excluding at least one object moving in the target space, based on the consecutive images.

[0124] In one embodiment of the present invention, a computer device for performing a three-dimensional map generation method may include at least one processor configured to execute computer-readable commands, wherein the at least one processor may be a computer device including an acquisition unit for acquiring continuous images of a target space and a generation unit for generating a three-dimensional map corresponding to the space itself, excluding at least one object moving in the target space, based on the continuous images.

[0125] In one embodiment of the present invention, the generating unit may be a computer device characterized in that it detects and then removes at least one object having the movement from the continuous images.

[0126] In one embodiment of the present invention, the generating unit may be a computer device characterized in that it performs inpainting on an area from which the object having the movement has been removed.

[0127] In another embodiment of the present invention, a method for generating a three-dimensional map performed by a computer device, comprising the steps of: acquiring sequential images of a target space; and generating a three-dimensional map corresponding to the space itself, excluding at least one object having movement in the target space, based on the sequential images, wherein the acquiring step further comprises: providing a guide and template for scanning the target space on a service platform so that the sequential images can be acquired using a portable terminal; and the generating step further comprises: detecting and removing at least one object having movement in the sequential images; generating the three-dimensional map by reflecting a user input generated through a service platform provided on the portable terminal; and including at least one content object on the three-dimensional map in response to a user input generated through the service platform provided on the portable terminal; and wherein the removing step comprises detecting at least one object having a change in position or shape in the sequential images as the at least one object having movement, wherein the generating step recognizes a distance moved by the object during a set time, a state of movement, or a movement of a joint, and sets a type-specific object that has been learned / stored in advance in the computer device. The method comprises the steps of: automatically classifying the object by type or classifying, processing (image removal) and storing the object through the computer device connected to an external database according to the average value of the distance moved during a period of time or the movement pattern, and utilizing the object for learning; and the step of generating comprises: in the case of the object that is fixed in a certain position for a certain period of time or longer, in the case of a temporary installation that is not a moving object or a fixed object among the images located within the shooting range according to the user's selection, classifying the temporary installation separately; and in the case of the temporary installation,A method for generating a 3D map may further include a feature that selectively provides removal or maintenance within an image according to a user's selection, wherein the removing step further includes a step of performing inpainting on an area from which an object with the movement has been removed.

[0128] In another embodiment of the present invention, the performing step may be a method for generating a 3D map, characterized in that the performing step is a step of performing inpainting using background information from the edge toward the inside of an area where the object having the movement has been removed.

[0129] In another embodiment of the present invention, the generating step may be a 3D map generating method characterized in that it includes a step of collecting scan point cloud information from consecutive images on which the inpainting has been performed and a step of generating the 3D map using the scan point cloud information.

[0130] In another embodiment of the present invention, the collecting step may be a three-dimensional map generation method characterized in that the collecting step is a step of collecting the scan point cloud information from the continuous images that only include information corresponding to the space itself, excluding at least one object having the movement.

[0131] In another embodiment of the present invention, a computer device for performing a three-dimensional map generation method, comprising at least one processor configured to execute computer-readable instructions, wherein the at least one processor comprises an acquisition unit for acquiring consecutive images of a target space and a generation unit for generating a three-dimensional map corresponding to the space itself, excluding at least one object having movement in the target space, based on the consecutive images, wherein the generation unit is characterized in that it detects and then removes at least one object having movement from the consecutive images, and the generation unit recognizes the distance moved by the object for a set period of time, the movement state, or the movement of a joint, and automatically classifies, classifies, processes (image removal), and stores the object by type or through the computer device connected to an external database according to an average value of the movement distance for a set period of time for each type that has been learned / stored in advance in the computer device, and utilizes it for learning, wherein the computer device, in the case of the object that has been fixed for a certain period of time or more, separately classifies temporary installations that are not moving objects or fixed objects among the images located within the shooting range according to the user's selection, and in the case of the temporary installations, in the image according to the user's selection, It may be a computer device that provides for optional removal or maintenance.

[0132] In another embodiment of the present invention, the generating unit may be a computer device characterized in that it performs inpainting on an area from which the object having the movement has been removed.

[0133] Although the embodiments described above have been described by way of limited examples and drawings, those skilled in the art will appreciate that various modifications and variations can be made based on the above teachings. For example, appropriate results can still be achieved even if the described techniques are performed in a different order than described, and / or components of the described systems, structures, devices, circuits, etc. are combined or combined in a different manner than described, or are replaced or substituted with other components or equivalents.

[0134] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims described below.

[0135] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will readily appreciate that the present invention can be readily modified into other specific forms without altering the technical spirit or essential characteristics of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single entity may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined manner.

[0136] The scope of the present invention is indicated by the claims set forth below, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention.

[0137] The mode for carrying out the invention has been described together with the best mode for carrying out the invention above.

[0138] This relates to a method and system for generating a 3D map, and to a technology for generating a 3D map that improves the accuracy of a Visual Positioning System (VPS), which has various industrial applications.

Claims

1. A method for generating a three-dimensional map performed by a computer device, A step of acquiring consecutive images of a target space; and A step of generating a three-dimensional map corresponding to the space itself, excluding at least one object moving in the target space, based on the above continuous images. A method for generating a three-dimensional map including:

2. In paragraph 1, The above generating steps are: A step of detecting and removing at least one object having the movement from the above continuous images. A method for generating a three-dimensional map, characterized by including:

3. In paragraph 2, The above removal steps are: A step of detecting at least one object having a change in position or shape in the above continuous images as at least one object having the movement. A method for generating a three-dimensional map, characterized by including:

4. In paragraph 2, The above removal steps are: A step of performing inpainting on the area where the object with the above movement has been removed. A method for generating a three-dimensional map, characterized by further including:

5. In paragraph 4, The steps performed above are: A method for generating a three-dimensional map, characterized in that it is a step of performing inpainting using background information from the edge of an area from which an object having the above movement has been removed toward the inside.

6. In paragraph 4, The above generating steps are: A step of collecting scan point cloud information from consecutive images on which the above inpainting has been performed; and A step of generating the 3D map using the above scan point cloud information. A method for generating a three-dimensional map, characterized by including:

7. In paragraph 6, The above collecting steps are: A method for generating a three-dimensional map, characterized in that it is a step of collecting the scan point cloud information from the continuous images that include only information corresponding to the space itself excluding at least one object having the movement.

8. In paragraph 1, The above obtaining steps are: A step of providing a guide and template for scanning the target space to enable acquisition of the above-described consecutive images using a portable terminal through a service platform on the portable terminal. A method for generating a three-dimensional map, characterized by further including:

9. In paragraph 1, The above generating steps are: A step of generating the three-dimensional map by reflecting user input generated through a service platform provided on a portable terminal A method for generating a three-dimensional map, characterized by including:

10. In paragraph 9, The above generating steps are: A step of including at least one content object on the three-dimensional map in response to a user input generated through a service platform provided on the portable terminal. A method for generating a three-dimensional map, characterized by further including:

11. A computer-readable recording medium having recorded thereon a computer program for executing a method for generating a 3D map on a computer device, The above 3D map creation method is, A step of acquiring consecutive images of a target space; and A step of generating a three-dimensional map corresponding to the space itself, excluding at least one object moving in the target space, based on the above continuous images. A computer-readable recording medium containing: In a computer device performing a method for generating a 12.3-dimensional map, At least one processor configured to execute computer-readable instructions Including, At least one processor of the above, An acquisition unit for acquiring continuous images of a target space; and A generation unit that generates a three-dimensional map corresponding to the space itself, excluding at least one object moving in the target space, based on the above continuous images. A computer device comprising:

13. In paragraph 12, The above generating unit, A computer device characterized by detecting and removing at least one object having the movement from the above consecutive images.

14. In paragraph 13, The above generating unit, A computer device characterized by performing inpainting on an area from which an object having the above movement has been removed.

Images