Computing device for indoor and outdoor wide-area space mapping, and operation method thereof

Abstract

The present disclosure relates to a computing device for indoor and outdoor wide-area space mapping, and an operation method thereof. The method may comprise: acquiring a plurality of datasets, each having connection point information about at least one connection point with respect to a wide-area space; and mapping the connection points of the datasets on the basis of the connection point information, thereby matching the datasets.

Description

Computing device for indoor / outdoor wide-area spatial mapping and its operating method

[0001] The present disclosure relates to a computing device for indoor / outdoor wide-area spatial mapping and a method of operating the same.

[0002] Typically, there is a technology that allows a device to roam around an unknown environment, collecting data about that environment and then providing services using that data. For indoor and outdoor wide-area spaces, multiple datasets are acquired multiple times, and services for that wide-area space are provided by matching these datasets. Here, dataset matching is achieved by matching descriptor information from camera images or LiDAR point clouds. However, errors can occur in dataset matching, as locations with similar but different environmental locations are judged to be the same location. Such errors are more likely to occur in indoor spaces of buildings with multiple floors that are environmentally similar. Meanwhile, if datasets are acquired with GPS signals, services for that wide-area space can be provided without dataset matching. However, for example, GPS signals cannot be received indoors or in areas shaded by buildings or trees.

[0003] The present disclosure provides a computing device and its operating method for matching between or within datasets to provide services for indoor and outdoor wide-area spaces.

[0004] The present disclosure provides a computing device and a method of operating the same for additionally collecting information about matching points and performing matching between or within datasets using the information.

[0005] The method of operating a computing device of the present disclosure may include the steps of acquiring a plurality of datasets, each dataset having information about at least one connection point for a wide area, and the step of matching the datasets by mapping the connection points of the datasets based on the information.

[0006] A computing device of the present disclosure comprises a memory, and a processor connected to the memory and configured to execute at least one instruction stored in the memory, wherein the processor is configured to acquire a plurality of datasets, each dataset having information about at least one connection point with respect to a wide area, and to match the datasets by mapping the connection points of the datasets based on the information.

[0007] According to the present disclosure, since each dataset includes connection point information, a computing device can more accurately align datasets using the connection points of the datasets. Specifically, the computing device can align datasets that each have connection point information for connection points at the same location. This allows, for example, the computing device to more accurately align different datasets or different sub-datasets within a single dataset, each collected at different time periods, i.e., periods separated by arbitrary time intervals. Accordingly, the possibility of errors occurring when the computing device aligns datasets can be reduced. Furthermore, the need for the computing device to acquire GPS information along with the datasets can be eliminated.

[0008] FIG. 1 is a block diagram illustrating a computing device for indoor / outdoor wide-area spatial mapping according to various embodiments.

[0009] FIG. 2a is an example diagram illustrating collection paths for each of the datasets acquired from the computing device of FIG. 1.

[0010] Figure 2b is an example diagram for explaining the alignment of datasets by the computing device of Figure 1.

[0011] Figure 3 is an example diagram for explaining the collection of datasets acquired from the computing device of Figure 1.

[0012] FIG. 4 is a flowchart illustrating an operation method of a computing device for indoor / outdoor wide-area spatial mapping according to various embodiments.

[0013] Figure 5a is an example diagram showing datasets acquired from the computing device of Figure 1.

[0014] Figure 5b is an example diagram showing datasets updated with connection point information from the datasets of Figure 5a.

[0015] Figure 5c is an example diagram showing the alignment of the datasets of Figure 5b.

[0016] FIG. 6 is a flowchart illustrating a step of acquiring a plurality of datasets, each having connection point information, for the wide space of FIG. 4 according to some embodiments.

[0017] FIGS. 7A to 7D are exemplary diagrams for explaining the user interface related to FIG. 6.

[0018] FIG. 8 is a flowchart illustrating a step of acquiring a plurality of datasets each having connection point information for the wide space of FIG. 4 according to other embodiments.

[0019] FIG. 9 is a block diagram illustrating an external device for assisting the computing device of FIG. 1 according to some embodiments.

[0020] FIG. 10 is a flowchart illustrating a method of operation of an external device for assisting the computing device of FIG. 1 according to some embodiments.

[0021] FIGS. 11A to 11E are exemplary diagrams for explaining a user interface related to FIG. 10.

[0022] Hereinafter, specific details for implementing the present disclosure will be described in detail with reference to the attached drawings. However, in the following description, specific descriptions of widely known functions or configurations will be omitted if they may unnecessarily obscure the gist of the present disclosure.

[0023] In the attached drawings, identical or corresponding components are assigned the same reference numerals. Furthermore, in the description of the embodiments below, duplicate descriptions of identical or corresponding components may be omitted. However, even if a description of a component is omitted, it is not intended that such component is not included in any embodiment.

[0024] The advantages and features of the disclosed embodiments, and methods for achieving them, will become clearer with reference to the embodiments described below, along with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided solely to ensure the completeness of the disclosure and to fully inform those skilled in the art of the scope of the invention.

[0025] The terms used in this disclosure will be briefly described, and the disclosed embodiments will be described in detail. The terms used in this disclosure have been selected from widely used and common terms, taking into account the functions of this disclosure. However, these terms may vary depending on the intentions of engineers working in the relevant field, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant invention. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of this disclosure.

[0026] In this disclosure, singular expressions include plural expressions unless the context clearly dictates otherwise. Furthermore, plural expressions include singular expressions unless the context clearly dictates otherwise. Throughout the specification, when a part is said to include a certain component, this does not exclude other components, but rather implies that other components may be included, unless otherwise specifically stated.

[0027] In addition, the term 'module' or 'part' used in the present disclosure means a software or hardware component, and the 'module' or 'part' performs certain roles. However, the 'module' or 'part' is not limited to software or hardware. The 'module' or 'part' may be configured to be on an addressable storage medium and may be configured to play one or more processors. Thus, as an example, the 'module' or 'part' may include at least one of components such as software components, object-oriented software components, class components, and task components, processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, or variables. The components and 'modules' or 'parts' may be combined into a smaller number of components and 'modules' or 'parts', or may be further separated into additional components and 'modules' or 'parts'.

[0028] According to the present disclosure, a 'module' or 'unit' may be implemented as a processor and a memory. 'Processor' should be broadly construed to include a general purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, and the like. In some circumstances, a 'processor' may also refer to an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), and the like. A 'processor' may also refer to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of a plurality of microprocessors, a combination of one or more microprocessors in conjunction with a DSP core, or any other such combination of configurations. In addition, 'memory' should be broadly construed to include any electronic component capable of storing electronic information. 'Memory' may refer to various types of processor-readable media, such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, magnetic or optical data storage, registers, etc. Memory is said to be in electronic communication with the processor if the processor can read information from, and / or write information to, the memory. Memory integrated in a processor is in electronic communication with the processor.

[0029]

[0030] Hereinafter, the present disclosure provides a computing device (100) for indoor / outdoor wide-area spatial mapping and an operating method thereof. The computing device (100) can acquire multiple datasets for a wide-area space and align the datasets. In some embodiments, the computing device (100) may be implemented in a device, such as a device that is portable or wearable by a user and movable by the user, or a robot capable of autonomous navigation. In such a case, the computing device (100) may directly collect the multiple datasets while moving within the wide-area space. In other embodiments, the computing device (100) may be implemented in a server that communicates with the device. In such a case, the device may collect the multiple datasets while moving within the wide-area space, and the computing device (100) may receive the multiple datasets from the device. Each of the datasets may be for at least a portion of the wide-area space. A single dataset may be divided into at least two different sub-datasets. Here, for at least some of the datasets, the corresponding regions may partially overlap. For example, the point at which collection for the first data set ends and the point at which collection for the second data set begins may be the same. Similarly, for at least some of the sub-data sets, corresponding regions may partially overlap.

[0031] FIG. 1 is a block diagram illustrating a computing device (100) for indoor / outdoor wide-area spatial mapping according to various embodiments. FIG. 2a is an exemplary diagram illustrating collection paths for each of the datasets acquired from the computing device (100) of FIG. 1, and FIG. 2b is an exemplary diagram illustrating matching of the datasets by the computing device (100) of FIG. 1. FIG. 3 is an exemplary diagram illustrating collection of the datasets acquired from the computing device (100) of FIG. 1.

[0032] Referring to FIG. 1, the computing device (100) may include at least one of a camera module (110), a sensor module (120), a communication module (130), an input module (140), an output module (150), a memory (160), or a processor (170). In some embodiments, at least one of the components of the computing device (100) may be omitted. In some embodiments, at least one other component may be added to the computing device (100). In some embodiments, at least two of the components of the computing device (100) may be implemented as a single integrated circuit.

[0033] The camera module (110) can capture images of the surrounding environment of the computing device (100). For example, the images can include moving images and still images. According to one embodiment, the camera module (110) can include at least one lens, at least one image sensor, an image signal processor, or a flash.

[0034] The sensor module (120) can detect the state of the surrounding environment of the computing device (100) and generate an electrical signal or data value corresponding thereto. The sensor module (120) may include, for example, at least one of a LiDAR sensor, a radar sensor, an infrared (IR) sensor, or a distance sensor. Additionally, the sensor module (120) may detect the operating state of the computing device (100) and generate an electrical signal or data value corresponding thereto. In this case, the sensor module (120) may further include at least one of a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0035] The communication module (130) can communicate with an external device in the computing device (100). For example, the external device may include at least one of a device, a base station, a server, or a satellite. The communication module (130) may include at least one of a wired communication module and a wireless communication module. The wired communication module may be wired and connected to the external device via a connection terminal (not shown) to communicate with the external device via a wire. The wireless communication module may include at least one of a short-range communication module and a long-range communication module. The short-range communication module may communicate with the external device via a short-range communication method. For example, the short-range communication method may include at least one of Bluetooth, Wi-Fi Direct, or infrared communication. The long-range communication module may communicate with the external device via a long-range communication method. Here, the long-range communication module may communicate with the external device via a network. For example, the network may include at least one of a cellular network, the Internet, or a computer network such as a LAN or WAN. In some embodiments, the communication module (130) may receive global navigation satellite system (GNSS) information. For example, the GNSS information may include global positioning system (GPS) information.

[0036] The input module (140) can input a signal to be used in at least one component of the computing device (100). In some embodiments, the input module (140) can include at least one of a microphone, a mouse, or a keyboard. In other embodiments, the input module (140) can include at least one of touch circuitry configured to detect a touch, or a sensor circuitry configured to measure the intensity of a force generated by a touch.

[0037] The output module (150) can output information from the computing device (100). At this time, the output module (150) can include at least one of a display module for visually displaying information or an audio module for audibly reproducing information. For example, the display module can include at least one of a display, a holographic device, or a projector. As an example, the display module can be implemented as a touch screen by being assembled with at least one of a touch circuit or a sensor circuit of the input module (140). For example, the audio module can include at least one of a speaker, a receiver, an earphone, or a headphone.

[0038] The memory (160) can store various data used by at least one component of the computing device (100). For example, the memory (160) can include at least one of volatile memory and non-volatile memory. The data can include at least one program and input data or output data related thereto. The program can be stored in the memory (160) as software including at least one command.

[0039] The processor (170) can control at least one component of the computing device (100) by executing a program in the memory (160). Through this, the processor (170) can perform data processing or calculations. Here, the processor (170) can execute instructions stored in the memory (160).

[0040] In various embodiments, the processor (170) may acquire multiple datasets for a wide area. Each of the datasets may be for at least a portion of the wide area. In some embodiments, the processor (170) may directly collect the datasets while the computing device (100) moves within the wide area. In this case, the processor (170) may collect the datasets via at least one of the camera module (110), the sensor module (120), or the communication module (130). In other embodiments, a separate device may collect the datasets while moving within the wide area. In this case, the processor (170) may receive the datasets from the separate device via the communication module (130). For example, the computing device (100) or the separate device may collect datasets while moving along collection paths distinguished by different colors within the wide area, as illustrated in FIG. 2A , and thus, the processor (170) may acquire datasets corresponding to each collection path.

[0041] In various embodiments, the processor (170) may update the datasets so that each dataset has information about at least one connection point (also referred to as connection point information). The connection point information may include location information of the connection point within a wide area and time information when the connection point is visited. To this end, the processor (170) may designate a connection point between or within each dataset for each dataset and generate connection point information for the corresponding connection point. Here, the connection point may be designated by user selection or based on preset conditions.

[0042] In some embodiments, the processor (170) can update the dataset by directly specifying a connection point for each dataset. For example, for a dataset acquired for at least a portion of a wide area, the processor (170) can display a pre-stored spatial shape for the corresponding region via the display module. Here, the spatial shape may be a drawing or schematic diagram that briefly represents at least a portion of the wide area. Then, the processor (170) can select at least one connection point within the spatial shape via the input module (140) based on user input and generate connection point information for the selected connection point. Thus, the processor (170) can update the corresponding dataset to include the generated connection point information.

[0043] In other embodiments, the processor (170) can update the dataset by specifying a connection point for each dataset via an external device (external device (900) of FIG. 9). For example, for a dataset acquired for at least a portion of a wide space, the external device (900) can display a pre-stored spatial shape for the corresponding region. Here, the spatial shape can be a drawing or schematic diagram that briefly represents at least a portion of the wide space. Then, the external device (900) can select at least one connection point within the spatial shape by user input, generate location information of the selected connection point, and transmit the location information of the selected connection point to the computing device (100). Accordingly, the processor (170) can receive location information of the selected connection point via the communication module (130) and generate connection point information of the selected connection point using the received connection point information. As a result, the processor (170) can update the corresponding dataset to include the generated connection point information. Here, the external device (900) may include, for example, at least one of a smart phone, a mobile phone, a navigation device, a computer, a laptop, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a tablet PC, a game console, a wearable device, an Internet of Things (IoT) device, a virtual reality (VR) device, an augmented reality (AR) device, or a robot.

[0044] In various embodiments, the processor (170) may align data sets. At this time, the processor (170) may align the data sets by mapping the connection points of the data sets. In some embodiments, the processor (170) may detect connection points of the same location information (within a preset distance), detect data sets each having time information of the connection points, and align these data sets. In other embodiments, the processor (170) may detect connection points of the same location information (within a preset distance), detect at least two different sub-data sets within a single data set each having time information of the connection points, and align these sub-data sets. At this time, multiple sub-spaces within a wide space, for example, a basement level and a ground level, can be clearly distinguished within the aligned data sets. As a result, the processor (170) may obtain a single point cloud for the wide space. For example, the processor (170) can align data sets to obtain a point cloud, as illustrated in FIG. 2B . Accordingly, the processor (170) can provide a service for a wide area based on the aligned data sets, i.e., the point cloud. For example, the service can include at least one of a map creation service or a positioning service (visual localization). Here, since multiple subspaces within the wide area are clearly distinguished, when providing a positioning service, not only location information for the current location but also meta information indicating the subspace to which the current location belongs can be obtained.

[0045] In some embodiments, the data in the datasets may be divided into service data and remaining data upon collection. The service data may relate to at least one internal space for mapping within a wide area, and the remaining data may relate to connection spaces between different internal spaces within the wide area. Here, the connection spaces may include, for example, elevators, escalators, and stairs. Accordingly, the processor (170) may acquire and align such datasets, and provide services for the wide area based on the aligned datasets. Since the service data is already divided upon collection, it does not need to be separately selected for providing services. In other words, the step of separately selecting service data for providing services may be omitted.

[0046] In other embodiments, data from datasets may be hierarchically divided into outdoor and indoor spaces, or into individual layers within a wide area, during collection. Accordingly, the processor (170) can acquire such datasets and hierarchically align them, i.e., from the lowest layer to the highest layer. This can accelerate the computational speed of the dataset alignment process. For example, as illustrated in FIG. 3, point clouds of the third layer (the 1st floor, the 2nd floor, the 5th floor, and the connection space between the 1st floor, the 2nd floor, and the 5th floor) at the lowest level can be generated, point clouds of the second layer (outdoor space, indoor space (the 1st floor, the 2nd floor, the 5th floor, and the connection space between the 1st floor, the 2nd floor, and the 5th floor) at the middle level can be generated, and point clouds of the second layer can be generated to point clouds of the first layer (wide space) at the highest level can be generated. As a result, the processor (170) can provide a service for the wide space based on the aligned datasets.

[0047] FIG. 4 is a flowchart illustrating an operation method of a computing device (100) for indoor / outdoor wide-area spatial mapping according to various embodiments. FIG. 5a is an exemplary diagram illustrating data sets acquired from the computing device (100) of FIG. 1, FIG. 5b is an exemplary diagram illustrating data sets updated with connection point information from the data sets of FIG. 5a, and FIG. 5c is an exemplary diagram illustrating matching of the data sets of FIG. 5b.

[0048] Referring to FIG. 4, the computing device (100) may acquire a plurality of datasets, each having connection point information for a wide area, at step 410. Specifically, the processor (170) may acquire a plurality of datasets for a wide area. Each of the datasets may relate to at least a portion of the wide area. At this time, the processor (170) may update the datasets so that each has connection point information for at least one connection point. The connection point information may include location information of the connection point within the wide area and time information when visiting the connection point. To this end, the processor (170) may designate a connection point between or within each of the datasets, and generate connection point information for the corresponding connection point. For example, the processor (170) may acquire datasets as illustrated in FIG. 5A, and then update the datasets so that each has connection point information for at least one connection point, as illustrated in FIG. 5B. Here, the connection point can be specified by user selection or based on preset conditions. Meanwhile, the connection point can be specified directly on the computing device (100) or through an external device (900), which will be described in more detail below.

[0049] Next, the computing device (100) can align the datasets based on the connection point information at step 420. Specifically, the processor (170) can align the datasets by mapping the connection points of the datasets. In some embodiments, the processor (170) can detect connection points of the same location information (within a preset distance), detect datasets each having time information of the corresponding connection points, and align these datasets. In other embodiments, the processor (170) can detect connection points of the same location information (within a preset distance), detect at least two different sub-datasets within a single dataset each having time information of the corresponding connection points, and align these sub-datasets. As a result, the processor (170) can obtain a single point cloud for a wide area. For example, the processor (170) can align the datasets by mapping the connection points of the datasets, as illustrated in FIG. 5C . At this time, multiple subspaces within the wide space, for example, the basement floor and the ground floor, can be clearly distinguished within the aligned datasets.

[0050] In some embodiments, the data in the datasets may be divided into service data and remaining data upon collection. The service data may relate to at least one internal space for mapping within a wide area, and the remaining data may relate to connection spaces between different internal spaces within the wide area. Here, connection spaces may include, for example, elevators, escalators, and stairs. Accordingly, the processor (170) may acquire and align such datasets. In other embodiments, the data in the datasets may be hierarchically divided into outdoor and indoor spaces, or each floor of an indoor space, upon collection within the wide area. Accordingly, the processor (170) may acquire such datasets and align the datasets hierarchically, i.e., from the lowest layer to the highest layer.

[0051] Finally, the computing device (100) can provide a service for a wide area based on the datasets aligned in step 430. Specifically, the processor (170) can provide a service based on a point cloud for the wide area. For example, the service may include at least one of a map-making service or a positioning service. Here, since multiple subspaces within the wide area are clearly distinguished, when providing a positioning service, not only location information for the current location but also meta information indicating the subspace to which the current location belongs can be obtained.

[0052] FIG. 6 is a flowchart illustrating a step (step 410) of acquiring multiple datasets, each having connection point information, for the wide-area space of FIG. 4 according to some embodiments. FIGS. 7A to 7D are exemplary diagrams illustrating a user interface related to FIG. 6. In these embodiments, connection points may be generated directly on the computing device (100).

[0053] Referring to FIG. 6, the computing device (100) can acquire a dataset for at least a portion of a wide area at step 610. In some embodiments, the processor (170) can directly collect the dataset while the computing device (100) moves within the area. In such a case, the processor (170) can collect the dataset through at least one of the camera module (110), the sensor module (120), or the communication module (130). In another embodiment, a separate device can collect the dataset while moving within the wide area. In such a case, the processor (170) can receive the dataset from the separate device through the communication module (130).

[0054] Next, the computing device (100) can display a spatial shape for the corresponding area in step 620. Specifically, the processor (170) can display the spatial shape through the display module. In one embodiment, the processor (170) can display the spatial shape in response to a user's request while acquiring a dataset. In another embodiment, the processor (170) can display the spatial shape in response to a user's request after acquiring all of the dataset. To this end, the memory (160) may store a spatial shape for at least a portion of the wide-area space. Here, the spatial shape may be a drawing or schematic diagram briefly representing at least a portion of the wide-area space. For example, the spatial shape may be downloaded from a design company of the wide-area space or from the Internet through the communication module (130), or input through the input module (140). For example, the processor (170) may display a screen as illustrated in FIG. 7a, and based on this, may pre-store an identifier (e.g., ID) of the corresponding dataset and the corresponding spatial shape by user input. Thereafter, the processor (170) may display a screen as illustrated in FIG. 7b, detect an identifier of the corresponding dataset input by the user based on this, and then, in response, display the corresponding spatial shape on a screen as illustrated in FIG. 7c.

[0055] Next, the computing device (100) can select at least one connection point within the spatial shape by user input at step 630. Specifically, while displaying the spatial shape, the processor (170) can select a connection point within the spatial shape by user input through the input module (140). For example, as illustrated in FIG. 7c, while displaying the spatial shape, the processor (170) can select a connection point within the spatial shape by user input, and thereby display an indicator corresponding to the selected connection point within the spatial shape, as illustrated in FIG. 7c.

[0056] Additionally, the processor (170) may determine whether there is a previously selected connection point adjacent to the selected connection point. If there is no previously selected connection point, the processor (170) may display an indicator corresponding to the selected connection point within the corresponding spatial shape. On the other hand, if there is a previously selected connection point, the processor (170) may display a notification message as illustrated in FIG. 7D to determine whether to maintain the selected connection point. If the selected connection point is maintained, the processor (170) may add an indicator corresponding to the selected connection point within the corresponding spatial shape. Otherwise, the processor (170) may not add an indicator corresponding to the selected connection point within the corresponding spatial shape.

[0057] Next, the computing device (100) may generate connection point information for the connection point selected in step 640. The connection point information may include location information of the connection point within a wide area and time information when the connection point is visited. Here, the location information may include identification information for the spatial shape and location coordinates of the connection point within the spatial shape. Meanwhile, the time information may be in units of seconds, for example, microseconds. Specifically, the processor (170) may identify the location information of the connection point for the corresponding spatial shape and identify the time point at which the computing device (100) or the device visited the corresponding connection point when collecting the corresponding dataset as time information.

[0058] Finally, the computing device (100) may update the dataset to include the connection point information generated in step 650. Specifically, the processor (170) may update the dataset by adding the connection point information to the dataset. Thus, the dataset may include connection point information as well as data collected by the computing device (100) or device. Thereafter, the computing device (100) may proceed to step 420 of FIG. 4 .

[0059] FIG. 8 is a flowchart illustrating a step (step 410) of acquiring a plurality of datasets, each having connection point information, for the wide-area space of FIG. 4 according to other embodiments. In these embodiments, the connection points may be specified via an external device (900).

[0060] Referring to FIG. 8, the computing device (100) can acquire a dataset for at least a portion of a wide area at step 810. In some embodiments, the processor (170) can directly collect the dataset while the computing device (100) moves within the area. In such a case, the processor (170) can collect the dataset through at least one of the camera module (110), the sensor module (120), or the communication module (130). In another embodiment, a separate device can collect the dataset while moving within the wide area. In such a case, the processor (170) can receive the dataset from the separate device through the communication module (130).

[0061] Next, in step 820, the computing device (100) may receive location information of at least one connection point of the corresponding dataset from the external device (900). The location information may indicate the location of the connection point within a wide space. Here, the location information may include identification information for a spatial shape of at least a portion of the wide space and location coordinates of the connection point within the spatial shape. Specifically, the processor (170) may receive the location information of the connection point from the external device (900) through the communication module (130). In one embodiment, the memory (160) may store a spatial shape for a portion of the wide space corresponding to the corresponding dataset. In another embodiment, the processor (170) may further receive, from the external device (900), the spatial shape for a portion of the wide space corresponding to the corresponding dataset together with the location information.

[0062] Subsequently, the computing device (100) may generate connection point information for the connection point using the location information received in step 830. The connection point information may include location information of the connection point within a wide area and time information when the connection point is visited. Here, the time information may be in units of seconds, for example, microseconds. Specifically, the processor (170) may use the received location information to determine the time at which the computing device (100) or the device visited the connection point when collecting the corresponding dataset, as time information. Thus, the processor (170) may generate connection point information using the received location information and the corresponding time information.

[0063] Finally, the computing device (100) may update the dataset to include the connection point information received in step 830. Specifically, the processor (170) may update the dataset by adding the connection point information to the dataset. Thus, the dataset may include connection point information as well as data collected by the computing device (100) or device. Thereafter, the computing device (100) may proceed to step 420 of FIG. 4 .

[0064] FIG. 9 is a block diagram illustrating an external device (900) for assisting the computing device (100) of FIG. 1 according to some embodiments.

[0065] Referring to FIG. 9, the external device (900) may include at least one of a camera module (910), a sensor module (920), a communication module (930), an input module (940), an output module (950), a memory (960), or a processor (970). In some embodiments, at least one of the components of the external device (900) may be omitted. In some embodiments, at least one other component may be added to the external device (900). In some embodiments, at least two of the components of the external device (900) may be implemented as a single integrated circuit. Here, the external device (900) may include, for example, at least one of a smartphone, a mobile phone, a navigation device, a computer, a laptop, a digital broadcasting terminal, a PDA, a PMP, a tablet PC, a game console, a wearable device, an IoT device, a VR device, an AR device, or a robot.

[0066] The camera module (910) can capture images of the surrounding environment of the external device (900). For example, the images can include moving images and still images. According to one embodiment, the camera module (910) can include at least one lens, at least one image sensor, an image signal processor, or a flash.

[0067] The sensor module (920) can detect the state of the surrounding environment of the external device (900) and generate an electrical signal or data value therefor. The sensor module (920) can include, for example, at least one of a lidar sensor, a radar sensor, an infrared sensor, or a distance sensor. Additionally, the sensor module (920) can detect the operating state of the external device (900) and generate an electrical signal or data value therefor. In this case, the sensor module (920) can further include at least one of a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0068] The communication module (930) can communicate with an external device from the external device (900). For example, the external device may include at least one of a computing device (100), a base station, a server, or a satellite. The communication module (930) may include at least one of a wired communication module and a wireless communication module. The wired communication module may be wiredly connected to the external device through a connection terminal (not shown) and may communicate with the external device through a wire. The wireless communication module may include at least one of a short-range communication module and a long-range communication module. The short-range communication module may communicate with the external device through a short-range communication method. For example, the short-range communication method may include at least one of Bluetooth, Wi-Fi Direct, or infrared communication. The long-range communication module may communicate with the external device through a long-range communication method. Here, the long-range communication module may communicate with the external device through a network. For example, the network may include at least one of a cellular network, the Internet, or a computer network such as a LAN or WAN. In some embodiments, the communication module (930) may receive GNSS information. As an example, the GNSS information may include GPS information.

[0069] The input module (940) can input a signal to be used for at least one component of the external device (900). In some embodiments, the input module (940) can include at least one of a microphone, a mouse, or a keyboard. In other embodiments, the input module (940) can include at least one of a touch circuit configured to detect a touch, or a sensor circuit configured to measure the intensity of a force generated by a touch.

[0070] The output module (950) can output information from an external device (900). At this time, the output module (950) can include at least one of a display module for visually displaying information or an audio module for audibly reproducing information. For example, the display module can include at least one of a display, a holographic device, or a projector. As an example, the display module can be implemented as a touch screen by being assembled with at least one of a touch circuit or a sensor circuit of the input module (940). For example, the audio module can include at least one of a speaker, a receiver, an earphone, or a headphone.

[0071] The memory (960) can store various data used by at least one component of the external device (900). For example, the memory (960) can include at least one of volatile memory and non-volatile memory. The data can include at least one program and input data or output data related thereto. The program can be stored in the memory (960) as software including at least one command.

[0072] The processor (970) can execute a program in the memory (960) to control at least one component of the computing device (900). Through this, the processor (970) can perform data processing or calculations. Here, the processor (970) can execute instructions stored in the memory (960).

[0073] In various embodiments, the processor (970) may designate at least one connection point for an arbitrary dataset. Specifically, for a dataset acquired for at least a portion of a wide area, the processor (970) may display a pre-stored spatial shape for the corresponding region via the display module. Here, the spatial shape may be a drawing or schematic diagram briefly representing at least a portion of the wide area. Then, the processor (970) may select at least one connection point within the spatial shape via the input module (940) based on user input and generate location information for the selected connection point. Accordingly, the processor (970) may transmit the location information to the computing device (100) via the communication module (940).

[0074] FIG. 10 is a flowchart illustrating an operation method of an external device (900) for assisting the computing device (100) of FIG. 1 according to some embodiments. FIGS. 11A to 11E are exemplary diagrams for explaining a user interface related to FIG. 10.

[0075] Referring to FIG. 10, the external device (900) can display a spatial shape for at least a portion of a wide-area space from which an arbitrary dataset was acquired in step 1010. Specifically, the processor (970) can display the spatial shape through the display module. To this end, the memory (960) may store the spatial shape for at least a portion of the wide-area space. Here, the spatial shape may be a drawing or schematic diagram briefly representing at least a portion of the wide-area space. For example, the spatial shape may be downloaded from a design company of the wide-area space or from the Internet through the communication module (930), or input through the input module (940). For example, the processor (970) may display a screen as illustrated in FIG. 11A, and based on this, store an identifier (e.g., ID) of the corresponding dataset and the corresponding spatial shape in advance through a user input. In addition, the processor (970) may further store identification information of the computing device (100) from which the corresponding dataset was acquired. After this, the processor (970) can display a screen as shown in FIG. 11b, detect an identifier of the corresponding dataset input by the user based on this, and then display the corresponding spatial shape in a screen as shown in FIG. 11c in response thereto.

[0076] Next, the external device (900) can select at least one connection point within the spatial shape by user input at step 1020. Specifically, while displaying the spatial shape, the processor (970) can select a connection point within the spatial shape by user input through the input module (940). For example, as illustrated in FIG. 11C, while displaying the spatial shape, the processor (970) can select a connection point within the spatial shape by user input, and thereby display an indicator corresponding to the selected connection point within the spatial shape, as illustrated in FIG. 11C.

[0077] Additionally, the processor (970) may determine whether there is a previously selected connection point adjacent to the selected connection point. If there is no previously selected connection point, the processor (970) may display an indicator corresponding to the selected connection point within the spatial shape. On the other hand, if there is a previously selected connection point, the processor (170) may display a notification message, as illustrated in FIG. 7D , to determine whether to maintain the selected connection point. If the selected connection point is maintained, the processor (170) may add an indicator corresponding to the selected connection point within the spatial shape. Otherwise, the processor (170) may not add an indicator corresponding to the selected connection point within the spatial shape.

[0078] Next, the external device (900) may generate location information for the selected connection point at step 1030. The location information may indicate the location of the connection point within a wide area. Here, the location information may include identification information for a spatial shape for at least a portion of the wide area and location coordinates of the connection point within the spatial shape. Here, the time information may be in units within seconds, for example, microseconds. Specifically, the processor (970) may generate location information for the selected connection point based on the spatial shape.

[0079] Finally, the external device (900) can transmit location information to the computing device (100) at step 1040. Specifically, the processor (970) can transmit the location information to the computing device (100) through the communication module (940). The processor (970) can display a screen as illustrated in FIG. 11D and transmit connection point information to the computing device (100) based on the screen. Here, the processor (970) can connect to the computing device (100) through the communication module (940) and then transmit the location information to the computing device (100). In one embodiment, the processor (970) can further transmit the corresponding spatial shape together with the location information.

[0080] According to the present disclosure, since each of the datasets includes connection point information, the computing device (100) can more accurately align the datasets using the connection points of the datasets. Specifically, the computing device (100) can align datasets that each have connection point information for a connection point at the same location. Thus, for example, the computing device (100) can more accurately align datasets collected in different time periods, i.e., periods separated by an arbitrary time interval. At this time, multiple subspaces within a wide space, such as the first basement floor and the first floor above ground, can be clearly distinguished within the aligned datasets. Accordingly, the possibility of errors occurring when the computing device (100) aligns the datasets can be reduced. Furthermore, the need for the computing device (100) to acquire GPS information along with the datasets can also be eliminated.

[0081]

[0082] In summary, the present disclosure provides a computing device (100) for indoor / outdoor wide-area spatial mapping and a method of operating the same.

[0083] The method of operating the computing device (100) of the present disclosure may include a step of acquiring a plurality of datasets, each dataset having connection point information for at least one connection point in a wide area (step 410), and a step of matching the datasets by mapping the connection points of the datasets based on the connection point information (step 420).

[0084] In the present disclosure, datasets are collected while moving within a wide space, and connection point information may include location information of a connection point within the wide space and time information when visiting a connection point.

[0085] In the present disclosure, the step of matching datasets (step 420) may include a step of matching at least two different datasets by mapping connection points of the same location information.

[0086] In the present disclosure, the step of aligning datasets (step 420) may include a step of aligning at least two different sub-datasets within one dataset by mapping connection points of the same location information.

[0087] In the present disclosure, connection points can be specified by user selection or based on preset conditions.

[0088] In the present disclosure, the step of acquiring datasets (step 410) may include the step of acquiring datasets for at least a portion of a wide space (step 610), the step of displaying a pre-stored spatial shape for at least a portion of a wide space (step 620), the step of selecting at least one connection point within the spatial shape by user input (step 630), the step of generating information for the selected connection point (step 640), and the step of updating the dataset to include the generated information (step 650).

[0089] In the present disclosure, the step of acquiring datasets (step 410) may include the steps of acquiring datasets for at least a portion of a wide area (step 810), receiving location information of a connection point from an external device (900) (step 820), and updating the dataset to include the received location information (steps 830 and 840).

[0090] In the present disclosure, an external device (900) can display a spatial shape (step 1010), select at least one connection point within the spatial shape by user input (step 1020), generate location information of the selected connection point (step 1030), and transmit the generated location information (step 1040).

[0091] In the present disclosure, the operating method of the computing device (100) may further include a step (step 430) of providing a service for a wide area based on aligned datasets. For example, the service may include at least one of a map-making service or a positioning service.

[0092] In the present disclosure, the data of the datasets can be divided into service data for at least one internal space for creating a map within a wide space at the time of collection and the remaining data for the connection space between different internal spaces within the wide space.

[0093] In the present disclosure, data of datasets can be hierarchically divided into outdoor space and indoor space, or each floor of the indoor space, for a wide area at the time of collection.

[0094] A computing device (100) of the present disclosure includes a memory (160), and a processor (170) connected to the memory (160) and configured to execute at least one instruction stored in the memory (160), wherein the processor (170) may be configured to acquire a plurality of data sets each having information about at least one connection point with respect to a wide area, and to match the data sets by mapping the connection points of the data sets based on the information.

[0095] In the present disclosure, datasets are collected while moving within a wide space, and the information may include location information of connection points within the wide space and time information when visiting connection points.

[0096] In the present disclosure, the processor (170) may be configured to align at least two different datasets by mapping connection points of the same location information.

[0097] In the present disclosure, the processor (170) may be configured to align at least two different sub-datasets within one dataset by mapping connection points of the same location information.

[0098] In the present disclosure, the computing device (100) further includes a display module and an input module (140) connected to a processor (170), and the processor (170) may be configured to acquire a dataset for at least a portion of a wide space, display a pre-stored spatial shape for at least a portion of the wide space through the display module, select at least one connection point within the spatial shape by a user input through the input module (140), generate information for the selected connection point, and update the dataset to include the generated information.

[0099] In the present disclosure, the computing device (100) further includes a communication module (130) connected to a processor (170), and the processor (170) is configured to acquire a dataset for at least a portion of a wide area of ​​space, receive information about a connection point from an external device (900) through the communication module, and update the dataset to include the received information, and the external device (900) may be configured to display a spatial shape, select at least one connection point within the spatial shape by a user input, generate information about the selected connection point, and transmit the generated information.

[0100] In the present disclosure, the processor (170) may be configured to provide a service for a wide area based on aligned datasets. For example, the service may include at least one of a map-making service or a positioning service.

[0101] In the present disclosure, the data of the datasets can be divided into service data for at least one internal space for creating a map within a wide space at the time of collection and the remaining data for the connection space between different internal spaces within the wide space.

[0102] In the present disclosure, data of datasets can be hierarchically divided into outdoor space and indoor space, or each floor of the indoor space, for a wide area at the time of collection.

[0103] The above-described method may be provided as a computer program stored on a computer-readable recording medium for execution on a computer. The medium may be one that continuously stores a computer-executable program, or one that temporarily stores it for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or multiple hardware combinations, and is not limited to a medium directly connected to a computer system, but may also be distributed over a network. Examples of the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and those configured to store program instructions, including ROM, RAM, and flash memory. In addition, examples of other media may include recording or storage media managed by app stores that distribute applications, sites that supply or distribute various software, servers, etc.

[0104] The methods, operations, or techniques of the present disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those skilled in the art will appreciate that the various exemplary logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various exemplary components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software will depend on the particular application and the design requirements imposed on the overall system. Those skilled in the art may implement the described functionality in various ways for each particular application, but such implementations should not be construed as departing from the scope of the present disclosure.

[0105] In a hardware implementation, the processing units used to perform the techniques may be implemented within one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described herein, a computer, or a combination thereof.

[0106] Accordingly, the various exemplary logical blocks, modules, and circuits described in connection with the present disclosure may be implemented or performed by any combination of a general-purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or those designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0107] In a firmware and / or software implementation, the techniques may be implemented as instructions stored on a computer-readable medium, such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, a compact disc (CD), a magnetic or optical data storage device, etc. The instructions may be executable by one or more processors and may cause the processor(s) to perform certain aspects of the functionality described herein.

[0108] While the embodiments described above have been described as utilizing aspects of the presently disclosed subject matter in one or more standalone computer systems, the present disclosure is not limited thereto and may be implemented in conjunction with any computing environment, such as a network or distributed computing environment. Furthermore, aspects of the present disclosure may be implemented in multiple processing chips or devices, and storage may be similarly affected across multiple devices. Such devices may include personal computers, network servers, and portable devices.

[0109] While this disclosure has been described with reference to certain embodiments, various modifications and variations can be made without departing from the scope of the present disclosure, which would be apparent to those skilled in the art. Furthermore, such modifications and variations should be considered to fall within the scope of the claims appended hereto.

Claims

1. In the method of operating a computing device, A step of acquiring a plurality of datasets, each dataset having information about at least one connection point for a wide area; and A step of aligning the datasets by mapping the connection points of the datasets based on the above information. Including, A method of operating a computing device.

2. In paragraph 1, The above datasets are collected while moving within the above wide space, The above information includes location information of the connection point within the wide space and time information when visiting the connection point. A method of operating a computing device.

3. In paragraph 2, The steps to align the above datasets are: A step of aligning at least two different datasets by mapping the connection points of the same location information. Including, A method of operating a computing device.

4. In paragraph 2, The steps to align the above datasets are: A step of aligning at least two different sub-datasets within a single dataset by mapping connection points of the same location information. Including, A method of operating a computing device.

5. In paragraph 1, The above connection points are either specified by user selection or based on preset conditions. A method of operating a computing device.

6. In paragraph 1, The steps for acquiring the above datasets are: A step of acquiring a dataset for at least a portion of the above wide area; A step of displaying a pre-stored spatial shape for at least a portion of the above wide space; A step of selecting at least one connection point within the spatial shape by user input; a step of generating information about the above selected connection point; and A step of updating the dataset to include the generated information. Including, A method of operating a computing device.

7. In paragraph 1, The steps for acquiring the above datasets are: A step of acquiring a dataset for at least a portion of the above wide area; A step of receiving information about a connection point from an external device; and A step of updating the dataset to include the received information. Including, The above external device is, Indicate the above spatial shape, Selecting at least one connection point within the spatial shape by user input, Generate information about the selected connection point above, Transmitting the information generated above, A method of operating a computing device.

8. In paragraph 1, A step for providing a service for the wide area based on the above-mentioned aligned datasets. Including more, A method of operating a computing device.

9. In paragraph 2, The data of the above datasets are divided into service data for at least one internal space for making a map within the wide space when collected, and the remaining data for the connection space between different internal spaces within the wide space. A method of operating a computing device.

10. In paragraph 2, The data of the above datasets are hierarchically divided into outdoor space, indoor space, or each floor of the indoor space when collected for the wide area. A method of operating a computing device.

11. A non-transitory computer-readable recording medium storing a computer program for executing the method of claim 1 on the computing device.

12. In computing devices, memory; and A processor coupled to said memory and configured to execute at least one instruction stored in said memory, The above processor, Acquire multiple datasets, each having information about at least one connection point for a wide area, and By mapping the connection points of the above datasets based on the above information, the above datasets are configured to be aligned. Computing device.

13. In paragraph 12, The above datasets are collected while moving within the above wide space, The above information includes location information of the connection point within the wide space and time information when visiting the connection point. Computing device.

14. In paragraph 13, The above processor, By mapping the connection points of the same location information, it is configured to align at least two different datasets. Computing device.

15. In paragraph 13, The above processor, By mapping the connection points of the same location information, it is configured to align at least two different sub-datasets within one dataset. Computing device.

16. In paragraph 12, Display module and input module connected to the above processor Including more, The above processor, Acquire a dataset for at least a portion of the above wide area, Through the above display module, a pre-stored spatial shape is displayed for at least a portion of the above wide space, Through the above input module, at least one connection point is selected within the spatial shape by user input, Generate information about the selected connection point above, configured to update said dataset to include the generated information; Computing device.

17. In paragraph 12, Communication module connected to the above processor Including more, The above processor, Acquire a dataset for at least a portion of the above wide area, Through the above communication module, information about the connection point is received from an external device, configured to update said dataset to include said received information; The above external device is, Indicate the above spatial shape, Selecting at least one connection point within the spatial shape by user input, Generate information about the selected connection point above, configured to transmit the above generated information, Computing device.

18. In paragraph 12, The above processor, It is configured to provide a service for the wide area based on the above aligned datasets. Computing device.

19. In paragraph 13, The data of the above datasets are divided into service data for at least one internal space for making a map within the above wide space when collected, and the remaining data for the connection space between different internal spaces within the above wide space. Computing device.

20. In paragraph 13, The data of the above datasets are hierarchically divided into outdoor space, indoor space, or each floor of the indoor space when collected for the wide area. Computing device.

Images