Vision-based localization method and apparatus thereof

Abstract

The present invention relates to a vision-based localization method performed by an operation server, comprising the steps of: transmitting map information about a target space to a worker terminal; receiving, from the worker terminal, image information captured at a specific location in the target space and additional information, related to the specific location, of the worker; and recognizing, on the basis of the image information and the additional information received from the worker terminal, the location of an image captured at the specific location.

Description

Vision-based location recognition method and device thereof

[0001] The present invention relates to vision-based location recognition technology, and more specifically, to a method and device for recognizing the location of an image captured by a camera within a target space in real time.

[0002] Vision-based positioning technology uses images captured by cameras to determine the location and angle of the camera from which the images were captured. This vision-based positioning technology is essential for spatial mapping, spatial reconstruction, autonomous robots, and augmented reality navigation.

[0003] Vision-based location recognition technologies typically utilize methods such as Structure-from-Motion (SFM) or Simultaneous Localization and Mapping (SLAM). These methods, when presented with a currently captured image, search for previously captured and processed images with similar locations and angles. They then estimate the current image's location by matching local features with these previously captured images.

[0004] However, conventional methods suffer from the problem of poor position recognition performance within a target space when the target space is repetitive or lacks features. Therefore, a method is needed to improve position recognition performance in repetitive or feature-poor spaces.

[0005] The present invention aims to solve the aforementioned problems and other issues. Another objective is to provide a method and device for recognizing the location of an image captured by a current camera within a target space in real time based on additional information input from a worker.

[0006] Another purpose is to provide a method and device for recognizing in real time the location of an image captured by a current camera within a target space based on adjacent node information input from a worker.

[0007] Another purpose is to provide a method and device for recognizing in real time the position of an image captured by a current camera within a target space based on camera position information input from a worker.

[0008] According to one aspect of the present invention to achieve the above or other purposes, a vision-based location recognition method performed by an operation server is provided, comprising: a step of transmitting map information about a target space to a worker terminal; a step of receiving image information captured at a specific location of the target space and additional information of a worker related to the specific location from the worker terminal; and a step of recognizing a location of an image captured at the specific location based on the image information and additional information received from the worker terminal.

[0009] According to another aspect of the present invention, a computing server including one or more processors and performing vision-based location recognition is provided, wherein the one or more processors perform: transmitting map information about a target space to a worker terminal; receiving image information captured at a specific location of the target space and additional information of a worker related to the specific location from the worker terminal; and recognizing a location of an image captured at the specific location based on the image information and additional information received from the worker terminal.

[0010] According to another aspect of the present invention, a vision-based location recognition method performed by a worker terminal is provided, comprising: a step of displaying map information for a target space on a display unit; a step of obtaining image information photographed at a specific location of the target space; a step of receiving additional information related to the specific location from a worker; and a step of recognizing a location of an image photographed at the specific location based on the obtained image information and the input additional information.

[0011] According to another aspect of the present invention, a worker terminal including one or more processors and performing vision-based location recognition is provided, wherein the one or more processors perform: displaying map information about a target space on a display unit; acquiring image information captured at a specific location in the target space; receiving additional information related to the specific location from a worker; and recognizing a location of an image captured at the specific location based on the acquired image information and the input additional information.

[0012] Additionally, the solutions to the aforementioned problems do not enumerate all features of the present invention. The various features of the present invention, along with their corresponding advantages and effects, can be understood in more detail by referring to the specific embodiments below.

[0013] The effects of a vision-based location recognition method and device according to embodiments of the present invention are described as follows.

[0014] According to at least one of the embodiments of the present invention, by performing real-time location recognition based on current image information captured by a camera and additional information input from a worker, the possibility of mismatching with other node images within the target space can be reduced, thereby improving the success rate and accuracy of location recognition for the currently captured image.

[0015] In addition, according to at least one of the embodiments of the present invention, by performing real-time location recognition based on current image information captured by a camera and adjacent node information input from a worker, there is an advantage in that the possibility of mismatching with other node images within the target space can be reduced, thereby improving the location recognition success rate and accuracy for the currently captured image.

[0016] In addition, according to at least one of the embodiments of the present invention, by performing real-time location recognition based on current image information captured by a camera and camera location information input from a worker, there is an advantage in that the possibility of mismatching with other node images within the target space can be reduced, thereby improving the success rate and accuracy of location recognition for the currently captured image.

[0017] However, the effects that can be achieved by the vision-based location recognition method and device according to embodiments of the present invention are not limited to those mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present invention pertains from the description below.

[0018] FIG. 1 is a diagram showing the configuration of a vision-based location recognition system according to a first embodiment of the present invention;

[0019] FIG. 2 is a diagram showing the configuration of a vision-based location recognition system according to a second embodiment of the present invention;

[0020] Figure 3 is a drawing illustrating a target space and map information of the target space;

[0021] FIG. 4 is a flowchart illustrating a vision-based location recognition method according to a first embodiment of the present invention;

[0022] FIG. 5 and FIG. 6 are drawings referenced to explain the position recognition method of FIG. 4;

[0023] FIG. 7 is a flowchart illustrating a vision-based location recognition method according to a second embodiment of the present invention;

[0024] Figures 8 and 9 are drawings referenced to explain the location recognition method of Figure 7.

[0025] FIG. 10 is a diagram showing the configuration of a vision-based location recognition system according to a third embodiment of the present invention;

[0026] FIG. 11 is a flowchart illustrating a vision-based location recognition method according to a third embodiment of the present invention;

[0027] FIG. 12 is a flowchart illustrating a vision-based location recognition method according to a fourth embodiment of the present invention;

[0028] Figure 13 is a block diagram of a computing device according to an embodiment of the present invention.

[0029] Hereinafter, embodiments disclosed in the present specification will be described in detail with reference to the attached drawings. Regardless of the drawing numbers, identical or similar components will be given the same reference numbers, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are given or used interchangeably only for the convenience of writing the specification, and do not have distinct meanings or roles in themselves. That is, the term "part" used in the present invention means a hardware component such as software, FPGA, or ASIC, and the "part" performs certain roles. However, the "part" is not limited to software or hardware. The "part" may be configured to be on an addressable storage medium, or may be configured to reproduce one or more processors. Thus, as an example, a 'part' may include components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. The functionality provided within the components and 'parts' may be combined into a smaller number of components and 'parts' or further separated into additional components and 'parts'.

[0030] In addition, when describing the embodiments disclosed in this specification, if it is determined that a detailed description of a related known technology may obscure the gist of the embodiments disclosed in this specification, the detailed description thereof will be omitted. In addition, the attached drawings are only intended to facilitate easy understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the attached drawings, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and technical scope of the present invention.

[0031] The present invention proposes a method and device for recognizing in real time the location of an image captured by a current camera within a target space based on additional information input from a worker. Furthermore, the present invention proposes a method and device for recognizing in real time the location of an image captured by a current camera within a target space based on adjacent node information input from a worker. Furthermore, the present invention proposes a method and device for recognizing in real time the location of an image captured by a current camera within a target space based on camera position information input from a worker.

[0032] The target space described herein includes all spaces subject to location recognition and mapping. For convenience of explanation, the following examples focus on indoor spaces. However, this is not necessarily limited to such spaces, and it will be apparent to those skilled in the art that outdoor spaces may also be used.

[0033] Hereinafter, various embodiments of the present invention will be described in detail with reference to the drawings.

[0034]

[0035] FIG. 1 is a diagram showing the configuration of a vision-based location recognition system according to a first embodiment of the present invention.

[0036] Referring to FIG. 1, a vision-based location recognition system (100) according to a first embodiment of the present invention may include a worker terminal (or scan terminal, 110), a computation server (or processing server, 120), a communication network (130), and a camera (140).

[0037] The worker terminal (110) and the operation server (120) can be connected through a communication network (130).

[0038] The communication network (130) may include wired networks and wireless networks, and specifically, may include various networks such as a local area network (LAN), a metropolitan area network (MAN), and a wide area network (WAN). In addition, the communication network (130) may include the well-known World Wide Web (WWW). However, the communication network (130) according to the present invention is not limited to the networks listed above, and may include at least one of a well-known wireless data network, a well-known telephone network, and a well-known wired / wireless television network.

[0039] The worker terminal (110) can request a separate camera (140) to take a video of the target space. The camera (140) can take a video of the target space according to the video request signal of the worker terminal (110) and then transmit the video to the worker terminal (110). At this time, the worker terminal (110) and the camera (140) can transmit and receive data through short-range wireless communication. At least one of Bluetooth, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies can be used as the short-range wireless communication.

[0040] The worker terminal (110) can transmit images acquired from the camera (140) to the operation server (120). The operation server (120) can create a map of the target space in real time based on the images received from the worker terminal (110).

[0041] The worker terminal (110) can receive map information constructed for the target space from the operation server (120). The worker terminal (110) can display the received map information on the display unit. This allows the worker to check the location recognition results and map creation results for images captured by the camera in real time.

[0042] For example, as illustrated in FIG. 3, if the target space (10) is an indoor space of an apartment, the worker terminal (140) can transmit an image captured at the location of node 17 to the operation server (120). The operation server (120) can create a map of the target space (10) in real time based on the image received from the worker terminal (110). The operation server (120) can transmit map information (20) constructed for the target space (10) to the worker terminal (110). The worker terminal (110) can display the received map information (20) on the display unit. At this time, the map information (20) can include a plurality of node information. Each node information corresponds to the order and location at which the image was captured.

[0043] When capturing an image using a camera (140), the worker can input additional information that is helpful for vision-based location recognition into the worker terminal (110). The worker terminal (110) can provide the additional information input by the worker to the operation server (120). In this case, the additional information can include at least one of the node information closest to the current shooting point among the plurality of node information included in the map information and the approximate camera location information on the map information.

[0044] The user terminal (110) described in this specification may include a mobile phone, a smart phone, a laptop computer, a desktop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a slate PC, a tablet PC, an ultrabook, a wearable device, and the like.

[0045] The operation server (120) can recognize the location of the image based on the image received from the worker terminal (110). At this time, the operation server (120) can estimate the location of the image using either the Structure-from-Motion (SFM) method or the Simultaneous Localization And Mapping (SLAM) method.

[0046] The operation server (120) can more accurately recognize the location of the image based on not only the image received from the worker terminal (110) but also separate additional information. At this time, the operation server (120) can detect a node adjacent to the current shooting point based on the additional information received from the worker terminal (110) and perform local feature matching with the image of the detected adjacent node to more accurately estimate the location of the image.

[0047] The operation server (120) can create a map of the target space based on the location information of the image received from the worker terminal (110). The operation server (120) can provide the map information created for the target space to the worker terminal (110).

[0048] The camera (140) can capture at least one of a 360-degree image, a panoramic image, and a general image of the target space. At this time, the camera (140) can capture the corresponding image in response to a shooting request signal from the worker terminal (110).

[0049] The camera (140) can transmit the captured image to the worker terminal (110). At this time, the camera (140) can transmit the image to the worker terminal (110) via short-range wireless communication.

[0050] As described above, the vision-based location recognition system according to the first embodiment of the present invention performs real-time location recognition based on current image information captured by a camera and additional information input from a worker, thereby reducing the possibility of mismatching with other node images within the target space and improving the location recognition success rate and accuracy for the currently captured image.

[0051]

[0052] FIG. 2 is a diagram showing the configuration of a vision-based location recognition system according to a second embodiment of the present invention.

[0053] Referring to FIG. 2, a vision-based location recognition system (200) according to a second embodiment of the present invention may include a worker terminal (210), an operation server (220), and a communication network (230).

[0054] Unlike the position recognition system (100) according to the first embodiment described above, the vision-based position recognition system (200) according to the present embodiment may omit a separate camera (140). In the case of the present embodiment, an image of a target space may be captured using a camera installed on a worker terminal (210). That is, the worker terminal (210) may capture an image of a target space using its own camera and provide the captured image to a computation server (220).

[0055] Meanwhile, the operation of the worker terminal (210), the operation server (220), and the communication network (230) excluding the shooting function is the same or similar to the operation of the worker terminal (110), the operation server (120), and the communication network (130) of the location recognition system (100) according to the first embodiment described above, so a detailed description thereof will be omitted.

[0056]

[0057] FIG. 4 is a flowchart illustrating a vision-based location recognition method according to a first embodiment of the present invention. The vision-based location recognition method according to this embodiment can be performed by the location recognition system (100, 200) according to the first and second embodiments. In the illustrated flowchart, the vision-based location recognition method is described by dividing it into a plurality of steps, but at least some of the steps may be performed in a reversed order, combined with other steps and performed together, omitted, divided into sub-steps and performed, or one or more steps not illustrated may be added and performed.

[0058] Referring to FIG. 4, the operation server (120, 220) can create a map of the target space in real time based on images received from the worker terminal (110, 210) (S410).

[0059] The operation server (120, 220) can transmit map information constructed for the target space to the worker terminal (110, 210) (S420).

[0060] The worker terminal (110, 210) can display map information received from the operation server (120, 220) on the display unit (S430).

[0061] For example, as illustrated in FIG. 5, the worker terminal (110, 210) can display map information (500) including seven node information on the display unit. Here, each node information corresponds to the order and location of images captured by the camera.

[0062] The worker can check the location recognition results and map creation results for previous images captured by the camera in real time through the map information displayed on the display unit.

[0063] Meanwhile, although not shown in the drawing, if the worker confirms that there are nodes whose locations are incorrectly estimated based on the map information, the worker can input a command to the worker terminal (110, 210) requesting the deletion of the nodes. This is to reduce in advance any errors in location recognition that may accumulate later due to incorrectly estimated node information.

[0064] The worker terminal (110, 210) can display a message requesting input of adjacent node information along with map information on the display unit.

[0065] When capturing video with a camera at a new location in a target space, the worker can input information about the node closest to the current shooting location into the worker terminal (110, 210) (S440). At this time, the worker can input the adjacent node information by touching the node displayed on the display unit or by calling up a separate node input menu.

[0066] For example, as shown in FIG. 5, if the node closest to the current shooting point is node 5 (510), the worker can input information about node 5 (510) into the worker terminal (110, 210).

[0067] When an image is captured at a new location in the target space (S450), the worker terminal (110, 210) can transmit the captured image information and adjacent node information to the operation server (120, 220) (S460).

[0068] Meanwhile, in this embodiment, it is exemplified that image information and adjacent node information are transmitted together from the worker terminal (110, 210), but it is not necessarily limited to this, and the information may be transmitted separately.

[0069] In addition, although the present embodiment exemplifies inputting adjacent node information before capturing an image at a new location in the target space, it is not necessarily limited thereto, and it will be apparent to those skilled in the art that adjacent node information may be input after capturing the image, or may be input simultaneously with capturing the image.

[0070] The operation server (120, 220) can recognize the location information of the currently captured image based on the image information and adjacent node information received from the worker terminal (110, 210) (S470). At this time, the operation server (120, 220) detects the image of the adjacent node input by the worker and performs local feature matching with the detected node image to more accurately recognize the location of the currently captured image. Through this, the operation server (120, 220) can reduce the possibility of mismatching with other node images within the target space, thereby improving the location recognition success rate and accuracy.

[0071] The operation server (120, 220) can update a map of the target space based on the location information of the currently captured image (S480).

[0072] The operation server (120, 220) can transmit updated map information to the worker terminal (110, 210) (S490).

[0073] The worker terminal (110, 210) can display the received map information on the display unit. For example, as illustrated in FIG. 6, the worker terminal (110, 210) can display map information (500) including eight node information on the display unit. Here, node number 8 (520) corresponds to the location of the currently captured image.

[0074] As described above, the vision-based location recognition method according to the first embodiment of the present invention performs real-time location recognition based on current image information captured by a camera and adjacent node information input from a worker, thereby reducing the possibility of mismatching with other node images within the target space and improving the location recognition success rate and accuracy for the currently captured image.

[0075]

[0076] FIG. 7 is a flowchart illustrating a vision-based location recognition method according to a second embodiment of the present invention. The vision-based location recognition method according to this embodiment can be performed by the location recognition system (100, 200) according to the first and second embodiments. In the illustrated flowchart, the vision-based location recognition method is described by dividing it into a plurality of steps, but at least some of the steps may be performed in a reversed order, combined with other steps and performed together, omitted, divided into substeps and performed, or one or more steps not illustrated may be added and performed.

[0077] Referring to FIG. 7, the operation server (120, 220) can create a map of the target space in real time based on images received from the worker terminal (110, 210) (S710).

[0078] The operation server (120, 220) can transmit map information constructed for the target space to the worker terminal (110, 210) (S720).

[0079] The worker terminal (110, 210) can display map information received from the operation server (120, 220) on the display unit (S730).

[0080] For example, as illustrated in FIG. 8, the worker terminal (110, 210) can display map information (800) including seven node information on the display unit. Here, each node information corresponds to the order and location of images captured by the camera.

[0081] The worker can check the location recognition results and map creation results for previous images captured by the camera in real time through the map information displayed on the display unit.

[0082] The worker terminal (110, 210) can display a message on the display unit requesting the worker to input the approximate location information of the current camera along with map information.

[0083] When capturing video with a camera at a new location in a target space, the worker can input the approximate location information of the current camera into the worker terminal (110, 210) (S740). At this time, the worker can input the camera location information by touching a specific point on the map area displayed on the display unit, or by calling up a separate location input menu.

[0084] For example, as illustrated in FIG. 8, a worker can input camera location information into a worker terminal (110, 210) by touching the point where the current camera is located on the map area with a finger (810).

[0085] If an image is captured at a new location in the target space (S750), the worker terminal (110, 210) can transmit the captured image information and camera location information to the operation server (120, 220) (S760).

[0086] Meanwhile, in this embodiment, it is exemplified that image information and camera position information are transmitted together from the worker terminal (110, 210), but it is not necessarily limited to this, and the corresponding information may be transmitted separately.

[0087] In addition, although the present embodiment exemplifies inputting camera position information before shooting an image at a new location in the target space, it is not necessarily limited thereto, and it will be apparent to those skilled in the art that the camera position information may be input after shooting the image, or may be input simultaneously with shooting the image.

[0088] The operation server (120, 220) can recognize the location information of the currently captured image based on the image information and camera location information received from the worker terminal (110, 210) (S770). At this time, the operation server (120, 220) detects the node closest to the current shooting point based on the camera location information input by the worker, and performs local feature matching with the image of the detected adjacent node to more accurately recognize the location of the currently captured image. Through this, the operation server (120, 220) can improve the location recognition success rate and accuracy by reducing the possibility of mismatching with images of other nodes within the target space.

[0089] The operation server (120, 220) can update a map of the target space based on the location information of the currently captured image (S780).

[0090] The operation server (120, 220) can transmit updated map information to the worker terminal (110, 210) (S790).

[0091] The worker terminal (110, 210) can display the received map information on the display unit. For example, as illustrated in FIG. 9, the worker terminal (110, 210) can display map information (800) including eight node information on the display unit. Here, node number 8 (820) corresponds to the location of the currently captured image.

[0092] As described above, the vision-based location recognition method according to the second embodiment of the present invention performs real-time location recognition based on current image information captured by a camera and camera location information input from a worker, thereby reducing the possibility of mismatching with other node images within the target space and improving the location recognition success rate and accuracy for the currently captured image.

[0093]

[0094] FIG. 10 is a diagram showing the configuration of a vision-based location recognition system according to a third embodiment of the present invention.

[0095] Referring to FIG. 10, a vision-based location recognition system (300) according to a third embodiment of the present invention may include a worker terminal (310) and a camera (320).

[0096] The worker terminal (310) can request a separate camera (320) to capture an image of a target space. The camera (320) can capture an image of the target space in response to a capture request signal from the worker terminal (310) and then transmit the image to the worker terminal (310). At this time, the worker terminal (310) and the camera (320) can transmit and receive data via short-range wireless communication.

[0097] The worker terminal (310) can recognize the location of the image based on the image received from the camera (320). At this time, the worker terminal (310) can estimate the location of the image using either the Structure-from-Motion (SFM) method or the Simultaneous Localization And Mapping (SLAM) method.

[0098] The worker terminal (310) can create a map of the target space in real time based on the location information of the image received from the camera (320).

[0099] The worker terminal (310) can display the created map information on the display unit. This allows the worker to check the location recognition results and map creation results for images captured by the camera in real time.

[0100] When capturing video using a camera (320), the worker may input additional information that is helpful for vision-based location recognition into the worker terminal (310). At this time, the additional information may include at least one of the node information closest to the current shooting point among the plurality of node information included in the map information and the approximate camera location information on the map information.

[0101] The worker terminal (310) can more accurately recognize the location of the image based on not only the image received from the camera (320) but also separate additional information. At this time, the worker terminal (310) can detect a node adjacent to the current shooting point based on the additional information received from the worker, and perform local feature matching with the image of the detected adjacent node to more accurately estimate the location of the image.

[0102] The camera (320) can capture at least one of a 360-degree image, a panoramic image, and a general image of the target space. At this time, the camera (320) can capture the corresponding image in response to a shooting request signal from the worker terminal (110).

[0103] The camera (320) can transmit the captured image to the worker terminal (310). At this time, the camera (320) can transmit the image to the worker terminal (310) via short-range wireless communication.

[0104] As described above, the vision-based location recognition system according to the third embodiment of the present invention performs real-time location recognition based on current image information captured by a camera and additional information input from a worker, thereby reducing the possibility of mismatching with other node images and improving the location recognition success rate and accuracy for the currently captured image.

[0105] Meanwhile, although not illustrated in the drawing, the vision-based position recognition system according to the fourth embodiment of the present invention, unlike the position recognition system (300) according to the third embodiment described above, may omit a separate camera (320). In this embodiment, an image of a target space may be captured using a camera installed in the worker terminal (310). In other words, the worker terminal (310) may perform a vision-based position recognition method without interfacing with other devices.

[0106]

[0107] Fig. 11 is a flowchart illustrating a vision-based location recognition method according to a third embodiment of the present invention. The vision-based location recognition method according to this embodiment can be performed by the location recognition system (300) according to the third and fourth embodiments. In the illustrated flowchart, the vision-based location recognition method is described by dividing it into a plurality of steps, but at least some of the steps may be performed in a reversed order, combined with other steps and performed together, omitted, divided into substeps and performed, or one or more steps not illustrated may be added and performed.

[0108] Referring to FIG. 11, the worker terminal (310) can create a map of the target space in real time based on images acquired from the camera (S1110).

[0109] The worker terminal (310) can display the created map information on the display unit (S1120). The worker can check the location recognition results and map creation results for previous images captured by the camera in real time through the map information displayed on the display unit.

[0110] When capturing video with a camera at a new location in a target space, the worker can input information about the node closest to the current shooting location into the worker terminal (310) (S1130). At this time, the worker can input the adjacent node information by touching the node displayed on the display unit or by calling up a separate node input menu.

[0111] When an image is captured at a new location in the target space (S1140), the worker terminal (310) can recognize the location information of the currently captured image based on the image information captured by the camera and the adjacent node information input by the worker (S1150). At this time, the worker terminal (310) can more accurately recognize the location of the currently captured image by performing local feature matching with the image of the adjacent node input by the worker. Through this, the worker terminal (310) can reduce the possibility of mismatching with other node images in the target space, thereby improving the location recognition success rate and accuracy.

[0112] The worker terminal (310) can update a map of the target space based on the location information of the currently captured image (S1160). The worker terminal (310) can display the updated map information on the display unit.

[0113] The worker terminal (310) can repeatedly perform the operations of steps 1120 to 1160 described above until the location recognition process or map creation process is terminated.

[0114] As described above, the vision-based location recognition method according to the third embodiment of the present invention performs real-time location recognition based on current image information captured by a camera and adjacent node information input from a worker, thereby reducing the possibility of mismatching with other node images within the target space and improving the location recognition success rate and accuracy for the currently captured image.

[0115]

[0116] Fig. 12 is a flowchart illustrating a vision-based location recognition method according to a fourth embodiment of the present invention. The vision-based location recognition method according to this embodiment can be performed by the location recognition system (300) according to the third and fourth embodiments. In the illustrated flowchart, the vision-based location recognition method is described by dividing it into a plurality of steps, but at least some of the steps may be performed in a reversed order, combined with other steps and performed together, omitted, divided into substeps and performed, or one or more steps not illustrated may be added and performed.

[0117] Referring to FIG. 12, the worker terminal (310) can create a map of the target space in real time based on images acquired from the camera (S1210).

[0118] The worker terminal (310) can display the created map information on the display unit (S1220). The worker can check the location recognition results and map creation results for previous images captured by the camera in real time through the map information displayed on the display unit.

[0119] When capturing video with a camera at a new location in a target space, the worker can input the approximate location information of the current camera into the worker terminal (310) (S1230). At this time, the worker can input the camera location information by touching a specific point on the map area displayed on the display unit, or by calling up a separate location input menu.

[0120] When an image is captured at a new location in the target space (S1240), the worker terminal (310) can recognize the location information of the currently captured image based on the image information captured by the camera and the camera location information input by the worker (S1250). At this time, the worker terminal (310) detects the node closest to the current shooting point based on the camera location information input by the worker, and performs local feature matching with the image of the detected adjacent node to more accurately recognize the location of the currently captured image. Through this, the worker terminal (310) can reduce the possibility of mismatching with images of other nodes in the target space, thereby improving the location recognition success rate and accuracy.

[0121] The worker terminal (310) can update a map of the target space based on the location information of the currently captured image (S1260). The worker terminal (310) can display the updated map information on the display unit.

[0122] The worker terminal (310) can repeatedly perform the operations of steps 1220 to 1260 described above until the location recognition process or map creation process is completed.

[0123] As described above, the vision-based location recognition method according to the fourth embodiment of the present invention performs real-time location recognition based on current image information captured by a camera and camera location information input from a worker, thereby reducing the possibility of mismatching with other node images within the target space and improving the location recognition success rate and accuracy for the currently captured image.

[0124]

[0125] Figure 13 is a block diagram of a computing device according to an embodiment of the present invention.

[0126] Referring to FIG. 13, a computing device (1300) according to an embodiment of the present invention includes at least one processor (1310), a computer-readable storage medium (1320), and a communication bus (1330). The computing device (1300) may implement the worker terminal (110, 210, 310) or the operation server (120, 220) described above.

[0127] The processor (1310) may cause the computing device (1300) to operate according to the exemplary embodiments described above. For example, the processor (1310) may execute one or more programs (1325) stored in a computer-readable storage medium (1320). The one or more programs may include one or more computer-executable instructions, which, when executed by the processor (1310), may be configured to cause the computing device (1300) to perform operations according to the exemplary embodiments.

[0128] The computer-readable storage medium (1320) is configured to store computer-executable instructions or program code, program data, and / or other suitable forms of information. A program (1325) stored in the computer-readable storage medium (1320) includes a set of instructions executable by the processor (1310). In one embodiment, the computer-readable storage medium (1320) may be a memory (volatile memory such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, any other form of storage medium that can be accessed by the computing device (1300) and store desired information, or a suitable combination thereof.

[0129] A communication bus (1330) interconnects various other components of the computing device (1300), including the processor (1310) and computer-readable storage medium (1320).

[0130] The computing device (1300) may also include one or more input / output interfaces (1340) and one or more network communication interfaces (1360) that provide interfaces for one or more input / output devices (1350). The input / output interfaces (1340) and the network communication interfaces (1360) are connected to a communication bus (1330).

[0131] The input / output device (1350) may be connected to other components of the computing device (1300) via the input / output interface (1340). Exemplary input / output devices (1350) may include input devices such as a pointing device (such as a mouse or a trackpad), a keyboard, a touch input device (such as a touchpad or a touchscreen), a voice or sound input device, various types of sensor devices and / or photographing devices, and / or output devices such as a display device, a printer, a speaker, and / or a network card. The exemplary input / output device (1350) may be included within the computing device (1300) as a component constituting the computing device (1300), or may be connected to the computing device (1300) as a separate device distinct from the computing device (1300).

[0132] The present invention described above can be implemented as computer-readable code on a medium recording a program. The computer-readable medium may be one that continuously stores a computer-executable program or one that temporarily stores it for execution or download. Furthermore, the medium may be a variety of recording or storage means, including a single or multiple hardware components, and is not limited to media directly connected to a computer system, but may also be distributed across a network. Examples of the medium include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and ROM, RAM, flash memory, and other media configured to store program instructions. Furthermore, other examples of media include recording or storage media managed by app stores that distribute applications, sites that supply or distribute various software, servers, etc. Therefore, the above detailed description should not be construed as limiting in all respects, but rather as illustrative. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

[0133] The present invention is not limited to the above-described embodiments and the attached drawings. It will be apparent to those skilled in the art that components of the present invention can be substituted, modified, and altered without departing from the technical spirit of the present invention.

Claims

1. In a vision-based location recognition method performed by an operation server, A step of transmitting map information about the target space to the worker terminal; A step of receiving image information captured at a specific location in the target space and additional information of a worker related to the specific location from the worker terminal; and A vision-based location recognition method including a step of recognizing the location of an image captured at a specific location based on image information and additional information received from the worker terminal.

2. In paragraph 1, A vision-based location recognition method, characterized in that the above-mentioned additional information is node information most adjacent to the specific location among the plurality of node information included in the above-mentioned map information.

3. In paragraph 2, A vision-based location recognition method, characterized in that the node information is input to the worker terminal through a touch input to any one of a plurality of nodes displayed on the worker terminal.

4. In paragraph 2, A vision-based location recognition method, characterized in that the above node information is input to the worker terminal through a node input menu displayed on the worker terminal.

5. In the second paragraph, the location recognition step is, A vision-based location recognition method characterized by detecting an image of a node closest to a specific location and performing local feature matching with the detected node image to recognize the location of an image captured at the specific location.

6. In paragraph 1, A vision-based location recognition method, characterized in that the above additional information is camera location information on map information corresponding to the specific location.

7. In paragraph 6, A vision-based location recognition method, characterized in that the above camera location information is input to the worker terminal through a touch input for a specific point in a map area displayed on the worker terminal.

8. In paragraph 6, A vision-based location recognition method, characterized in that the above camera location information is input to the worker terminal through a location input menu displayed on the worker terminal.

9. In paragraph 6, the location recognition step is, A vision-based location recognition method characterized by detecting a node closest to a specific location based on the camera location information and performing local feature matching with the detected node image to recognize the location of an image captured at the specific location.

10. In paragraph 1, A vision-based location recognition method further comprising a step of updating the map information based on location information of an image captured at the specific location.

11. In a computational server that includes one or more processors and performs vision-based location recognition, One or more of the above processors: Transmitting map information about the target space to the worker terminal; Receiving image information captured at a specific location in the target space and additional information of a worker related to the specific location from the worker terminal; and An operation server that recognizes the location of an image captured at a specific location based on image information and additional information received from the worker terminal.

12. In a vision-based location recognition method performed by a worker terminal, A step of displaying map information for a target space on a display unit; A step of acquiring image information captured at a specific location in the above target space; A step of receiving additional information related to the above specific location from a worker; and A vision-based location recognition method including a step of recognizing the location of an image captured at a specific location based on the acquired image information and the input additional information.

13. In paragraph 12, A vision-based location recognition method, characterized in that the above-mentioned additional information is node information most adjacent to the specific location among the plurality of node information included in the above-mentioned map information.

14. In the 13th paragraph, the location recognition step, A vision-based location recognition method characterized by detecting an image of a node closest to the specific location and performing local feature matching with the detected node image to recognize the location of an image captured at the specific location.

15. In paragraph 12, A vision-based location recognition method, characterized in that the above additional information is camera location information on map information corresponding to the specific location.

16. In paragraph 15, the location recognition step is: A vision-based location recognition method characterized by detecting a node closest to a specific location based on the camera location information and performing local feature matching with the detected node image to recognize the location of an image captured at the specific location.

17. In paragraph 12, A vision-based location recognition method further comprising a step of deleting at least one of a plurality of node information included in the map information according to a node deletion command of the worker.

Images