System and method for providing visualization envirnment to supprort robot operation based on uav-derived orthomosaic images
The system addresses inaccuracies in robot operation maps by generating high-resolution orthomosaics from UAV images, aligning them with robot coordinates, and constructing web tile maps for precise path planning, improving operational accuracy and efficiency.
Patent Information
- Application Number
- KR1020260076382
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2026-04-28
- Publication Date
- 2026-11-06
- Estimated Expiration
- 2046-04-28
AI Technical Summary
Existing map technologies, such as commercial maps and SLAM-based maps, fail to accurately represent the latest terrain and work conditions at construction sites with frequent changes, leading to inaccuracies in robot operation paths and inefficient map updates.
A system that generates high-resolution orthomosaics using UAV-captured images, aligns them with a robot operation coordinate system, and constructs web tile maps for intuitive robot path setting.
Provides a global map reflecting the latest site conditions, enhancing operational accuracy and efficiency by allowing precise robot path planning and overcoming limitations of commercial and SLAM-based maps.
Smart Images

Figure 112026051528867-PAT00002_ABST
Abstract
Description
Technology Field
[0001] The present invention relates to a system for providing a visualization environment, and more particularly to a system and method for providing a visualization environment for supporting robot operation based on UAV orthophotos, which enables the application of aerial images acquired using an Unmanned Aerial Vehicle (UAV) to robot operation. Background Technology
[0002] In existing construction sites, robots performing various tasks, such as mobile and transport robots and construction robots, are operated. Commercial maps or SLAM-based maps have been primarily utilized for operation, but various problems exist as follows.
[0003] First, as the terrain and work routes at large-scale construction sites change in short cycles due to equipment movement, material stacking, and earthworks, there is a problem in that existing map information does not sufficiently reflect the latest conditions of the site.
[0004] In particular, commercial maps have difficulty accurately representing detailed work areas, equipment layouts, temporary roads, and work routes within a construction site due to limitations such as shooting restrictions, resolution, and update cycles.
[0005] In addition, since SLAM-based maps are generated based on local areas observed by robots, there are limitations in intuitively grasping the global structure of large outdoor construction sites, and there are restrictions on using them as global operational reference maps due to accumulated errors.
[0006] In addition, at construction sites with frequent terrain changes, relying solely on SLAM-based maps to build or continuously update global maps requires iterative data acquisition, map regeneration, and correction work, which consumes a significant amount of time and cost, thus presenting economic and practical limitations in actual field operations.
[0007] FIGS. 1 to 3 are drawings illustrating major terrain shapes that cause difficulties when operating a robot using a satellite map or general map provided by the public according to the prior art.
[0008] Referring to Fig. 1, in the case of a golf course, public satellite maps or general maps are useful for identifying the overall layout of the golf course, the location of holes, approximate movement paths, and green areas. However, these maps have limitations in terms of the time of capture and resolution, and it is difficult to accurately reflect the detailed boundaries, turf conditions, terrain relief, work exclusion zones, open spaces around trees, and the actual work paths of robots required for actual turf management.
[0009] For example, at a golf course, the boundary between the fairway and the rough (a), the slope and elevation difference around bunkers (b), sensitive surface areas around the green (c), areas to avoid around trees (d), and the intersection of cart paths and grass areas directly affect the work plan of the lawn mower robot. Since fairways and roughs differ in grass height and maintenance standards, the work method may vary; around bunkers, driving stability may be reduced due to sandy terrain and slopes; and on the green, low-speed, precision work or restricted access may be required because surface flatness and grass quality are important.
[0010] Referring to FIG. 2, in the case of a pedestrian walkway, the curb and step difference (a) between the sidewalk and the roadway, the distinction between the pedestrian surface and the flowerbed or building boundary (b), the arrangement of street facilities such as bollards, benches, flowerpots, and streetlights (c), the stacking of temporarily placed boxes, pallets, and construction materials (d), and the actual passable width that changes accordingly have a direct influence on the work plan and path setting.
[0011] In particular, while sidewalk sections may appear as continuous paths of uniform width on maps, in actual field conditions, street furniture or temporary obstacles may reduce the width available for a robot to pass safely or require detours. Additionally, differences in elevation or boundary shapes with the roadway can be critical factors in determining the robot's ability to enter, driving stability, and setting safety distances.
[0012] Referring to Fig. 3, in the case of a construction site, the overall site layout and work flow (a), excavation boundaries (b), material storage locations (c), temporary fences, unpaved work surfaces, heavy equipment working radius (d), and worker movement paths continuously change as the process progresses. These factors directly affect the robot's driving range, restricted access zones, avoidance paths, and safety distance settings, but it is difficult to identify them in detail using only public satellite maps or general maps.
[0013] Therefore, if robot operation paths are established based solely on public satellite maps or commercial maps, discrepancies may occur with avoidance zones and sloped terrain. Consequently, limitations arise in establishing precise work plans and stable movement paths for the robot. Accordingly, high-resolution map information generated and updated based on the actual site is required for outdoor management spaces, sidewalks, construction sites, and other areas. Prior art literature
[0014] Registered Patent Publication No. 10-2918791 (Date of publication: January 28, 2026) The problem to be solved
[0015] The present invention has been devised to solve the aforementioned problems, and has the objective of providing a global map capable of reflecting the latest conditions of a construction site with frequent terrain changes by generating the latest high-resolution orthomosaic using UAV-captured images.
[0016] In addition, the present invention has another objective of ensuring that the position on the orthophoto corresponds to the robot operation position information by converting and aligning the position information of the generated orthophoto with a coordinate system suitable for the robot operation coordinate system or visualization environment.
[0017] In addition, the present invention has another objective of enabling efficient loading and output in a robot operation visualization environment by constructing orthophotos in a web tile map format.
[0018] In addition, the present invention has another objective in providing a work environment that allows an operator to more intuitively specify the initial position, target point, or waypoint of a robot by referring to an orthophoto on a visualization interface.
[0019] In addition, the present invention has another challenge in improving the accuracy and convenience of robot operation preparation by supplementing the latest global information of construction sites, which is difficult to obtain sufficiently with commercial maps or SLAM-based maps alone. means of solving the problem
[0020] To solve the aforementioned problem, a system for providing a visualization environment for supporting robot operation based on UAV orthophotos according to an embodiment of the present invention may include: an image acquisition unit that receives a plurality of original aerial images of a construction site taken from one or more UAVs; an image processing unit that aligns and corrects the plurality of original aerial images through one or more image processing techniques and generates an orthophoto by reflecting the geometric features of the actual terrain included in the corrected images; a coordinate system conversion unit that converts the geographic coordinate system of the orthophoto into a web coordinate system corresponding to the robot operation environment; a map construction unit that constructs a web tile map by dividing the orthophoto converted into the web coordinate system into a tile structure to generate a plurality of tile data; and an interface unit that displays the web tile map through a visualization interface in which one or more of the robot's initial position, target point, and waypoint are set.
[0021] The above UAV is equipped with RTK and GCP-based correction means and can generate the aerial image by correcting the position and image correction of the captured raw image through the correction means.
[0022] The above aerial imagery may include one or more of the latest terrain, work zones, equipment deployments, and temporary routes of the area.
[0023] The above image processing unit can convert geographical coordinates, including latitude and longitude defined in the above orthophoto, into planar coordinates in meters and convert them into the above web coordinate system.
[0024] The above map construction unit can generate the web tile map by dividing the web coordinate system into multiple tiles in N×N (N is a natural number) pixel units, setting XYZ coordinates for each tile data, and saving them in a folder structure.
[0025] The above web tile map may include a plurality of zoom levels and grid indices for the orthophoto.
[0026] The above coordinate system transformation unit can convert the geographical coordinates of the orthophoto into the robot's global coordinates or coordinates according to the visualization environment, thereby adjusting a specific point within the orthophoto to correspond to the robot's operational location information.
[0027] The above interface unit can set the robot's initial position, target point, or waypoint on the web tile map according to the user's selection input by providing Rviz, Mapviz, or other visualization tools.
[0028] The above interface unit can provide a global visual reference map for the site by providing one or more of a commercial map and a SLAM map.
[0029] In addition, to solve the aforementioned problem, a method for providing a visualization environment for supporting robot operation based on UAV orthophotos according to an embodiment of the present invention is a method for supporting robot operation by a UAV orthophoto-based robot operation support system, and may include the steps of: receiving a plurality of original aerial images of a construction site taken from one or more UAVs; aligning and correcting the plurality of original aerial images through one or more image processing techniques to generate an orthophoto by reflecting the geometric features of the actual terrain included in the corrected images; converting the geographic coordinate system of the orthophoto into a web coordinate system corresponding to the robot operation environment; dividing the orthophoto converted into the web coordinate system into a tile structure to generate a plurality of tile data and constructing a web tile map; and displaying the web tile map through a visualization interface in which one or more of the robot's initial position, target point, and waypoint are set. Effects of the invention
[0030] According to an embodiment of the present invention, by generating an orthophoto based on aerial imagery acquired using a UAV, and converting and aligning it to the operational coordinate system of a robot system to establish a robot operation visualization environment, it is possible to provide a high-resolution global map that reflects the latest conditions of a construction site with frequent terrain changes.
[0031] In addition, the present invention has another effect of enabling more accurate identification of internal work zones and equipment layout status within a construction site by ensuring higher resolution and up-to-dateness compared to commercial maps.
[0032] In addition, the present invention has another effect of directly linking a point visually selected by a user to the actual robot operation coordinates by aligning the orthophoto with the global coordinate system of the robot system.
[0033] In addition, the present invention has another effect of improving operational convenience and work efficiency by enabling intuitive setting of the robot's initial position and target point in a ROS2-based robot operation environment.
[0034] In addition, the present invention has another effect of increasing the operational stability and usability of robots at large-scale construction sites by providing a global reference map that complements the limitations of SLAM-based local maps. Brief explanation of the drawing
[0035] FIGS. 1 to 3 are drawings illustrating major terrain shapes that cause difficulties when operating a robot using a satellite map or general map provided by the public according to the prior art. FIG. 4 is a schematic diagram showing the overall system configuration to which a visualization environment providing system for supporting robot operation based on UAV orthophotos according to an embodiment of the present invention is applied. FIG. 5 is a diagram showing the structure of a visualization environment providing support for robot operation based on UAV orthophotos according to an embodiment of the present invention. FIG. 6 is a diagram illustrating a map screen provided by an interface provided by a system for providing a visualization environment for supporting robot operation based on UAV orthophotos according to an embodiment of the present invention. FIG. 7 is a diagram illustrating a robot path setting screen provided by an interface of a system providing a visualization environment for supporting robot operation based on UAV orthophotos according to an embodiment of the present invention. FIG. 8 is a diagram illustrating a method for providing a visualization environment for supporting robot operation based on UAV orthophotos according to an embodiment of the present invention. Specific details for implementing the invention
[0036] The present invention as described above will be explained in detail through the attached drawings and embodiments.
[0037] It should be noted that the technical terms used in this invention are used merely to describe specific embodiments and are not intended to limit the invention. Furthermore, unless specifically defined otherwise in this invention, the technical terms used in this invention should be interpreted in the sense generally understood by those skilled in the art to which this invention pertains, and should not be interpreted in an overly broad or overly narrow sense. Additionally, if a technical term used in this invention is an incorrect technical term that fails to accurately express the concept of the invention, it should be replaced with a technical term that can be correctly understood by those skilled in the art. Moreover, general terms used in this invention should be interpreted according to their prior definitions or the context, and should not be interpreted in an overly narrow sense.
[0038] Furthermore, singular expressions used in the present invention include plural expressions unless the context clearly indicates otherwise. In the present invention, terms such as "composed of" or "comprising" should not be interpreted as necessarily including all of the various components or steps described in the invention, and should be interpreted as meaning that some of the components or steps may not be included, or that additional components or steps may be included.
[0039] Additionally, terms including ordinal numbers, such as first, second, etc., used in the present invention may be used to describe components, but the components should not be limited by these terms. The terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component.
[0040] Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the attached drawings. Identical or similar components are given the same reference number regardless of the drawing symbols, and redundant descriptions thereof will be omitted.
[0041] In the following description, the system of the present invention and each component constituting the same may be implemented in the form of a computer program executable by a known microprocessor and stored on a read-write recording medium.
[0042] FIG. 4 is a schematic diagram showing the overall system configuration to which a visualization environment providing system for supporting robot operation based on UAV orthophotos according to an embodiment of the present invention is applied.
[0043] Referring to FIG. 4, the system for providing a visualization environment for supporting robot operation based on UAV orthophotos according to an embodiment of the present invention is applied to a system for operating autonomous robots (90), etc., in golf courses, general roads, and large-scale construction sites, and is characterized by implementing and applying a robot support mapping pipeline that enables the use of a visualization environment for robot operation by generating an orthophoto based on an aerial photograph of a construction site obtained through a UAV (Unmanned Aerial Vehicle; 10), and converting and aligning it to be suitable for a coordinate system for operating a robot system.
[0044] As a configuration for this purpose, the UAV (10) is a device that captures images of the ground from the air to generate maps, orthophotos, 3D terrain data, etc., and is equipped with a multicopter or fixed wing, and is equipped with sensors for capturing such as RGB cameras and Lidar, and sensors for measuring position information such as GPS, RTK, and PPK, etc., to generate raw images of the area to be analyzed, and can transmit them in real time to the visualization environment providing system (100) of the present invention operating on the ground via an information and communication network.
[0045] In particular, the UAV (10) according to the embodiment of the present invention is operated in one or more ways to generate the latest aerial imagery and provide a map that reflects the latest conditions of a construction site where terrain changes occur frequently, thereby making it easier to move and control the robot (90) when operating it at the site.
[0046] While publicly available satellite maps or general maps are useful for providing global information over wide areas, they have limitations in precise work planning or operational path setting for robots because they fail to accurately reflect detailed on-site obstacles, boundaries, work surface conditions, walkable areas, clearance around equipment, and actual work routes.
[0047] For example, in the case of a lawn mowing robot at a golf course, satellite maps alone cannot accurately reflect the actual grass boundaries, areas to avoid around trees, work exclusion zones around bunkers and slopes, and the detailed shape of the area to be managed, making it difficult to establish a precise work path plan.
[0048] Furthermore, even in the case of sidewalk sections where patrol robots travel, it is difficult to accurately identify actual pedestrian boundaries, elevation differences between sidewalks and roadways, flowerbeds, bollards, the placement of street facilities, temporary obstructions, and passable widths using only general maps or satellite imagery, which may result in discrepancies with the actual operational route plan.
[0049] On the other hand, according to an embodiment of the present invention, a more precise work plan can be established by collecting images of the current terrain in real time taken by the UAV (10) and utilizing them for robot operation.
[0050] The user terminal (50) can control the robot by connecting locally or remotely to the visualization environment providing system (100) and using a web tile map based on aerial images provided by the UAV (10).
[0051] Various types of terminal devices, including stationary PCs, tablet PCs, and smartphones, can be used as such user terminals (50), and any device equipped with a communication means and a web browser that can access the visualization environment providing system (100) through an information and communication network such as the internet can be applied.
[0052] Additionally, although the drawing illustrates a structure in which the user terminal (50) and the visualization environment providing system (100) are implemented as separate devices, the visualization environment providing system (100) may be implemented in a form in which it is stored on a recording medium and mounted on and executed on the user terminal (50). That is, the user terminal (50) executes the visualization environment providing system (100) through a mounted microprocessor and receives aerial images provided from the UAV (10) to directly perform functions such as image conversion and robot (90) path setting.
[0053] Through this, the user can operate the robot by setting the current position of the robot (90) through a global map based on an orthophoto, rather than a map with misaligned coordinates or distorted terrain, in the visualization environment provided by the system of the present invention, and by specifying waypoints (wp1, wp2) to specify the path of the robot (90).
[0054] The visualization environment providing system (100) can receive original aerial images of the site captured from one or more of the aforementioned UAVs (10) in real time, and can combine them to generate a single high-resolution orthophoto by applying one or more image processing techniques. To this end, the visualization environment providing system (100) may apply photogrammetry techniques, etc., and can construct a global image that geometrically reflects the actual terrain.
[0055] An orthophoto refers to an image that has been made to look like a real map by removing tilting and distortion. In the case of general aerial images, there were problems such as tilting of objects on the ground depending on the shooting direction of the UAV (10), displaying different sizes depending on the height, or making it impossible to measure distances. However, an orthophoto is an accurate projection image that has been converted from aerial images to remove distortion so that it can be used like a map.
[0056] In addition, the visualization environment providing system (100) can convert the coordinate system of the currently generated orthophoto to be suitable for robot (90) operation and, based on this, generate a web tile map consisting of multiple tile data. To this end, a widely known tile data conversion tool, such as GDAL2Tiles, can be used, and a tile map in a form that can be loaded according to zoom level can be constructed.
[0057] In addition, the visualization environment providing system (100) can display the constructed web tile app as a background map in a visualization environment based on ROS2, etc., and provides this visualization environment to a user terminal (50) so that a path necessary for operating the robot (90) can be set. Furthermore, the system (100) provides an integrated power map with a high-resolution orthophoto overlay, thereby enabling the user operating the robot (90) to perform tasks by visually checking the orthophoto-based global map in an online or offline environment.
[0058] According to this configuration, the visualization environment providing system according to an embodiment of the present invention can provide the latest global map at a large-scale construction site where terrain changes frequently, and use it to support setting the robot's initial position, target point, and movement path more intuitively.
[0059] In particular, according to an embodiment of the present invention, there is an advantage in overcoming the dual limitations of existing SLAM-based maps, which have low long-term consistency in changing environments and fail to provide sufficient up-to-date, detailed information required for precise work planning.
[0060] Hereinafter, a system for providing a visualization environment for supporting robot operation based on UAV orthophotos according to an embodiment of the present invention will be described in detail with reference to the drawings.
[0061] FIG. 5 is a diagram showing the structure of a visualization environment providing support for robot operation based on UAV orthophotos according to an embodiment of the present invention.
[0062] Referring to FIG. 5, a visualization environment providing system (100) according to an embodiment of the present invention may include: an image collection unit (110) that receives a plurality of original aerial images of a construction site taken from one or more UAVs; an image processing unit (120) that aligns and corrects the plurality of original aerial images through one or more image processing techniques and generates an orthophoto by reflecting the geometric features of the actual terrain included in the corrected images; a coordinate system conversion unit (130) that converts the geographic coordinate system of the orthophoto into a web coordinate system corresponding to a robot operation environment; a map construction unit (140) that constructs a web tile map by dividing the orthophoto converted into a web coordinate system into a tile structure to generate a plurality of tile data; and an interface unit (150) that displays the web tile map through a visualization interface in which one or more of the robot's initial position, target point, and waypoint are set.
[0063] The image collection unit (110) can collect and aggregate original aerial images that serve as a source for map generation and alignment of the site. The visualization environment providing system (100) according to an embodiment of the present invention can be linked with one or more UAVs (10), and can secure data for a visualization environment by aggregating multiple aerial images transmitted in real time from them.
[0064] To this end, the UAV (10) can photograph the entire construction site using precise position information based on RTK and GCP, and can acquire multiple original aerial images for generating orthophotos. Here, RTK (Real-Time Kinematic) corrects GPS coordinate data in real time, so that accurate coordinates can be automatically generated at every moment of shooting when aerial images are captured, and GCP can provide reference points that can correct the entire image by matching the actual measured coordinates with the pixel locations of the image.
[0065] Through these aerial images, the image collection unit (110) can obtain image data that reflects the latest terrain, work area, equipment placement and temporary movement paths of the site.
[0066] The image processing unit (120) can generate a single high-resolution orthophoto by aligning and correcting a plurality of aerial images acquired by the image acquisition unit (110) using one or more image processing techniques.
[0067] To this end, the image processing unit (120) may be equipped with a photogrammetry technique or an orthophoto generation tool such as OpenDroneMap, and through this, can construct an orthophoto, which is a global image that geometrically reflects the actual terrain from an aerial image.
[0068] Here, OpenDroneMap is an open-source processing engine that takes captured images as input and generates orthophotos, 3D models, and terrain data, and can provide a photogrammetry processing pipeline that converts aerial images provided by a UAV (10) into survey and map data.
[0069] The coordinate system conversion unit (130) can convert the coordinate information of the orthophoto generated through the image processing unit (120) into a coordinate system that can be utilized in a robot operation environment. For example, the orthophoto may include geographical coordinates, and the coordinate system conversion unit (130) can convert the coordinates into a coordinate system suitable for web maps and visualization environments, particularly for robot operation, thereby supporting the subsequent map construction unit (140) to construct a tile map and easily apply it to the visualization process.
[0070] This is a function that converts geographical coordinates into local coordinates, and can be implemented, for example, by converting the coordinates of each pixel forming an orthophoto into a Universal Transverse Mercator (UTM), setting a reference point, calculating the difference from the currently set robot coordinates, and inputting the result into the Robot Operating System 2 (ROS2).
[0071] The map building unit (140) can generate multiple tile data by dividing the orthophoto, which has completed coordinate system transformation, into a web tile map format. The map building unit (140) may be equipped with a tool such as GDAL2Tiles, through which it can generate an XYZ tile structure based on the orthophoto and build a tile map that can be loaded according to zoom level. Here, GDAL2Tiles can provide a function to divide and form the orthophoto, which is a geo-referenced raster image, into a tile map for a web map.
[0072] To this end, the orthophoto can be divided into multiple resolutions (zoom levels) and divided into multiple small tiles of size N×N (N is a natural number), for example, 256×256. In addition, such a tile map can be output as a web tile map, which is a format that can be displayed on the web.
[0073] The interface unit (150) provides a visualization environment in which a web tile map can be viewed and a robot's path can be set on a user terminal (50), and the web tile map built by the map building unit (140) can be displayed as a background map in a ROS2-based visualization environment.
[0074] Additionally, the interface section (150) can load locally stored tiles from Mapviz or a similar visualization interface, thereby providing a function that allows the user terminal (50) to visually check the orthophoto-based global map even in an offline environment rather than an online environment.
[0075] According to this structure, the visualization environment providing system (100) of the present invention can provide an interface including a visualization environment to a user terminal (50) connected online or offline, and the user can select an initial location, target point, or waypoint of a robot by referring to a visualized global map through this. In addition, the visualization environment providing system (100) can utilize the selected point as location information necessary for robot operation.
[0076] In addition, the user can set a path by designating waypoints on a general map in a visualization environment, and the interface unit (150) allows the robot system to set a path by receiving waypoints on an integrated global map overlaid with a high-resolution orthophoto. Through this, the user can prepare for robot operation while intuitively understanding the conditions of the entire site.
[0077] That is, the visualization environment providing system according to the embodiment of the present invention can provide a visualized environment in which a user can more intuitively set location information necessary for robot operation while checking the latest status of the grass boundaries of a wide golf course, areas to avoid trees, work exclusion areas around bunkers and slopes, earthwork terrain of a construction site, material storage areas, equipment placement, temporary roads and work routes, and patrol areas on general roads.
[0078] Additionally, the visualization environment providing system according to an embodiment of the present invention can improve robot operation readiness and user convenience at construction sites by providing a global visual reference map separately from the local map provided by incorporating SLAM (Simultaneous Localization And Mapping).
[0079] According to this configuration, the visualization environment providing system according to an embodiment of the present invention can establish a pipeline of UAV image acquisition, orthophoto generation, coordinate system transformation, tile map construction, offline visualization, and result and utilization.
[0080] Hereinafter, the technical concept of the present invention will be explained in detail through an example of an interface screen provided to a user terminal by a visualization environment providing system according to an embodiment of the present invention.
[0081] FIG. 6 is a diagram illustrating a map screen provided by an interface provided by a system for providing a visualization environment for supporting robot operation based on UAV orthophotos according to an embodiment of the present invention.
[0082] Referring to FIG. 6, a visualization environment providing system according to an embodiment of the present invention can provide an interface familiar to the user, such as a Windows GUI, to a connected user terminal, FIG. 6(a) shows a basic RviZ screen with no background map applied, and FIG. 6(b) shows a background screen with a map applied to the campus being used.
[0083] In addition, Fig. 6(c) illustrates a screen showing a web tile map generated based on aerial images acquired from a UAV, and it can be seen that the errors and distortions present in the aerial images have been corrected in this web tile map.
[0084] Also, FIG. 6(d) illustrates a screen of an integrated global map in which an orthophoto is overlaid on a general map through an interface provided by a system according to an embodiment of the present invention, and the user can determine the robot's path on this type of global map and set waypoints.
[0085] FIG. 7 is a diagram illustrating a robot path setting screen provided by an interface of a system providing a visualization environment for supporting robot operation based on UAV orthophotos according to an embodiment of the present invention.
[0086] Referring to FIG. 7, a screen {Fig. 7(a)} for setting a path by designating waypoints for a robot on a general map and a visualization environment providing system according to an embodiment of the present invention for setting a path by designating waypoints on an integrated global map overlaid with a high-definition orthophoto provided through an interface {Fig. 7(b)} are compared. In the visualization environment provided according to an embodiment of the present invention, the user can intuitively check the current terrain conditions of the site and specifically set the movement path of the robot operating in the site to perform the work.
[0087] Hereinafter, a method for providing a visualization environment by the aforementioned system for providing a visualization environment for supporting robot operation based on UAV orthophotos will be described in detail with reference to the drawings.
[0088] FIG. 8 is a diagram illustrating a method for providing a visualization environment for supporting robot operation based on UAV orthophotos according to an embodiment of the present invention. In the following description, the executing entity for each step is the visualization environment providing system and its components according to an embodiment of the present invention, even if not otherwise specified.
[0089] Referring to FIG. 8, a method for providing a visualization environment according to an embodiment of the present invention is a method for supporting robot operation by a visualization environment providing system, and may include the steps of: receiving a plurality of original aerial images of a construction site taken from one or more UAVs (S100); aligning and correcting the plurality of original aerial images through one or more image processing techniques to generate an orthophoto by reflecting the geometric features of the actual terrain included in the corrected images (S110); converting the geographic coordinate system of the orthophoto into a web coordinate system corresponding to the robot operation environment (S120); dividing the orthophoto converted into the web coordinate system into a tile structure to generate a plurality of tile data and constructing a web tile map (S130); and displaying the web tile map through a visualization interface in which one or more of the robot's initial position, target point, and waypoint are set (S140).
[0090] First, a step (S100) of receiving multiple original aerial images of the entire construction site from one or more UAVs, and a step of collecting and aggregating aerial images of the site in real time from UAVs linked to the system.
[0091] Next, in the step (S110) of generating an orthophoto by aligning and correcting multiple original aerial images through one or more image processing techniques to reflect the geometric features of the actual terrain included in the corrected images, an image processing technique such as photogrammetry or OpenDroneMap is applied to the multiple aerial images collected through step S100 to align and correct them, thereby generating an orthophoto that reflects the actual terrain. Through this step S110, a high-resolution orthophoto can be generated.
[0092] Next, in the step (S120) of converting the geographic coordinate system of the orthophoto into a web coordinate system corresponding to the robot operation environment, the geographic coordinate system for the orthophoto generated in step S110 is converted into a coordinate system usable in the robot operation environment, that is, a coordinate system suitable for web maps and visualization environments. Accordingly, a specific point on the orthophoto is made to correspond to robot operation location information.
[0093] Next, in the step (S130) of constructing a web tile map by dividing the orthophoto converted to a web coordinate system into a tile structure to generate multiple tile data, the orthophoto with the completed coordinate system conversion is constructed in the form of a web tile map. In this step S130, the orthophoto is divided into a tile structure having multiple zoom levels and grid indices.
[0094] And, as a step (S140) of displaying the web tile map through a visualization interface in which one or more of the robot's initial position, target point, and waypoint are set, the tile map constructed in step S130 is displayed on the robot operation visualization interface, thereby enabling the user to set the robot's initial position, target point, or waypoint on the visualization interface. At this time, the visualization interface includes Rviz, Mapviz, or a similar robot operation visualization environment.
[0095] In addition, at step S140, the web tile map provided by the visualization interface is used separately or together with a commercial map or a SLAM-based map to provide a global visual reference map of the construction site.
[0096] Although many details are described in detail in the above description, this should be interpreted as an example of a preferred embodiment rather than as a limitation to the scope of the invention. Accordingly, the invention should not be determined by the described embodiment, but by the claims and equivalents thereof. Explanation of the symbols
[0097] 10 : UAV 50 : User terminal 90 : Robot 100 : Visualization environment providing system 110: Image acquisition unit 120: Image processing unit 130: Coordinate system transformation unit 140: Map construction unit 150 : Interface section
Claims
Claim 1 An image acquisition unit that receives multiple original aerial images of the entire construction site captured by one or more UAVs, and RTK and GCP-based position information corresponding to the multiple original aerial images; an image processing unit that aligns the multiple original aerial images through one or more image processing techniques, corrects the position and image distortion of the original aerial images based on the RTK and GCP-based position information, and generates an orthophoto that reflects the geometric features of the actual terrain; a coordinate system conversion unit that converts geographic coordinates including latitude and longitude corresponding to each pixel of the orthophoto into UTM coordinates, calculates the difference between the UTM coordinates and the robot coordinates based on a set reference point, inputs the difference into ROS2 so that the coordinates of the orthophoto correspond to the robot operation environment, and converts the orthophoto into a web coordinate system; and a web tile map that can be loaded in a robot operation visualization environment by dividing the orthophoto converted into the web coordinate system into multiple tile data in N×N (N is a natural number) pixel units according to multiple zoom levels, setting XYZ coordinates and a grid index for each of the multiple tile data, and storing the multiple tile data in a folder structure. A system for providing a visualization environment for supporting robot operation based on UAV orthophotos, comprising: a map building unit for building; and an interface unit that loads and displays the web tile map as a background map of a ROS2-based robot operation visualization environment, and sets one or more of the robot's initial position, target point, and waypoint based on robot operation location information corresponding to a point selected on the web tile map according to a user's selection input. Claim 2 delete Claim 3 In claim 1, the original aerial image is a system for providing a visualization environment for supporting robot operation based on UAV orthophotos, comprising one or more of the latest terrain, work zones, equipment placement, and temporary movement paths of the construction site. Claim 4 delete Claim 5 delete Claim 6 delete Claim 7 A system for providing a visualization environment for supporting robot operation based on UAV orthophotos, wherein the coordinate system transformation unit adjusts a specific point within the orthophoto to correspond to the global coordinates of the robot or robot operation position information used in the robot operation visualization environment. Claim 8 A system for providing a visualization environment for supporting robot operation based on UAV orthophotos, wherein, in claim 1, the interface unit provides the ROS2-based robot operation visualization environment using Rviz, Mapviz, or other visualization tools. Claim 9 A system for providing a visualization environment for supporting robot operation based on UAV orthophotos, wherein the interface unit provides an integrated global map by overlaying the web tile map onto a commercial map, or provides a global visual reference map for the construction site by displaying the web tile map together with a SLAM-based local map. Claim 10 A method for providing a visualization environment for robot operation support of a UAV orthophoto-based robot operation support system, comprising: receiving a plurality of original aerial images capturing the entire construction site from one or more UAVs, and RTK and GCP-based position information corresponding to the plurality of original aerial images; aligning the plurality of original aerial images through one or more image processing techniques, and correcting the position and image distortion of the plurality of original aerial images based on the RTK and GCP-based position information to generate an orthophoto image reflecting the geometric features of the actual terrain; converting geographic coordinates including latitude and longitude corresponding to each pixel of the orthophoto image into UTM coordinates, calculating the difference between the UTM coordinates and the robot coordinates based on a set reference point, inputting the difference into ROS2 so that the coordinates of the orthophoto image correspond to the robot operation environment, and converting the orthophoto image into a web coordinate system; dividing the orthophoto image converted into the web coordinate system into a plurality of tile data in N×N pixel units according to a plurality of zoom levels, setting XYZ coordinates and a grid index for each of the plurality of tile data, and storing the plurality of tile data in a folder structure to enable robot operation A method for providing a visualization environment for supporting robot operation based on UAV orthophotos, comprising: a step of constructing a web tile map that can be loaded in a visualization environment; and a step of loading and displaying the web tile map as a background map of a ROS2-based robot operation visualization environment, and setting one or more of the robot's initial position, target point, and waypoint based on robot operation location information corresponding to a point selected on the web tile map according to a user's selection input.
Citation Information
Patent Citations
System and method for managing and controlling disaster situation using drone
KR1020210015624A
Image matching method of image taken by unmanned aerial vehicle and device therefor
KR1020240005607A
