Method and system for obtaining photograph for safety inspection of facility by using drone

Abstract

The present disclosure relates to a technology for automatically capturing and obtaining photographs to be used for inspecting the safety of a facility, by using a drone. According to an embodiment of the present disclosure, a method of obtaining photographs for a safety inspection of a facility includes: an operation A, performed by a user terminal device, of displaying a user interface for setting a flight path of a drone, and receiving, from a user, an input of a flight path and actions to be performed at points on the flight path; and an operation B, performed by the drone, of receiving flight path data from the user terminal device, and performing the actions at the points on the set flight path while moving along the flight path.

Description

METHOD AND SYSTEM FOR OBTAINING PHOTOGRAPH FOR SAFETY INSPECTION OF FACILITY BY USING DRONE

[0001] The present disclosure relates to a method and system for obtaining a photograph for a safety inspection of a facility by using drone.

[0002] Structures such as bridges, tunnels, ports, dams, buildings, and ancillary facilities thereof built during construction works may age or develop defects over time. Consequently, safety inspections have been conducted recently on structures and social infrastructure and relevant regulations have been enacted to prevent significant damage to lives and property caused by unforeseen accidents.

[0003] Facilities are classified into Class-I facilities, which are large-scale facilities that require special management or advanced technology for structural safety and maintenance to promote convenient and safe public use, Class-II facilities, which are at high risk of disaster or require continuous management to prevent disasters and include infrastructure other than Class-I facilities, and Class-III facilities, which are small-scale facilities that require safe management and are at high risk of disaster or require continuous management to prevent disasters, such as facilities used by the general public, other than Class-II and Class-II facilities. Inspection of facilities are conducted according to the above classes.

[0004] In order to manage the lifespan of a facility and ensure the safety of the facility, several inspections may be performed. Regardless of the type of facility, it is important to detect and repair cracks and damage that may occur on the exterior of a facility at an early stage, which necessitates inspection work.

[0005] However, because related-art methods of inspecting the exterior of a facility are mostly conducted through direct visual checks by personnel, there are challenges in accurately inspecting areas of the facility that are high up or difficult for personnel to access.

[0006] Recently, with the increasing use of drones (unmanned aerial vehicles) across society, there have been attempts to utilize drones for inspecting the exteriors of facilities. However, when inspectors who are not familiar with drone operation use drones for inspections of the exterior of facilities, there may be issues such as a drone colliding with the facility or failing to photograph the exterior from an intended location, leading to difficulties in accurate inspections.

[0007] The present disclosure provides a method of obtaining photographs for a safety inspection of a facility by using a drone, and a system for performing the method.

[0008] In addition, the present disclosure provides a method and system for setting a flight path of a drone and an action to be performed at each point, and causing the drone to perform autonomous flight to capture photographs to be used for inspecting the safety of a facility.

[0009] According to an embodiment of the present disclosure, a method of obtaining photographs for a safety inspection of a facility includes: an operation A, performed by a user terminal device, of displaying a user interface for setting a flight path of a drone, and receiving, from a user, an input of a flight path and actions to be performed at points on the flight path; and an operation B, performed by the drone, of receiving flight path data from the user terminal device, and performing the actions at the points on the set flight path while moving along the flight path.

[0010] In an embodiment, in the operation A, a current location of the drone is set as a start point.

[0011] In an embodiment, in the operation A, information about a latitude, a longitude, an altitude, and an angle which are input by the user or measured by the drone is set as reference coordinates for an action point.

[0012] In an embodiment, in the operation A, grid flight or vertical flight is input as a flight type of the drone.

[0013] In an embodiment, based on the grid flight being input as the flight type of the drone, first reference coordinates to third reference coordinates for action points are input, fourth reference coordinates are generated according to Equation 1:

[0014] (Equation 1) x4= x3- (x2- x1), y4= y3- (y2- y1), z4= z3- (z2- z1),

[0015] ((x1, y1, z1): first reference coordinates / (x2, y2, z2) : second reference coordinates / (x3, y3, z3) : third reference coordinates / (x4, y4, z4) : fourth reference coordinates),

[0016] and then a plurality of action points in a grid shape, which are located at intervals set by the user within a figure having vertices corresponding to the first reference coordinates to the fourth reference coordinates are generated.

[0017] In an embodiment, based on the vertical flight being input as the flight type of the drone, reference coordinates of two points for action points are input, and a plurality of action points, which are located at intervals set by the user between the reference coordinates of the two points, are generated.

[0018] In an embodiment, in the operation B, the drone photographs a facility at an action point.

[0019] In an embodiment, in the operation B, the user terminal device displays points as distinct icons on a map in a user interface for performing autonomous flight of the drone, and displays a current location of the drone while updating the current location continuously or at set time intervals.

[0020] In an embodiment, the method further includes, after the operation B, an operation C, performed by the user terminal device, of receiving, from the drone, and storing a photograph of a facility.

[0021] According to another embodiment of the present disclosure, a system for obtaining photographs for a safety inspection of a facility includes: a user terminal device configured to display a user interface for setting a flight path of a drone, and receive, from a user, an input of a flight path and actions to be performed at points on the flight path; and a drone configured to receive flight path data from the user terminal device, and perform the actions at the points on the set flight path while moving along the flight path.

[0022] In another embodiment, the drone photographs a facility at an action point.

[0023] According to the present disclosure, a user may freely set a flight path of a drone and actions to be performed at points along the flight path, and thus, photographs of a facility may be captured at locations intended by the user. The captured photographs may be used to inspect the safety of the facility, helping to prevent accidents from occurring at the facility.

[0024] FIG. 1 is a diagram schematically illustrating a facility safety inspection system using a drone, according to an embodiment of the present disclosure.

[0025] FIGS. 2A to 2E are diagrams illustrating a user interface for setting a flight path, according to an embodiment of the present disclosure.

[0026] FIG. 3 is a diagram illustrating a user interface for performing autonomous flight of a drone, according to an embodiment of the present disclosure.

[0027] FIG. 4 is a diagram illustrating a user interface for viewing and modifying a flight path, according to an embodiment of the present disclosure.

[0028] FIG. 5 is a diagram illustrating an example of a process in which a drone performs autonomous flight.

[0029] FIGS. 6A and 6B are diagrams illustrating examples of processes in which a drone performs grid flight and vertical flight, respectively.

[0030] FIG. 7 is a diagram schematically illustrating a method of obtaining a photograph for a safety inspection of a facility by using a drone, according to an embodiment of the present disclosure.

[0031] FIGS. 8A to 8E are diagrams illustrating a user interface for classifying facility photographs, according to an embodiment of the present disclosure.

[0032] FIG. 9 is a diagram schematically illustrating a method of classifying photographs for a safety inspection of a facility, according to an embodiment of the present disclosure.

[0033] FIGS. 10A to 10E are diagrams illustrating a user interface for detecting damage to a facility, according to an embodiment of the present disclosure.

[0034] FIG. 11 is a diagram schematically illustrating a method of detecting damage to a facility from a photograph for a safety inspection of the facility, according to an embodiment of the present disclosure.

[0035] FIG. 12 is a diagram schematically illustrating a configuration of a drone, according to an embodiment of the present disclosure.

[0036] FIG. 13 is a diagram schematically illustrating a configuration of a computing device according to an embodiment of the present disclosure.

[0037] According to an embodiment of the present disclosure, a method of obtaining photographs for a safety inspection of a facility includes: an operation A, performed by a user terminal device, of displaying a user interface for setting a flight path of a drone, and receiving, from a user, an input of a flight path and actions to be performed at points on the flight path; and an operation B, performed by the drone, of receiving flight path data from the user terminal device, and performing the actions at the points on the set flight path while moving along the flight path.

[0038] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Advantages and features of the present disclosure and a method for achieving the same will be apparent with reference to embodiments described below together with the attached drawings. However, the present disclosure is not limited to the embodiments described below, but may be implemented in various different forms, the embodiments are solely provided to make the technical spirit of the present disclosure complete and to inform those of skill in the art to which the present disclosure pertains, of the full scope of the present disclosure, and the technical spirit of the present disclosure is only defined by the scope of the claims.

[0039] In the drawings, the same elements are assigned the same reference numerals although they are shown in different drawings. In addition, in describing the present disclosure, a detailed description of known functions and configurations incorporated herein will be omitted when it is determined that the description may make the gist of the present disclosure rather unclear.

[0040] Unless otherwise defined, all terms, including technical and scientific terms, used herein have the same meaning as commonly understood by those of skill in the art to which the present disclosure pertains. In addition, terms, such as those defined in commonly used dictionaries, will not be interpreted in an idealized or overly formal sense unless expressly so defined herein. Terms used herein are for the purpose of describing embodiments, and are not intended to limit the present disclosure. The singular expression used herein also includes the plural meaning unless mentioned otherwise.

[0041] In addition, in describing components of the present disclosure, expressions such as 'first', 'second', 'A', 'B', '(a)', or '(b)' may be used. These expressions are only intended to distinguish one component from another, and do not limit the nature, order, or sequence of the components. It should be understood that, when it is described that a first element is "connected", "coupled", or "connected" to a second element, the first element may be directly connected, coupled, or connected to the second element, and a third element may be connected, coupled, or connected between the first and second elements.

[0042] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0043] FIG. 1 is a diagram schematically illustrating a facility safety inspection system using a drone, according to an embodiment of the present disclosure.

[0044] A facility safety inspection system 100 (hereinafter, referred to as the system 100) using a drone according to an embodiment of the present disclosure may include a drone 110, a user terminal device 120, and a server computer 130.

[0045] The drone 110 may approach and photograph a facility through flight. In addition, the drone 110 may fly autonomously in accordance with a flight path set by a user, or may be manually controlled by the user. A configuration of the drone 110 will be described in detail below with reference to FIG. 12.

[0046] The user terminal device 120 may receive, from the user, an input for setting a flight path of the drone, and actions to be performed by the drone at points. The user may be a facility inspector who has been requested or instructed by a facility manager to conduct a safety inspection of the facility. The user terminal device 130 may be a computing device, for example, any one of a smart phone, a tablet computer, a laptop computer, a smart watch, a personal digital assistant (PDA), a video game console, a desktop computer, and a set-top box.

[0047] The server computer 130 may transmit, to the user terminal device 120, first application software or a web page for setting the flight path and the actions to be performed at the points. The server computer 130 may be a single computing device or a collection of a plurality of computing devices connected to each other via a computer network. The server computer 130 may be composed of, for example, a plurality of rack-mounted servers, blade servers, etc., and may be connected to a network device such as a router.

[0048] The user terminal device 120 and the server computer 130 may be connected to the computer network. The computer network may be, for example, the Internet, and may transmit or receive data by using data communication protocols such as Transmission Control Protocol (TCP) / Internet Protocol (IP).

[0049] The drone 110 may perform data communication with a drone control device or the user terminal device 120 located at a close range by using a frequency band of a short-range communication technology (e.g., Wi-Fi, Bluetooth, or Zigbee). Alternatively, the drone 110 may connect to the computer network by using a frequency band of a long-range communication technology (e.g., cellular or satellite communication), and may perform data communication with the user terminal device 120 or the server computer 130 located at a long range.

[0050] The user may control flight of the drone 110 by manually manipulating the drone control device or the user terminal device 120. Alternatively, when the user sets a flight path and actions to be performed at points by using the user terminal device 120, the user terminal device 120 or the server computer 130 may process commands for controlling the flight of the drone 110 and the actions to be performed at the points into data, and then transmit the data to the drone 110 such that the drone 110 may perform the commands.

[0051] The user terminal device 120 may execute the first application software installed therein. The user terminal device 120 may display a user interface by the first application software. Through the user interface, the user may set a flight path and actions to be performed at points on the flight path, and input a command to perform autonomous flight. The first application software is a web browser, and may receive a web page including a user interface from the server computer 130. Alternatively, the first application software may be provided by an application store of a platform to which the user terminal device 120 belongs, and a user interface may be programmed inside the first application software.

[0052] In the first application software, the user may select a menu item for connecting to the drone 110. The user terminal device 120 may recognize the drone 110 selected by the user, and transmit data to the drone 110 or receive data from the drone 110.

[0053] FIGS. 2A to 2E are diagrams illustrating a user interface for setting a flight path, according to an embodiment of the present disclosure.

[0054] When the user selects a menu item (not shown) for setting a flight path in the user interface, the user terminal device 120 may display a screen of a user interface for setting a flight path as illustrated in FIG. 2A.

[0055] In a flight path, a start point is where the drone 110 begins flight, an action point is where the drone 110 performs an action set by the user, and a pass point is where the drone 110 passes through without performing any actions.

[0056] In a first screen of the user interface for setting a flight path illustrated in FIG. 2A, a menu item for setting a start point of a flight path may be displayed.

[0057] In a region a1 of the first screen, an image being captured by the drone 110 connected to the user terminal device 120 may be displayed.

[0058] By selecting a region a2, the user may select a folder in which the set flight path is to be stored. By selecting a region a3, the user may create a folder in which the set flight path is to be stored. In a region A4, the user may input a name of the set flight path. In a region a5, the user may input an execution speed of the set flight path.

[0059] In a region a6, the user may set a start point. The user terminal device 120 connected to the drone 110 may receive, from the drone 110, information about a latitude, a longitude, and an altitude. When the user selects the region a6, the user terminal device 120 may set the current location of the connected drone 110 as the start point.

[0060] In a region a7, a front view image captured by the drone 110 at the start point may be displayed. In a region a8, information about the latitude, longitude, and altitude of the start point.

[0061] When the user selects a menu item to proceed to the next step in a region a9, the user terminal device 120 may display a second screen. Otherwise, when a menu item to revert to the previous step is selected, the user terminal device 120 may display an initial screen (not shown) of the user interface.

[0062] In a second screen of the user interface for setting a flight path illustrated in FIG. 2B, points set on the flight path may be displayed.

[0063] In a region b1 of the second screen, the name of a folder in which the set flight path is to be stored, a name of the flight path, an execution speed, and the number of set points may be displayed.

[0064] When the user selects a region b2, the user terminal device 120 may display a third screen for adding an action point. When the user selects a region b3, the user terminal device 120 may add the current location of the drone 110 as a pass point. A pass point may include information about a latitude, a longitude, and an altitude.

[0065] In a region b4, information about a latitude, a longitude, and an altitude for each set point, and an image captured by the drone 110 at the point may be displayed.

[0066] When the user selects a region b5, the user terminal device 120 may display a menu for deleting, copying, and modifying a point. The user terminal device 120 may delete, copy, or modify a point in response to a menu item selected by the user.

[0067] When the user selects a menu item to proceed to the next step in a region b6, the user terminal device 120 may display a fifth screen illustrated in FIG. 2E. Otherwise, when the user selects a menu item to revert to the previous step, the user terminal device 120 may display the first screen illustrated in FIG. 2A.

[0068] In the third screen of the user interface for setting a flight path illustrated in FIG. 2C, a menu for setting an action point on the flight path may be displayed.

[0069] In a region c1 of the third screen, the user may input a name of an action point. In a region c2, the user may select a flight type. For example, the user may select any one of grid flight and vertical flight as the flight type. In a region c3, the user may select an action to be performed by the drone 110 at the action point. For example, the user may select an action for the drone 110 to photograph the facility from the action point.

[0070] In a region c4, the user may select a manner of inputting reference coordinates. For example, the user may select any one of a manner in which the user manually inputs a latitude, a longitude, an altitude, an angle, and the like, or a manner in which coordinate information measured by the drone 110 is set as the reference coordinates.

[0071] In a region c5, the user may input a distance between the reference coordinates manually input by the user, and the drone 110. In a region c6, the user may delete the input reference coordinates.

[0072] In a region c7, the user may input reference coordinates. In response to the manner of inputting the reference coordinates that is selected in the region c4, a fourth screen illustrated in FIG. 2D, which is a dialog for the user to manually input a latitude, a longitude, an altitude, and an angle, may be displayed, or coordinate information measured by the drone 110 may be displayed in the region c7. In addition, in a case in which the flight type selected in the region c2 is "grid flight", a menu item for inputting coordinates of three vertices of a quadrangle may be displayed. Otherwise, in a case in which the flight type selected in the region c2 is "vertical flight", a menu item for inputting coordinates of two points may be displayed.

[0073] When the user selects a menu item for adding an action point in a region c8, the user terminal device 120 may store information about the action point set by the user, and display the second screen again. At this time, in the region b4 of the second screen, information about the latitude, longitude, and altitude of the added action point may be displayed. Otherwise, when the user selects a menu item to revert to the previous step, the user terminal device 120 may display the second screen again without storing the information about the action point set by the user.

[0074] In the fourth screen of the user interface for setting a flight path illustrated in FIG. 2D, a dialog for inputting reference coordinates of an action point may be displayed.

[0075] In regions d1 to d4 in the fourth screen, the user may input a latitude, a longitude, an altitude, and an angle of the action point. For example, a latitude and a longitude may be input with up to 13 decimal places, an altitude may be input with up to 2 decimal places, and an angle may be input with up to 1 decimal place. The angle may refer to an azimuth by which the facility is rotated clockwise or counterclockwise with respect to a north point. The north point may be any one of true north, grid north, and magnetic north.

[0076] When the user selects a 'Complete' menu item in a region d5, the latitude, longitude, altitude, and angle input by the user may be displayed in the region c7 of the third screen.

[0077] In the fifth screen of the user interface for setting a flight path illustrated in FIG. 2E, the flight path and points set by the user may be displayed.

[0078] In a region e1 of the fifth screen, the name of the folder in which the set flight path is to be stored, the name of the flight path, the execution speed, the number of set points, the number of images captured by the drone 110, and an expected flight duration may be displayed.

[0079] In a region e4, information about a latitude, a longitude, and an altitude for each set point, and an image captured by the drone 110 at the point may be displayed.

[0080] When the user selects a menu item for generating a path in a region e3, the user terminal device 120 may store information about a plurality of points set by the user in a file with the name of the flight path in the folder. The user terminal device 120 may transmit the stored file or data containing the flight path to the drone 110 or the server computer 130.

[0081] The drone 110 may receive the file or data containing the flight path from the user terminal device 120 directly or via the server computer 130. In accordance with the set flight path, the drone 110 may move between the points and perform the actions set by the user at the action points.

[0082] FIG. 3 is a diagram illustrating a user interface for performing autonomous flight of a drone, according to an embodiment of the present disclosure.

[0083] When the user selects a menu item for performing autonomous flight of a drone in the user interface, the user terminal device 120 may display a screen of a user interface for performing autonomous flight of a drone as illustrated in FIG. 3.

[0084] In a region f1 of the screen, the states of the drone 110 and the drone control device both connected to the user terminal device 120 may be displayed. For example, the state of charge of a battery of the drone 110, the state of charge of a battery of the drone control device, whether the drone 110 or the drone control device is charging, a global positioning system (GPS) signal reception state, the operation state of a compass (geomagnetic sensor) of the drone 110, the operation state of an inertial sensor (inertial measurement unit (IMU)) of the drone 110, and the like may be displayed.

[0085] In a region f2, an image being captured by the drone 110 connected to the user terminal device 120 may be displayed.

[0086] When the user selects a region f3 and then selects a stored flight path file, the user terminal device 120 may load the flight path file selected by the user. In a region f4, the name of the flight path, the name of a folder in which the flight path file is stored, an execution speed, the number of set points, the number of images captured by the drone 110, and an expected flight duration may be displayed.

[0087] In a region f5, the state of the drone 110 may be displayed. For example, the region f5 may show that the state of charge of the battery of the drone 110 is low ('Low battery'), that the drone 110 has temporarily stopped at one position ('Paused'), that the drone 110 is unable to perform flight ('Unable to fly'), that the drone 110 is flying ('In flight'), that the drone 110 is ready to fly ('Ready to fly'), or that the drone 110 is ready to receive an input for a flight path ('Waiting for input').

[0088] When the user selects a region f6, the user terminal device 120 may transmit a command to start from the start point, move to the next set point, and perform an action at the point, to the drone 110 directly or via the server computer 130. In response to the received command, the drone 110 may move to the next point and then perform the action. Alternatively, the user terminal device 120 may transmit the flight path file directly to the drone 110 or to the drone 110 via the server computer 130. By referring to the locations of the points stored in the received flight path file, the drone 110 may move to the next point and perform an action at the point.

[0089] When the user selects a region f7, the user terminal device 120 may transmit a command to stop flight to the drone 110 directly or via the server computer 130. The drone 110 may receive the command and then stop flying.

[0090] In a region f8, the flight path may be displayed along with a map. The user terminal device 120 may display distinct icons for respective points (e.g., a start point, action points, and pass points) on the map. The user terminal device 120 may receive coordinates (latitude, longitude, and altitude) of the drone 110 continuously or at set time intervals, and continuously update and display the current location of the flying drone 110 on the map.

[0091] FIG. 4 is a diagram illustrating a user interface for viewing and modifying a flight path, according to an embodiment of the present disclosure.

[0092] When the user selects a menu item (not shown) for viewing and modifying a flight path in the user interface, the user terminal device 120 may display a screen of a user interface for viewing and modifying a flight path as illustrated in FIG. 4.

[0093] In a region g1 of the screen, a list of project folders in which flight path files are stored may be displayed. When the user selects a project folder, a list of flight path files stored in the selected project folder may be displayed in a region g2. When the user selects a flight path file, information about the selected flight path file may be displayed in a region g3. For example, the name of a flight path, the name of a folder in which the flight path file is stored, an execution speed, the number of set points, the number of images captured by the drone 110, an expected flight duration, a date of creation or modification of the flight path file, and the like may be displayed.

[0094] When the user selects a region g4, a user interface for modifying a flight path may be displayed. The user interface for modifying a flight path may be the same as the user interface for setting a flight path illustrated in FIGS. 2A to 2E. In response to flight path information modified by the user, the user terminal device 120 may modify the flight path file.

[0095] When the user selects a region g5, a user interface for performing autonomous flight as illustrated in FIG. 3 may be displayed.

[0096] In a region g6, a front view image captured by the drone 110 at the start point may be displayed. In a region g7, the flight path may be displayed along with a map. The user terminal device 120 may display distinct icons for respective points (e.g., a start point, action points, and pass points) on the map. In a region g8, information about a latitude, a longitude, and an altitude for each point on the flight path may be displayed. For an action point, a name and a flight type may be displayed.

[0097] FIG. 5 is a diagram illustrating an example of a process in which a drone performs autonomous flight.

[0098] The drone 110 may start flying from a start point, pass through a plurality of pass points, and then photograph the facility at an action point. The drone 110 may receive information about a next point to be passed through, or a flight path file from the user terminal device 120 or the server computer 130, and move in response thereto. In addition, the drone 110 may obtain a photographic image by photographing a photography spot region of the facility.

[0099] When the drone 110 is not located at a start point S, the drone 110 may automatically move to the start point, or may receive a command from the user terminal device 120 or the server computer 130, move to the start point, and then move in response to the flight path set by the user. This is because, when the drone 110 starts flying from a location other than the start point, the drone 110 may not arrive exactly near the facility, making it impossible to photograph the facility, and there is a risk of colliding with other objects located along the flight path.

[0100] FIGS. 6A and 6B are diagrams illustrating examples of processes in which a drone performs grid flight and vertical flight, respectively.

[0101] When inputting an action point, the user may select grid flight or vertical flight as a flight type.

[0102] When grid flight is selected as the flight type, the user may input reference coordinates of three points as illustrated in (a) of FIG. 6A. The reference coordinates may represent vertices or corners of a region where the facility is to be photographed. The reference coordinates may be input by the user manually inputting a latitude, a longitude, and an altitude, or may be set based on coordinate information measured by the drone 110.

[0103] The user terminal device 120 or the server computer 130 may generate fourth reference coordinates by using the input first to third reference coordinates. When the first reference coordinates are represented as (x1, y1, z1), the second reference coordinates are represented as (x2, y2, z2), and the third reference coordinates are represented as (x3, y3, z3) in the Cartesian coordinate system, the fourth reference coordinates to be generated, i.e., (x4, y4, z4), may be represented as in Equation 1 below.

[0104] [Equation 1]

[0105]

[0106] The user may set an interval between action points. As illustrated in (b) of FIG. 6A, the user terminal device 120 or the server computer 130 may generate action points positioned at set intervals within a figure having vertices corresponding to the reference coordinates of the four points, respectively. The generated action points may be positioned at set intervals in a direction from the first reference coordinates toward the second reference coordinates, and in a direction from the first reference coordinates toward the fourth reference coordinates, thereby forming a grid shape.

[0107] The drone 110 may receive coordinate information about the generated action points, start from the reference coordinates of one point, and photograph each part of the facility while moving in a zigzag manner. For example, the drone 110 may start from the first reference coordinates, pass through a plurality of action points to the fourth reference coordinates, then move by one action point in the direction from the fourth reference coordinates toward the third reference coordinates, and then pass through a plurality of action points again in the direction from the fourth reference coordinates toward the first reference coordinates, and repeat such process, to finally move from the third reference coordinates to the second reference coordinates through a plurality of action points.

[0108] When vertical flight is selected as the flight type, the user may input reference coordinates of two points and an interval between action points, as illustrated in (a) of FIG. 6B. As illustrated in (b) of FIG. 6B, the user terminal device 120 or the server computer 130 may generate action points positioned at set intervals in a direction from the first reference coordinates toward the second reference coordinates.

[0109] The drone 110 may receive coordinate information about the generated action points, and photograph each part of the facility while moving in one direction or the opposite direction.

[0110] FIG. 7 is a diagram schematically illustrating a method of obtaining a photograph for a safety inspection of a facility by using a drone, according to an embodiment of the present disclosure. In the method, each step (S1100 to S1300) may be optionally included and performed.

[0111] In a flight path setting operation S1100, the user may set a flight path of the drone 110 and actions to be performed at points on the path, by using the user terminal device 120.

[0112] The user may input information about a start point, which is a location where the drone 110 is to start flying, action points, which are locations where the drone 110 is to photograph the facility or to perform other actions, and pass points, which are locations where the drone 110 is to pass through to photograph the facility. Input of pass point information may be optional. For example, the user may input a latitude, a longitude, an altitude, and an angle of a point, an action to be performed at the point, the type of flight to be performed at the point, and the like.

[0113] In an autonomous flight operation S1200, the drone 110 may receive flight path data from the user terminal device 120 directly or via the server computer 130.

[0114] The drone 110 may receive data including information about all points on the flight path. Alternatively, at each point, the drone 110 may receive information about the next point to move to.

[0115] The drone 110 may start from the start point, move to the next set point, and then photograph the facility or perform other action in response to an action set for an action point.

[0116] In a facility photograph transmission operation S1300, the drone 110 may transmit captured facility photographs to the user terminal device 120 or the server computer 130.

[0117] The drone 110 may add metadata to the facility photographs. The metadata may include, for example, the latitude, longitude, altitude, and angle of a location where the facility photograph was captured, the type and identification number of the drone 110, the date and time the facility photograph was captured, information about the camera, a focal length, a flash, an International Organization for Standardization (ISO) sensitivity, an aperture, and a shutter speed, an image description, and the like.

[0118] The user terminal device 120 or the server computer 130 may store the received facility photographs, and execute a task of classify the facility photographs by section and component, and detecting damage from the facility photographs.

[0119] The user terminal device 120 may receive, from the user, an input for classifying the facility photographs by section and component. The server computer 130 may transmit, to the user terminal device 120, second application software or a web page for classifying facility photographs by section and component.

[0120] The user terminal device 120 may execute the second application software installed therein. The user terminal device 120 may display a user interface by the second application software. In the user interface, the user may input a command to classify facility photographs by section and component. The second application software is a web browser, and may receive a web page including a user interface from the server computer 130. Alternatively, the second application software may be provided by an application store of a platform to which the server computer 130 or the user terminal device 120 belongs, and a user interface may be programmed inside the second application software.

[0121] FIGS. 8A to 8E are diagrams illustrating a user interface for classifying facility photographs, according to an embodiment of the present disclosure.

[0122] When the user selects a menu item (not shown) for classifying facility photographs in the user interface, the user terminal device 120 may display a screen of a user interface for classifying facility photographs as illustrated in FIG. 8A.

[0123] In a first screen of the user interface for classifying facility photographs illustrated in FIG. 8A, a menu item for classifying facility photographs.

[0124] When the user selects a region h1 of the first screen and then selects a facility photograph file in a dialog, the user terminal device 120 may load the facility photograph file selected by the user and display the file name in a region h2. The user terminal device 120 may display the number of loaded facility photograph files in a region h3.

[0125] When the user selects a facility photograph file in the region h2 or selects all loaded facility photograph files by selecting a region h4, the user terminal device 120 may display the number of facility photograph files selected, in a region h5.

[0126] When the user selects a region h6, the user terminal device 120 may remove the selected facility photograph file from the list displayed in the region h2.

[0127] When the user selects a filter in a region h7, the user terminal device 120 may display only facility photograph files that meet conditions of the filter, in the region h2. The filter may be to select facility photograph files that match, for example, date or location information.

[0128] When the user selects a region h8, the user terminal device 120 may automatically classify facility photographs by section and component. When the user selects a region h9, the user terminal device 120 may display a user interface for the user to manually classify facility photographs.

[0129] When the user selects a region h10, the user terminal device 120 may store a result of classifying facility photographs automatically or manually by the user.

[0130] In a region h11, icons corresponding to facility photographs may be displayed along with a map.

[0131] When the user selects a region h12, the user terminal device 120 may display a general map in a region h11, and when the user selects a region h13, the user terminal device 120 may display a satellite map in the region h11. When the user selects a region h14, the user terminal device 120 may display a cadastral map in the region h11.

[0132] When the user selects a region h15 and then drags and drops on the map in the region h11, the user terminal device 120 may calculate a distance on the map in proportion to the dragged length.

[0133] When the user adjusts a slider in a region h16, the user terminal device 120 may enlarge or reduce the map in the region h11 in correspondence with the adjusted slider.

[0134] In a region h17, a mode indicating whether the map may be moved or edited (e.g. "MOVE" or "EDIT") may be displayed. In a region h18, the latitude and longitude of a location selected on the map or where the cursor is located may be displayed.

[0135] As illustrated in FIG. 8B, on the map in the region h11, an icon may be displayed at a location where a facility photograph was captured.

[0136] When the user selects a facility photograph in the region h2 or an icon on the map in the region h11, the user terminal device 120 may display a pop-up window h19. In the pop-up window h19, the name of the facility photograph, the date the facility photograph was captured, an image of the facility photograph, and the latitude, longitude, and altitude of the location where the facility photograph was captured may be displayed.

[0137] When the user selects the region h8 of the first screen, the user terminal device 120 may display a dialog for selecting a classification criterion.

[0138] When the user selects a menu item for classifying facility photographs by distance in the dialog, the user may additionally input a distance. The user terminal device 120 may classify facility photographs corresponding to icons located within the input distance from each reference point on the map, into one group.

[0139] When the user selects a menu item for classifying facility photographs based on a cadastral map in the dialog, the user terminal device 120 may classify facility photographs corresponding to icons located within any one of the location, lot number, category, and boundary line of land, into one group.

[0140] The user terminal device 120 may group facility photographs by using a machine learning model. Alternatively, the user terminal device 120 may transmit facility photographs to the server computer 130, and the server computer 130 may group the received facility photographs. The machine learning model may be based on, for example, any one of K-means clustering, K-nearest neighbors, and deep neural network.

[0141] As illustrated in FIG. 8C, on the map in the region h11, the user terminal device 120 may display icons corresponding to grouped facility photographs such that the facility photographs in each group are indicated in the same manner.

[0142] For example, the icons in the same group may be displayed to be identical to each other in one or more of shape and color. Alternatively, the user terminal device 120 may display a figure enclosing the icons in each group, such as a circle, an ellipse, or a polygon. In addition, in a region h20 located near the figure, the name of the group and the number of facility photographs belonging to the group may be displayed.

[0143] The user terminal device 120 may generate folders for the respective groups, classify the facility photographs, and move the facility photographs to the corresponding folders according to a result of the classification. The name of the folder may be the same as the name of the group.

[0144] As illustrated in FIG. 8D, when the user selects the region h9 of the first screen, the user terminal device 120 may display a menu for manually classifying groups.

[0145] When the user selects icons displayed on the map in the region h11 or draws a figure enclosing icons on the map, the user terminal device 120 may classify facility photographs corresponding to the selected icons or the icons enclosed within the figure, into one group. When the user sets three or more points on the map, icons enclosed within a closed curve formed by the points may be classified into one group.

[0146] When an icon is selected redundantly or is already enclosed within another figure, the user terminal device 120 may display a warning message indicating that the facility photograph corresponding to the icon cannot be classified redundantly into another group, and may stop the classification.

[0147] On the map in the region h11, a region h21 where the user may dissolve a group that has been set by the user may be displayed for each group. When the user selects the region h21, the user terminal device 120 may release the classification of facility photographs into the group. The user may classify again the facility photographs of which the classification has been released.

[0148] When the user selects 'Reclassify' in a region h22 after completing a classification task, the user terminal device 120 may move the facility photographs to folders corresponding to groups into which the facility photographs have been classified.

[0149] When there are unclassified facility photographs, the user terminal device 120 may display a dialog notifying of the presence of unclassified facility photographs. When the user selects a menu item for canceling classification in the dialog, the user terminal device 120 may highlight or focus on icons corresponding to the unclassified facility photographs on the map in the region h11. Otherwise, when the user selects a menu item for performing classification in the dialog, the user terminal device 120 may move only the classified facility photographs to folders corresponding to groups into which the facility photographs have been classified.

[0150] As illustrated in FIG. 8E, when the user completes a classification task and then selects the region h9 of the first screen, the user terminal device 120 may display a menu item for editing a group into which facility photographs have been classified.

[0151] When the user selects the region h21 that is displayed for each group of icons on the map in the region h11, the user terminal device 120 may release the classification of facility photographs corresponding to the icons into the corresponding group. In the example, it may be seen that the classification of icons into groups G1 and G2 in FIG. 8D has been released by selecting the region h21 indicated for each of the groups G1 and G2, as illustrated in FIG. 8E.

[0152] The user may classify again the icons of which the classification has been released, and this process is the same as described above with reference to FIG. 8D.

[0153] FIG. 9 is a diagram schematically illustrating a method of classifying photographs for a safety inspection of a facility, according to an embodiment of the present disclosure. In the method, each step (S2100 to S2400) may be optionally included and performed.

[0154] In a facility photograph loading operation S2100, by using the user terminal device 120, the user may select to load one or more facility photographs captured by a drone.

[0155] The user terminal device 120 may display a list of facility photograph files that are loaded from a user interface for classifying facility photographs. In addition, the user terminal device 120 may display icons corresponding to the facility photographs on a map in the user interface.

[0156] In a facility photograph classification operation S2200, by using the user terminal device 120, the user may select to automatically or manually classify the facility photographs.

[0157] When the user selects automatic classification, the user terminal device 120 may classify the facility photographs based on a distance or a cadastral map. For example, when classifying the facility photographs based on a distance, facility photographs corresponding to icons located within a distance input by the user from each reference point on the map may be classified into one group. Alternatively, when classifying the facility photographs based on a cadastral map, facility photographs corresponding to icons located within any one of the location, lot number, category, and boundary line of land, into one group.

[0158] When the user selects manual classification, and the user selects icons displayed on the map or draws a figure enclosing icons on the map, the user terminal device 120 may classify facility photographs corresponding to the selected icons or the icons enclosed within the figure, into one group.

[0159] The user terminal device 120 may display icons in the same group identically to each other, or display an area enclosing icons in the same group as a figure, in the user interface.

[0160] In a facility photograph classification editing operation S2300, by using the user terminal device 120, the user may release classification of facility photographs and classify again the facility photographs into another group.

[0161] The user terminal device 120 may provide, in the user interface, a menu item for releasing classification of facility photographs corresponding to icons that are already classified into the corresponding group. The user may select the menu item for releasing classification, return to the facility photograph classification operation S2200, and then classify the facility photographs again.

[0162] In a facility photograph moving operation S2400, the user terminal device 120 may move the facility photographs to folders corresponding to groups into which the facility photographs have been classified.

[0163] The user terminal device 120 may add metadata to a facility photograph file. The metadata may include, for example, information about sections and components for the group into which the facility photograph is classified.

[0164] The user terminal device 120 may receive, from the user, a command to detect damage from a facility photograph. The server computer 130 may transmit, to the user terminal device 120, third application software or a web page for detecting damage from a facility photograph.

[0165] The user terminal device 120 may execute the third application software installed therein. The user terminal device 120 may display a user interface by the third application software. In the user interface, the user may input a command to detect damage from a facility photograph. The third application software is a web browser, and may receive a web page including a user interface from the server computer 130. Alternatively, the third application software may be provided by an application store of a platform to which the server computer 130 or the user terminal device 120 belongs, and a user interface may be programmed inside the third application software.

[0166] FIGS. 10A to 10E are diagrams illustrating a user interface for detecting damage to a facility, according to an embodiment of the present disclosure.

[0167] When the user selects a menu item (not shown) for detecting damage from a facility photograph in the user interface, the user terminal device 120 may display a screen of a user interface for detecting damage from a facility photograph as illustrated in FIG. 10A.

[0168] In a first screen of the user interface for detecting facility damage illustrated in FIG. 10A, a menu for managing a project may be displayed.

[0169] When the user selects a region i1 of the first screen, the user terminal device 120 may generate a project, which is a list of facility photograph files from which damage is to be detected. When the user selects a region i2, the user terminal device 120 may load a stored project.

[0170] When the user selects a region i3, the user terminal device 120 may display a dialog for selecting a facility photograph file. When the user selects one or more facility photograph files, the user terminal device 120 may load the facility photograph files selected by the user and display the same in a region i4. The user terminal device 120 may display the number of loaded facility photograph files in a region i5.

[0171] When the user selects a facility photograph file in the region i4 or selects all loaded facility photograph files by selecting a region i6, the user terminal device 120 may display the number of facility photograph files selected, in a region i7.

[0172] When the user selects a filter in a region i8, the user terminal device 120 may display only facility photograph files that meet conditions of the filter, in the region i4. The filter may be to select facility photograph files that match, for example, date or location information.

[0173] When the user selects a facility photograph file in the region i4, the user terminal device 120 may display a facility photograph image for the selected facility photograph file, in a region i9.

[0174] When the user selects a region i10, the user terminal device 120 may display facility damage information in a pop-up window. The pop-up window displaying damage information will be described with reference to FIG. 10D.

[0175] When the user selects a region i11, the user terminal device 120 may display, in a pop-up window, metadata of the facility photograph displayed in the region i9 as illustrated in FIG. 10B, or hide the displayed pop-up window.

[0176] As illustrated in FIG. 10B, the metadata displayed in a pop-up window j1 may include, for example, the type or format of the facility photograph file, the numbers or lengths of horizontal and vertical pixels, a focal length, the width and height of a camera sensor, an object distance, a ground sample distance (GSD), and the like. The object distance may refer to the distance between the drone 110 and the facility, i.e., the object.

[0177] When the user changes the object distance in a region j2 of the pop-up window and then selects a region j3, the user terminal device 120 may change the object distance of the facility photograph in correspondence with a value input by the user, and change the pixel distance (cm / px) (the distance in the horizontal (X) and vertical (Y) directions) of the GSD in correspondence with the input object distance.

[0178] When the user adjusts a slider in a region i12 of the first screen, the user terminal device 120 may enlarge or reduce the facility photograph image displayed in the region i9 in correspondence with the adjusted slider.

[0179] In a region i13, the facility photograph image may be displayed in a reduced size, and a portion of the facility photograph image displayed in the region i9 may be highlighted with a quadrangle. When the user enlarges the facility photograph image by adjusting the slider in the region i12, the user terminal device 120 may reduce the quadrangle displayed in the region i13, whereas, when the user reduces the facility photograph image, the user terminal device 120 may enlarge the quadrangle.

[0180] When the user selects a region i14, the user terminal device 120 may store a result of detecting damage from a facility photograph. The user may store the project in any one of a cloud service or the user terminal device 120.

[0181] A second screen of the user interface for detecting facility damage illustrated in FIG. 10C, a menu for detecting damage to a facility may be displayed.

[0182] When the user selects a menu item for analysis (not shown), the user terminal device 120 may display a second screen including regions k1 to k3, instead of the regions i1 to i3 and i14 in the first screen. In the second screen, the regions i4 to i13 may be displayed in the same manner as in the first screen.

[0183] When the user selects a menu item "Analyze all files" in the region k1 of the second screen, the user terminal device 120 may detect damage from all facility photograph files displayed in the region i4. Alternatively, when the user selects "Analyze single file", the user terminal device 120 may detect damage from the facility photograph displayed in the region i9. Alternatively, when the user selects "Analyze selected files", the user terminal device 120 may detect damage from the selected facility photograph files in the region i4.

[0184] When the user selects "Reset list" in the region k2, the user terminal device 120 may remove all facility photograph files from the region i4. When the user selects "Remove selected files", the user terminal device 120 may remove the selected facility photograph files from the region i4.

[0185] When the user selects "Extract results of all files" in the region k3, the user terminal device 120 may export, as a file, information about damage detected from all facility photograph files displayed in the region i4. When the user selects "Extract results of selected files" in the region k3, the user terminal device 120 may export, as a file, information about damage detected from the selected facility photograph files in the region i4. The user terminal device 120 may display and export detected damage as an image file or a computer-aided design (CAD) file, and store and export a list of pieces of damage information as a document file or a spreadsheet file.

[0186] When the user selects a menu item in the region k1, the user terminal device 120 may detect damage from a facility photograph. Alternatively, the user terminal device 120 may transmit the facility photograph to the server computer 130, and the server computer 130 may detect damage from the received facility photograph. The user terminal device 120 or the server computer 130 may detect damage from a facility photograph by using a machine learning model. The machine learning model may be, for example, any one of a convolutional neural network, a model derived from a convolutional neural network, and a You Only Look Once (YOLO) model.

[0187] The machine learning model may include a plurality of neural networks. Among the plurality of neural networks, a first neural network may separate a background and a facility image by removing the background from the facility photograph. A second neural network may detect damage from the separated facility image. The first neural network and the second neural network may be trained based on different images.

[0188] After detecting damage, the user terminal device 120 may indicate the detected damage on the facility photograph image in the region i9. The user terminal device 120 may indicate a damaged area in a first color (e.g., blue). In addition, each damage may be indicated with a number (e.g., 1, 2, 3, 4, ...). The user terminal device 120 may indicate damaged areas or boundary lines of the damaged areas with different figures depending on the type of damage.

[0189] When the user selects the region i10, the user terminal device 120 may display a list of pieces of detected damage information in a pop-up window as illustrated in FIG. 10D, or hide the displayed pop-up window.

[0190] A region l2 of a pop-up window l1 may display a list of pieces of information about the detected damage. The list may include the number, type, width, and length of damage. In a region l3, the number of pieces of damage information displayed in the list.

[0191] When the user selects a damage item in the region l2, the user terminal device 120 may highlight or focus on the selected damage item in the region i9.

[0192] The user may select damage information one by one in the region l2, or select all damage information in a region l4. When the user selects a region l5, the user terminal device 120 may remove the selected damage information from the list displayed in the region l2.

[0193] A third screen of the user interface for detecting facility damage illustrated in FIG. 10D, a menu for drawing damage to a facility may be displayed.

[0194] When the user selects a menu item for drawing damage (not shown), the user terminal device 120 may display the third screen including regions m1 to m8, instead of the regions i1 to i3 and i14 in the first screen. In the third screen, the regions i4 to i13 may be displayed in the same manner as in the first screen.

[0195] When the user selects a region m1, the user terminal device 120 may allow the user to draw damage in the region i9. The user may select the region m1 and then draw damage on a facility photograph image displayed in the region i9 by using an input device such as a touch screen, a digital pen, or a mouse. The user terminal device 120 may display the damage drawn by the user, in a second color (e.g., purple).

[0196] When the user draws damage and then performs a particular input (e.g., pressing the Enter key on a keyboard or attempting a double-click with a mouse), the user terminal device 120 may display a dialog for inputting the width of the damage. When the user inputs the width of the damage, the user terminal device 120 may analyze the type of the damage drawn, calculate the length of the damage, and then display the type, length, and width of the damage in the pop-up window l1.

[0197] The user terminal device 120 may determine the type of the damage by using a machine learning model. The machine learning model may be, for example, any one of a convolutional neural network and a model derived from a convolutional neural network. In addition, the user terminal device 120 may calculate the length of the damage drawn, in proportion to the size of the image adjusted by the user in the region i12.

[0198] When the user selects one or more damage items in the region i9 and then selects a region m2, the user terminal device 120 may remove the selected damage items from the region i9.

[0199] When the user selects a plurality of damage items in the region i9 and then selects a region m3, the user terminal device 120 may merge the plurality of selected damage items and display a result of the merging. The user terminal device 120 may indicate the type of the merged damage as "Merged", calculate the length of the merged damage by summing the lengths of all of the damage items before merged, and determine the width of the merged damage as the largest width of the damage items before merged.

[0200] When the user selects the merged damage in the region i9 and then selects a region m4, the user terminal device 120 may separate the merged damage into damage items. The user terminal device 120 may restore the damage items with information about their types, lengths, and widths before merged.

[0201] The user may select a region m5 and then draw damage in the region i9 as a figure (e.g., a circle, a triangle, or a quadrangle). In addition, the user may select a region m6 and then draw damage in the region i9 as a line.

[0202] When the user selects a region m7, the user terminal device 120 may select all damage items. When the user selects a region m8, the user terminal device 120 may unselect the damage items.

[0203] FIG. 11 is a diagram schematically illustrating a method of detecting damage to a facility from a photograph for a safety inspection of the facility, according to an embodiment of the present disclosure. In the method, each step (S3100 to S3400) may be optionally included and performed.

[0204] In a facility photograph loading operation S3100, by using the user terminal device 120, the user may select to load one or more facility photographs captured by a drone.

[0205] The user terminal device 120 may display a list of facility photograph files that are loaded from a user interface for detecting damage from a facility photograph. In addition, the user terminal device 120 may display a facility photograph image in the user interface.

[0206] In a facility damage detection operation S3200, by using the user terminal device 120, the user may input a command to detect damage from a facility photograph.

[0207] In response to the command of the user, the user terminal device 120 or the server computer 130 may detect damage from a selected facility photograph. The user terminal device 120 or the server computer 130 may detect damage from a facility photograph by using a machine learning model (e.g., a convolutional neural network, a model derived from a convolutional neural network, or a YOLO model).

[0208] In a facility damage editing operation S3300, the user may modify or remove the detected damage, or generate new damage.

[0209] The user terminal device 120 may provide a menu item for the user to modify or remove information about detected damage (e.g., type, width, length, or object distance), to merge a plurality of damage items, or to separate merged damage items. In response to a user input, the user terminal device 120 may modify or remove the detected damage.

[0210] The user terminal device 120 may provide a menu item for the user to draw damage in the user interface. In response to damage drawn by the user, the user terminal device 120 may generate new damage.

[0211] In a result extraction operation S3400, the user terminal device 120 may store and export information about damage that is detected or edited by the user, as a file.

[0212] The user terminal device 120 may display and export the detected damage as an image file or a CAD file, and store and export a list of pieces of damage information as a document file or a spreadsheet file.

[0213] FIG. 12 is a diagram schematically illustrating a configuration of a drone, according to an embodiment of the present disclosure.

[0214] The drone 110 may include a detection unit 111, a drive unit 112, a photographing unit 113, a control unit 114, and a communication unit 115.

[0215] The detection unit 111 may measure the state of the drone 110 and information about environments around the drone 110. The detection unit 111 may include a plurality of sensors and may obtain information by measuring the moving speed, inertia, and inclination of the drone (110). In addition, the detection unit 111 may obtain information by measuring the latitude, longitude, and altitude of a location where the drone 110 is located, an angle with respect to the north point, temperature, humidity, illuminance, and atmospheric pressure.

[0216] The drive unit 112 may generate power to enable the drone 110 to fly. The drive unit 112 may include, for example, a plurality of motors, gears, propellers, and a battery. The drone 110 may move as the plurality of motors are rotated by energy accumulated in the battery, and the propellers connected to the shafts of the motors through the gears are rotated together. By adjusting the angles of the rotation shafts of the propellers by the motors, the direction in which the drone 110 moves may be changed.

[0217] The photographing unit 113 may photograph a facility and environments outside the drone 110. The photographing unit 113 may include a digital camera, and may receive light reflected from the facility and environments, then convert the light into an electrical signal, and store the electrical signal as binary data.

[0218] The control unit 114 may control the operation of other components. The control unit 114 may include a microprocessor or a microcontroller and may read commands for flight, detection, and photographing, and instruct the corresponding components to process the commands.

[0219] The communication unit 115 may receive control commands transmitted to the drone 110, and transmit data measured or photographed by the drone 110. The communication unit 115 may include a data transceiver configured to transmit or receive data by using a communication link corresponding to, for example, cellular, Wi-Fi, Bluetooth, Zigbee, or satellite communication protocols, or by using a communication link of a particular frequency band (e.g., 900 MHz, 1.3 MHz, 2.4 MHz, or 5.8 GHz).

[0220] A drone control device 140 may enable a user to control flight of the drone 110. The drone control device 140 may be implemented as a joystick or a touch screen and may receive an input from the user to control the flight of the drone 110 and convert the input into an electric signal. In addition, the drone control device 140 may include a data transceiver that is the same as that of the drone 110. The data transceiver may transmit a command for controlling flight of the drone 110 or receive data from the drone 110.

[0221] FIG. 13 is a diagram schematically illustrating a configuration of a computing device according to an embodiment of the present disclosure.

[0222] A computing device 200 may include one or more processors 210, a bus 250, a communication interface 240, a memory 220 configured to load a computer program 260 to be executed by the processors 210, and a storage 230 storing the computer program 260. The drawing illustrates only components that are relevant to embodiments of the present disclosure. However, those of skill in the art will understand that other general-purpose components may be further included in addition to the components illustrated in the drawing.

[0223] The processors 210 control the overall operation of each component of the computing device 200. The processors 210 may be configured to include at least one of a central processing unit (CPU), a microprocessor unit (MPU), a microcontroller unit (MCU), a graphics processing unit (GPU), or any type of processor well known in the technical field of the present disclosure. In addition, the processors 210 may perform an arithmetic operation on at least one application or program for executing methods / operations according to various embodiments of the present disclosure. In addition, the computing device 200 may include one or more processors.

[0224] The memory 220 stores various types of data, commands, and / or information. The memory 220 may load one or more programs 260 from the storage 230 to execute methods / operations according to various embodiments of the present disclosure. For example, the memory 220 may be random-access memory (RAM) but is not limited thereto.

[0225] The bus 250 provides a communication function between components of the computing device 200. The bus 250 may be implemented as various types of buses, such as an address bus, a data bus, a control bus, etc.

[0226] The communication interface 240 supports wired and wireless Internet communication of the computing device 200. The communication interface 240 may support various communication methods in addition to the Internet communication. To this end, the communication interface 240 may be configured to include a communication module well known in the technical field of the present disclosure.

[0227] The storage 230 may non-temporarily store one or more computer programs 260. The storage 230 may be configured to include a non-volatile memory such as read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory, a hard disk, a solid-state drive (SSD), a detachable disk, or any type of computer-readable recording medium well known in the art to which the present disclosure pertains.

[0228] The computer program 260 may include one or more instructions implementing methods / operations according to various embodiments of the present disclosure.

[0229] When the computer program 260 is loaded into the memory 220, the processors 210 may execute the one or more instructions to perform methods / operations according to various embodiments of the present disclosure.

[0230] The technical spirit of the present disclosure described above may be implemented as computer-readable code on a computer-readable medium. The computer-readable recording medium may be, for example, a removable recording medium (e.g., a compact disc (CD), a digital video disc (DVD), a Blu-ray disc, a Universal Serial Bus (USB) storage device, or a removable hard disk) or a fixed recording medium (e.g., ROM, RAM, or a computer-attached hard disk). The computer program recorded on the computer-readable recording medium may be transmitted to another computing device through a network such as the Internet, and installed on and thus used by the other computing device.

[0231] While the embodiments of the present disclosure have been described with reference to the accompanying drawings, it will be understood by those of skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure. Therefore, it should be understood that the above-described embodiments of the present disclosure are exemplary in all respects and are not limited. The scope of protection of the present disclosure should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the technical idea defined by the present disclosure.

[0232] PRIOR ART DOCUMENTS

[0233] PATENT DOCUMENTS

[0234] (Patent Document 1) Structures such as bridges, tunnels, ports, dams, buildings, and ancillary facilities thereof built during construction works may age or develop defects over time.

[0235] Consequently, safety inspections have been conducted recently on structures and social infrastructure and relevant regulations have been enacted to prevent significant damage to lives and property caused by unforeseen accidents.

[0236] (Patent Document 2) Facilities are classified into Class-I facilities, which are large-scale facilities that require special management or advanced technology for structural safety and maintenance to promote convenient and safe public use, Class-II facilities, which are at high risk of disaster or require continuous management to prevent disasters and include infrastructure other than Class-I facilities, and Class-III facilities, which are small-scale facilities that require safe management and are at high risk of disaster or require continuous management to prevent disasters, such as facilities used by the general public, other than Class-II and Class-II facilities. Inspection of facilities are conducted according to the above classes.

[0237] (Patent Document 3) In order to manage the lifespan of a facility and ensure the safety of the facility, several inspections may be performed.

[0238] Regardless of the type of facility, it is important to detect and repair cracks and damage that may occur on the exterior of a facility at an early stage, which necessitates inspection work.

[0239] (Patent Document 4) However, because related-art methods of inspecting the exterior of a facility are mostly conducted through direct visual checks by personnel, there are challenges in accurately inspecting areas of the facility that are high up or difficult for personnel to access.

[0240] (Patent Document 5) Recently, with the increasing use of drones (unmanned aerial vehicles) across society, there have been attempts to utilize drones for inspecting the exteriors of facilities.

[0241] However, when inspectors who are not familiar with drone operation use drones for inspections of the exterior of facilities, there may be issues such as a drone colliding with the facility or failing to photograph the exterior from an intended location, leading to difficulties in accurate inspections.

Claims

1.A method of obtaining photographs for a safety inspection of a facility, the method comprising:an operation A, performed by a user terminal device, of displaying a user interface for setting a flight path of a drone and receiving, from a user, an input of a flight path and actions to be performed at points on the flight path; andan operation B, performed by the drone, of receiving flight path data from the user terminal device, and performing the actions at the points on the set flight path while moving along the flight path.2.The method of claim 1, wherein, in the operation A, a current location of the drone is set as a start point.3.The method of claim 1, wherein, in the operation A, information about a latitude, a longitude, an altitude, and an angle which are input by the user or measured by the drone is set as reference coordinates for an action point.4.The method of claim 1, wherein, in the operation A, grid flight or vertical flight is input as a flight type of the drone.5.The method of claim 4, wherein, based on the grid flight being input as the flight type of the drone, first reference coordinates to third reference coordinates for action points are input, fourth reference coordinates are generated according to Equation 1:[Equation 1] x4= x3- (x2- x1), y4= y3- (y2- y1), z4= z3- (z2- z1),((x1, y1, z1): first reference coordinates / (x2, y2, z2) : second reference coordinates / (x3, y3, z3) : third reference coordinates / (x4, y4, z4) : fourth reference coordinates),and then a plurality of action points in a grid shape, which are located at intervals set by the user within a figure having vertices corresponding to the first reference coordinates to the fourth reference coordinates are generated.6.The method of claim 4, wherein, based on the vertical flight being input as the flight type of the drone, reference coordinates of two points for action points are input, and a plurality of action points, which are located at intervals set by the user between the reference coordinates of the two points, are generated.7.The method of claim 1, wherein, in the operation B, the drone photographs a facility at an action point.8.The method of claim 1, wherein, in the operation B, the user terminal device displays points as distinct icons on a map in a user interface for performing autonomous flight of the drone, and displays a current location of the drone while updating the current location continuously or at set time intervals.9.The method of claim 1, further comprising, after the operation B, an operation C, performed by the user terminal device, of receiving, from the drone, and storing a photograph of a facility.10.A system for obtaining photographs for a safety inspection of a facility, the system comprising:a user terminal device configured to display a user interface for setting a flight path of a drone and receive, from a user, an input of a flight path and actions to be performed at points on the flight path; andthe drone configured to receive flight path data from the user terminal device and perform the actions at the points on the set flight path while moving along the flight path.11.The system of claim 10, wherein the drone photographs a facility at an action point.

Images