Method for detecting damages of facilities from photos for safety inspection of facilities, and system thereof

Abstract

The present disclosure relates to technology for detecting damage of a facility from a facility photo taken by a drone. An embodiment of the present disclosure includes an A step of displaying, by a user terminal device, a user interface for detecting damage from a facility photo and importing the facility photo; and a B step of inputting, by the user terminal device, a facility photo selected by a user to a machine learning model to detect damage and displaying a detected damage on the user interface.

Description

METHOD FOR DETECTING DAMAGES OF FACILITIES FROM PHOTOS FOR SAFETY INSPECTION OF FACILITIES, AND SYSTEM THEREOF

[0001] The present disclosure relates to a method for inspecting the safety of a facility by using a drone and a system that performs the method.

[0002] More specifically, the present disclosure relates to technology for detecting damage of a facility from a facility photo taken by a drone.

[0003] Structures, such as bridges, tunnels, ports, dams, buildings, and their auxiliary facilities, which are built through construction work, may age or may be defective over time. Accordingly, safety inspections have been conducted on structures and social infrastructure recently, and related laws have been enacted to prevent accidents that may cause massive damage to life and property.

[0004] Facilities are classified into a first type facility, which is a large-scale facility that requires special management to ensure public convenience and safety or requires advanced technology for structural safety and maintenance, a second type facility, which is a facility, such as social infrastructure facility except for the first type facility, that has high risk of disasters or requires continuous management to prevent disasters, and a third type facility, which is a small-scale facility requiring safety management except for the first type facility and the third type facility and is a facility, such as a multi-use facility, that has high risk of disasters or requires continuous management to prevent disasters.

[0005] In order to manage the life of a facility and ensure safety, various inspections may be performed. Even when the types of facilities are different, it is important to detect and repair cracks and damage that may occur in the exterior early, and accordingly, inspection work therefor may be performed.

[0006] However, because most of the conventional methods for inspecting the exterior of a facility are performed by people directly checking with their own eyes, there is a problem in that it is difficult to accurately inspect high points and points that are difficult for people to access.

[0007] In addition, as the use of drones has increased throughout society, attempts are being made to utilize a drone when inspecting the exterior of a facility. However, when an inspector who is not familiar with an operation of a drone inspects the exterior of a facility by using the drone, there may be a problem in that the drone crashes into the facility or fails to take a photo of the exterior of the facility at an intended position, which makes it difficult to conduct an accurate inspection.

[0008] Also, when visually determining the damaged area in a facility photo, the type and a size of damage may be incorrectly determined, and accordingly, a method for more accurately searching for damaged areas in the facility photograph is required.

[0009] An object of the present disclosure is to solve the above problems and to provide a method for detecting damage of a facility from a photo and a system for performing the method.

[0010] In addition, another object of the present disclosure is to provide a method and system that may automatically detect damage of a facility or allow a user to draw the damage manually.

[0011] In order to achieve the above-described object, an embodiment of the present disclosure includes an A step of displaying, by a user terminal device, a user interface for detecting damage from a facility photo and importing the facility photo; and a B step of inputting, by the user terminal device, a facility photo selected by a user to a machine learning model to detect damage and displaying a detected damage on the user interface.

[0012] In one embodiment, in the B step, the machine learning model includes a first neural network that deletes a background from the facility photo to separate the background from the facility image, and a second neural network that detects damage of a separated facility image.

[0013] In one embodiment, a C step of editing damage information in response to an input of the user is further included after the step B.

[0014] In one embodiment, in the C step, information on detected damage is modified or deleted, multiple damages are merged, or merged damages are separated.

[0015] In one embodiment, a distance between pixels of a ground reference resolution of the facility photo is changed in response to a photo-taking distance input by the user.

[0016] In one embodiment, in the C step, new damage is generated in response to damage drawn by the user on the user interface.

[0017] In one embodiment, after the B step or the C step, the user terminal device displays the damage in an image file or a CAD file and stores the image file of the CAD file, and stores the damage information in a document file or a spreadsheet file and exports the document file or the spreadsheet file.

[0018] Another embodiment of the present disclosure includes a user terminal device that displays a user interface for detecting damage from a facility photo, detects the damage by inputting a facility photo selected by a user to a machine learning model; and displays a detected damage on the user interface; and a server computer that provides the user interface to the user terminal device.

[0019] In another embodiment, the machine learning model includes a first neural network that deletes a background from the facility photo to separate the background from the facility image, and a second neural network that detects damage of a separated facility image.

[0020] In another embodiment, the user terminal device displays a user interface for modifying or deleting information on detected damage, merging multiple damages, or separating merged damages.

[0021] In another embodiment, the user terminal device displays the damage in an image file or a CAD file and stores the image file of the CAD file, and stores the damage information in a document file or a spreadsheet file and exports the document file or the spreadsheet file.

[0022] In order to achieve the above-described object, an embodiment of the present disclosure includes an A step of displaying, by a user terminal device, a user interface for detecting damage from a facility photo and importing the facility photo; and a B step of inputting, by the user terminal device, a facility photo selected by a user to a machine learning model to detect damage and displaying a detected damage on the user interface.

[0023] In one embodiment, in the B step, the machine learning model includes a first neural network that deletes a background from the facility photo to separate the background from the facility image, and a second neural network that detects damage of a separated facility image.

[0024] In one embodiment, a C step of editing damage information in response to an input of the user is further included after the step B.

[0025] In one embodiment, in the C step, information on detected damage is modified or deleted, multiple damages are merged, or merged damages are separated.

[0026] In one embodiment, a distance between pixels of a ground reference resolution of the facility photo is changed in response to a photo-taking distance input by the user.

[0027] In one embodiment, in the C step, new damage is generated in response to damage drawn by the user on the user interface.

[0028] In one embodiment, after the B step or the C step, the user terminal device displays the damage in an image file or a CAD file and stores the image file of the CAD file, and stores the damage information in a document file or a spreadsheet file and exports the document file or the spreadsheet file.

[0029] Another embodiment of the present disclosure includes a user terminal device that displays a user interface for detecting damage from a facility photo, detects the damage by inputting a facility photo selected by a user to a machine learning model; and displays a detected damage on the user interface; and a server computer that provides the user interface to the user terminal device.

[0030] In another embodiment, the machine learning model includes a first neural network that deletes a background from the facility photo to separate the background from the facility image, and a second neural network that detects damage of a separated facility image.

[0031] In another embodiment, the user terminal device displays a user interface for modifying or deleting information on detected damage, merging multiple damages, or separating merged damages.

[0032] In another embodiment, the user terminal device displays the damage in an image file or a CAD file and stores the image file of the CAD file, and stores the damage information in a document file or a spreadsheet file and exports the document file or the spreadsheet file.

[0033] FIG. 1 is a diagram schematically illustrating a safety inspection system for a facility using a drone, according to an embodiment of the present disclosure.

[0034] FIGS. 2A to 2E are diagrams schematically illustrating user interfaces for setting flight paths, in an embodiment of the present disclosure.

[0035] FIG. 3 is a diagram schematically illustrating a user interface for performing automatic flight of a drone, in an embodiment of the present disclosure.

[0036] FIG. 4 is a diagram schematically illustrating a user interface for searching for and modifying a flight path, in an embodiment of the present disclosure.

[0037] FIG. 5 is a view schematically illustrating an example of a process of an automatic flight of a drone.

[0038] FIGS. 6A and 6B respectively illustrate examples of processes in which a drone performs a grid flight and a vertical flight.

[0039] FIG. 7 is a flowchart schematically illustrating a method of obtaining photos for safety inspection of a facility by using a drone, in an embodiment of the present disclosure.

[0040] FIGS. 8A to 8E are diagrams schematically illustrating user interfaces for classifying facility photos, in an embodiment of the present disclosure.

[0041] FIG. 9 is a flowchart briefly illustrating a method of classifying photos for safety inspection of a facility, in an embodiment of the present disclosure.

[0042] FIGS. 10A to 10E are diagrams illustrating user interfaces for detecting damage to a facility, in an embodiment of the present disclosure.

[0043] FIG. 11 is a flowchart briefly illustrating a method of detecting damage to a facility from a photo for safety inspection of a facility, in an embodiment of the present disclosure.

[0044] FIG. 12 is a diagram briefly illustrating a configuration of a drone, in an embodiment of the present disclosure.

[0045] FIG. 13 is a diagram briefly illustrating a configuration of a computing device, in an embodiment of the present disclosure.

[0046] In order to achieve the above-described object, an embodiment of the present disclosure includes an A step of displaying, by a user terminal device, a user interface for detecting damage from a facility photo and importing the facility photo; and a B step of inputting, by the user terminal device, a facility photo selected by a user to a machine learning model to detect damage and displaying a detected damage on the user interface.

[0047] Hereinafter, embodiments of the present disclosure are described in detail with reference to the attached drawings. Advantages and features of the present disclosure and a method for achieving the advantages and features will be clear with reference to the embodiments described in detail below together with the attached drawings. However, the technical idea of the present disclosure is not limited to the embodiments below and may be implemented in various different forms, and the embodiments below are provided only to complete the technical idea of the present disclosure and to fully inform a person skilled in the art to which the present disclosure belongs of the scope of the present disclosure, and the technical idea of the present disclosure is defined only by the scope of the claims.

[0048] When adding reference signs to components of each drawing, it should be noted that the same components are given the same reference numerals as much as possible even when the components are illustrated in different drawings. In addition, when describing the present disclosure, when it is determined that a certain description of a related known configuration or function may obscure the gist of the present disclosure, the detailed description thereof is omitted.

[0049] Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used with a meaning that may be commonly understood by a person skilled in the art to which the present disclosure belongs. In addition, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly specifically defined. The terms used in the present specification are for the purpose of describing embodiments and are not intended to limit the present disclosure. In the present specification, the singular also includes the plural unless specifically stated in the phrase.

[0050] In addition, terms such as "first", "second", "A", "B", "(a)", "(b)", etc. may be used to describe components of the present disclosure. The terms are only intended to distinguish the component from other components, and the nature, order, or sequence of the component is not limited by the terms. When a component is described as being "connected," "combined," or "coupled" to another component, it should be understood that the component may be directly connected or coupled to the other component, but another component may also be "connected," "combined," or "coupled" between respective components.

[0051] Hereinafter, embodiments of the present disclosure are described in detail with reference to the attached drawings.

[0052] FIG. 1 is a diagram schematically illustrating a photo acquisition system for safety inspection of a facility using a drone, according to an embodiment of the present disclosure.

[0053] A safety inspection system 100 (hereinafter referred to as a "system") for a facility using a drone according to an embodiment of the present disclosure may include a drone 110, a user terminal device 120, and a server computer 130.

[0054] The drone 110 may approach a facility through flight and take pictures. In addition, the drone 110 may automatically fly through a flight path set by a user or may be manually controlled by a user. A configuration of the drone 110 is described in detail with reference to FIG. 12.

[0055] The user terminal device 120 may receive an input from a user for setting a drone's flight path and an operation to be performed at points. The user may be a facility inspector who is requested or instructed to perform safety inspection of the facility from the facility manager. The user terminal device 120 may be a computing device and may be, for example, one of a smartphone, a tablet computer, a laptop computer, a smart watch, a PDA, a video game console, a desktop computer, and a set-top box.

[0056] The server computer 130 may transmit first application software or a web page for setting a flight path and an operation to be performed at points to the user terminal device 120. The server computer 130 may be a single computing device or a set of multiple computing devices connected to each other through a computer network. The server computer 130 may be composed of, for example, multiple rack-mounted servers, blade servers, or so on, and may be connected to a network device such as a router.

[0057] The user terminal device 120 may be connected to the server computer 130 through a computer network. The computer network may be, for example, the Internet, and may transmit or receive data by using a data communication protocol, such as TCP / IP.

[0058] The drone 110 may communicate data with a drone control device or the user terminal device 120 located in a short distance by using a frequency band of a short-distance communication technology (Wi-Fi, Bluetooth, Zigbee, or so on). Alternatively, the drone 110 may be connected to a computer network by using a frequency band of a long-distance communication technology (cellular, satellite communication, or so on) and may communicate data with the user terminal device 120 or the server computer 130 located in a long distance.

[0059] A user may manually operate a drone control device or the user terminal device 120 to control the flight of the drone 110. Alternatively, when a user sets a flight path and an operation to be performed at a point to the user terminal device 120, the user terminal device 120 or the server computer 130 may process a command for controlling the flight of the drone 110 and the operation to be performed at the point into data and then transmit the data to the drone 110 such that the drone 110 may perform flight and the operation.

[0060] The user terminal device 120 may execute first application software installed thereon. The user terminal device 120 may display a user interface according to the first application software. In the user interface, a user may set a flight path and an operation to be performed at a point on a path and may input a command to perform automatic flight. The first application software is a web browser and may receive a web page including the user interface from the server computer 130. Alternatively, the first application software may be provided by an application store of a platform to which the user terminal device 120 belongs, and the user interface may be programmed in the first application software.

[0061] A user may select a menu item that is connected to the drone 110 from the first application software. The user terminal device 120 may recognize the drone 110 selected by a user and transmit data to the drone 110 or receive data from the drone 110.

[0062] FIGS. 2A to 2E are diagrams illustrating user interfaces for setting flight paths, in an embodiment of the present disclosure.

[0063] When a user selects a menu (not illustrated) for setting a flight path from the user interface, the user terminal device 120 may display a screen of a user interface for setting a flight path as illustrated in FIG. 2A.

[0064] In the flight path, a start point is a place where the drone 110 starts flight, and an action point is a point where the drone 110 performs an operation set by a user. A pass point is a point where the drone 110 passes by without performing any special operation.

[0065] On a first screen of a flight path setting user interface illustrated in FIG. 2A, a menu for setting the start point of a flight path may be displayed.

[0066] In an area a1 of the first screen, an image being taken by the drone 110 connected to the user terminal device 120 may be displayed.

[0067] When a user selects an area a2, the user may select a folder in which a set flight path is stored. When the user selects an area a3, a folder for storing the set flight path may be generated. A user may input a name of the set flight path in an area a4. The user may input an execution speed of the set flight path in an area a5.

[0068] The user may set a start point in an area a6. The user terminal device 120 connected to the drone 110 may receive latitude, longitude, and altitude information from the drone 110. When the user selects the area a6, the user terminal device 120 may set a current position of the connected drone 110 as the start point.

[0069] An area a7 may display a front view image taken by the drone 110 at the start point. An area a8 may display latitude, longitude, and altitude information of the start point.

[0070] When the user selects a next menu from an area a9, the user terminal device 120 may display a second screen. Otherwise, when the user selects a back menu, the user terminal device 120 may display an initial screen (not illustrated) of the user interface.

[0071] In the second screen of the flight path setting user interface illustrated in FIG. 2B, a point set in the flight path may be displayed.

[0072] In an area b1 of the second screen, a folder name in which the set flight path is stored, a name of the flight path, an execution speed, and the number of set points may be displayed.

[0073] When the user selects an area b2, the user terminal device 120 may display a third screen for adding action points. When the user selects an area b3, the user terminal device 120 may add a current position of the drone 110 as a pass point. The pass point may include latitude, longitude, and altitude information.

[0074] Latitude, longitude, and altitude information and images taken by the drone 110 at points may be displayed for each set point in an area b4.

[0075] When the user selects an area b5, the user terminal device 120 may display a menu for deleting, copying, and modifying points. The user terminal device 120 may delete, copy, and modify the points in response to the menu selected by the user.

[0076] When the user selects a next menu in an area b6, the user terminal device 120 may display a fifth screen illustrated in FIG. 2E. Otherwise, when the user selects the back menu, the user terminal device 120 may display the first screen illustrated in FIG. 2A.

[0077] In the third screen of the flight path setting user interface illustrated in FIG. 2C, a menu for setting an action point in the flight path may be displayed.

[0078] The user may input a name of the action point in an area c1 of the third screen. The user may select a flight type from an area c2. For example, the user may select one of grid flight and vertical flight as a flight type. The user may select an operation to be performed by the drone 110 at the action point in an area c3. For example, the user may select taking a photo of a facility at the action point by using the drone 110.

[0079] The user may select a method of inputting a reference coordinate in an area c4. For example, the user may select one of a method of manually inputting latitude, longitude, altitude, angle, and so on, and a method of setting the coordinate information measured by the drone 110 as a reference coordinate.

[0080] The user may input a distance between the reference coordinate directly input by the user and the drone 110 to an area c5. The user may delete the input reference coordinate from an area c6.

[0081] The user may input the reference coordinate to an area c7. In response to the method of inputting the reference coordinate selected from the area c4, a fourth screen of FIG. 2D, which is a dialog for a user to manually input latitude, longitude, altitude, and angle, may be displayed, or coordinate information measured by the drone 110 may be displayed in the area c7. In addition, when a flight type selected from the area c2 is "grid flight," a menu for inputting coordinates of three corners of a square may be displayed. Otherwise, when the flight type selected from the area c2 is "vertical flight," a menu for inputting two coordinates may be displayed.

[0082] When the user selects a menu for adding an action point in an area c8, the user terminal device 120 may store information on the action point set by the user and display the second screen again. At this time, the area b4 of the second screen may display latitude, longitude, and altitude information of the added action point. Otherwise, when the back menu is selected, the user terminal device 120 may display the second screen again without storing the information of the action point set by the user.

[0083] In the fourth screen of the flight path setting user interface illustrated in FIG. 2D, a dialog for inputting the reference coordinate of the action point may be displayed.

[0084] The user may input latitude, longitude, altitude, and angle of the action point in areas d1 to d4 of the fourth screen. For example, the latitude and longitude may be input up to 13 decimal places, the altitude may be input up to 2 decimal places, and the angle may be input up to 1 decimal place. The angle may represent an azimuth in which a facility rotates clockwise or counterclockwise based on a north point. The north point may be one of true north, geographic north, and magnetic north.

[0085] When a user selects a completion menu from an area d5, the latitude, longitude, altitude, and angle input by the user may be displayed in an area c7 of the third screen.

[0086] On a fifth screen of the flight path setting user interface illustrated in FIG. 2E, the flight path and a point set by a user may be displayed.

[0087] In an area e1 of the fifth screen, a name of a folder for storing the set flight path, a name of the flight path, an execution speed, the number of set points, the number of images taken by the drone 110, and the expected flight time may be displayed.

[0088] In an area e2, latitude, longitude, and altitude information and images taken by the drone 110 at points may be displayed for each set point.

[0089] When a user selects a menu for generating a path from an area e3, the user terminal device 120 may store information on multiple points set by the user in a file having a name of the flight path in the folder. The user terminal device 120 may transmit data including a stored file or a flight path to the drone 110 or the server computer 130.

[0090] The drone 110 may receive a file or data including a flight path directly from the user terminal device 120 or via the server computer 130. The drone 110 may move between points in response to the set flight path and perform an operation set by a user at an action point.

[0091] FIG. 3 is a diagram illustrating a user interface for performing automatic flight of a drone, in an embodiment of the present disclosure.

[0092] When a user selects a menu for performing automatic flight of a drone in a user interface, the user terminal device 120 may display a screen of the user interface for performing the automatic flight of the drone as illustrated in FIG. 3.

[0093] An area f1 of a screen may display a state of the drone 110 and a drone control device connected to the user terminal device 120. For example, a battery charging state of the drone 110, a battery charging status of the drone control device, whether the drone 110 or the drone control device is being charged, a reception state of a GPS signal, an operation state of a compass (geomagnetic sensor) of the drone 110, an operation state of an inertial sensor (inertial measurement unit; IMU), and so on may be displayed.

[0094] An area f2 may display an image being taken by the drone 110 connected to the user terminal device 120.

[0095] When a user selects an area f3 and selects a stored flight path file, the user terminal device 120 may import the flight path file selected by the user. An area f4 may display a name of a flight path, a name of a folder where the flight path file is stored, an execution speed, the number of set points, the number of images taken by the drone 110, and the expected flight time.

[0096] An area f5 may display a state of the drone 110. For example, the area f5 displays a state in which battery charging of the drone 110 is insufficient (low battery), or the drone 110 is temporarily stopped at one position (paused), may not fly (unable to fly), is flying (in flight), is ready to fly (ready for flight), or is ready to receive an input of a flight path (waiting for input).

[0097] When a user selects an area f6, the user terminal device 120 may move from a start point to a next set point, and directly transmit a command to perform an operation at a point to the drone 110 or may transmit the command to the drone 110 via the server computer 130. The drone 110 may move to the next point and perform an operation in response to the received command. Alternatively, the user terminal device 120 may directly transmit a flight path file to the drone 110 or transmit the flight path file to the drone 110 via the server computer 130. The drone 110 may move to the next point and perform an operation at a point by referring to a point position stored in the received flight path file.

[0098] When a user selects an area f7, the user terminal device 120 may directly transmit a command to stop the flight to the drone 110 or transmit the command to the drone 110 via the server computer 130. The drone 110 may stop flying after receiving the command.

[0099] An area f8 may display a flight path along with a map. The user terminal device 120 may display icons on the map by classifying the icons for each point (start, an action, and a pass point). The user terminal device 120 may receive coordinates (latitude, longitude, and altitude) of the drone 110 continuously or at a set time interval, and continuously update and display a current position of the drone 110 in flight on the map.

[0100] FIG. 4 is a diagram illustrating a user interface for searching for and modifying a flight path, in an embodiment of the present disclosure.

[0101] When a user selects a menu (not illustrated) for searching for and modifying a flight path from a user interface, the user terminal device 120 may display a screen of the user interface for searching for and modifying the flight path as illustrated in FIG. 4.

[0102] An area g1 of the screen may display a list of project folders that store flight path files. When a user selects one project folder, the area g2 may display a list of flight path files stored in the selected project folder. When a user selects one flight path file, an area g3 may display information on the selected flight path file. For example, a name of the folder where a name of the flight path and the flight path file is stored, an execution speed, the number of set points, the number of images taken by the drone 110, the expected flight time, the date of a generated or modified flight path file, and so on may be displayed.

[0103] When a user selects an area g4, a user interface for modifying the flight path may be displayed. The user interface for modifying the flight path may be the same as the user interface for setting the flight path illustrated in FIGS. 2A to 2E. The user terminal device 120 may change the flight path file in response to the flight path information modified by a user.

[0104] When a user selects an area g5, the user interface for performing the automatic flight illustrated in FIG. 3 may be displayed.

[0105] An area g6 may display a panoramic image taken by the drone 110 at a start point. An area g7 may display the flight path along with a map. The user terminal device 120 may display icons on the map by classifying the icons for each point (start, an action, and a pass point). An area g8 may display latitude, longitude, and altitude information for each point of the flight path. In the case of an action point, a name and flight type may be displayed.

[0106] FIG. 5 is a view illustrating an example of a process in which a drone performs an automatic flight.

[0107] The drone 110 may start flying from a start point, pass through multiple pass points, and then take photos of a facility at an action point. The drone 110 may receive information of the next point to pass through or a flight path file from the user terminal device 120 or the server computer 130, and move in response thereto. In addition, the drone 110 may take photos of a spot area of the facility and obtain a photographic image.

[0108] When the drone 110 is not located at a start point S, the drone 110 may automatically move to the start point, or move to the start point by receiving a command from the user terminal device 120 or the server computer 130, and then move in response to a flight path set by a user. This is because, when the drone 110 starts flying from a position other than the start point, the drone may not take photos of a facility because the drone does not accurately reach the vicinity of the facility and has a high risk of collision with another object located on the flight path.

[0109] FIGS. 6A and 6B respectively illustrate examples of processes in which a drone performs grid flight and vertical flight.

[0110] When inputting an action point, a user may select grid flight or vertical flight as a flight type.

[0111] When the grid flight is selected as the flight type, the user may input three reference coordinates as illustrated in (A) of FIG. 6A. The reference coordinates may represent vertices or edges of an area where a photo of a facility is to be taken. An input of the reference coordinates may be performed by either a method of directly inputting, by a user, latitude, longitude, and altitude, or a method of setting coordinate information measured by the drone 110 as the reference coordinates.

[0112] The user terminal device 120 or the server computer 130 may generate a remaining fourth reference coordinate by using the input first to third reference coordinates. When the first reference coordinate, the second reference coordinate, and the third reference coordinate of the Cartesian coordinate system are respectively represented as (x1, y1, z1), (x2, y2, z2), and (x3, y3, z3), the fourth reference coordinate (x4, y4, z4) to be generated may be represented as the following Equation 1.

[0113] [Equation 1]

[0114]

[0115] A user may set an interval between action points. As illustrated in (B) of FIG. 6A, the user terminal device 120 or the server computer 130 may generate action points located at each set interval in a shape in which four reference coordinates form vertices. The generated action points may be located at each set interval in a direction in which the first reference coordinate faces the second reference coordinate and in a direction in which the first reference coordinate faces the fourth reference coordinate, thereby forming a grid shape.

[0116] The drone 110 may receive coordinate information of the generated action points, start from one reference coordinate to take a photo of each part of a facility while moving in a zigzag. For example, the drone 110 starts from the first reference coordinate, passes through multiple action points to the fourth reference coordinate, moves by one action point in a direction from the fourth reference coordinate to the third reference coordinate, and then moves in a direction from the fourth reference coordinate to the first reference coordinate, and by repeating the above processes, the drone 110 may eventually complete movement from the third reference coordinate to the second reference coordinate via the multiple action points.

[0117] When vertical flight is selected as a flight method, a user may input two reference coordinates and intervals between the action points, as illustrated in (A) of FIG. 6B. As illustrated in (B) of FIG. 6B, the user terminal device 120 or the server computer 130 may generate action points located at each set interval in a direction from the first reference coordinate to the second reference coordinate.

[0118] The drone 110 may receive coordinate information of the generated action points and take photos of each part of a facility while moving in one direction or the opposite direction.

[0119] FIG. 7 is a flowchart briefly illustrating a method of obtaining photos for safety inspection of a facility by using a drone, in an embodiment of the present disclosure. In a method, each of step S1100 to step S1300 may be optionally included and performed.

[0120] In a flight path setting step S1100, a user may set a flight path of the drone 110 and operations to be performed at points on the path by using the user terminal device 120.

[0121] A user may input information on a start point which is a position where the drone 110 starts flying, an action point which is a position where the drone 100 take photos of a facility or perform other operations at the facility, and a pass point which is a position to be passed through to take photos of the facility. The input of the pass point information may be optional. For example, latitude, longitude, altitude, angle of a point, an operation to be performed at the point, the type of flight at the point, and so on may be input by a user.

[0122] In an automatic flight step (S1200), the drone 110 may receive flight path data directly from the user terminal device 120 or may receive the flight path data via the server computer 130.

[0123] The drone 110 may receive data including information on all points on the flight path. Alternatively, the drone 110 may receive information on a next point to be moved all the time at each point where the drone 110 is located.

[0124] The drone 110 may start from the start point, move to the next set point, and then take photos of a facility or perform other operations in response to the operation set at the action point.

[0125] In a facility photo transmission step S1300, the drone 110 may transmit the taken facility photos to the user terminal device 120 or the server computer 130.

[0126] The drone 110 may add metadata to the facility photos. The metadata may include, for example, the latitude, longitude, altitude, and angle of a position where the facility photos was taken, a model, identification number, and photo-taking date and time of the drone 110, a camera, a focal length, flash, ISO sensitivity, an aperture, shutter speed, image description, and so on.

[0127] The user terminal device 120 or the server computer 130 may store the received facility photos, classify the photos by section and member, and perform a task of detecting damage from the facility photos.

[0128] The user terminal device 120 may receive an input from a user to classify the facility photos by section and member. The server computer 130 may transmit second application software or a web page for classifying the facility photos by section and member to the user terminal device 120.

[0129] The user terminal device 120 may execute the second application software installed thereon. The user terminal device 120 may display a user interface according to the second application software. In the user interface, a user may input a command to classify the facility photos by section and member. The second application software is a web browser and may receive a web page including the user interface from the server computer 130. Alternatively, the second application software may be provided by an application store of a platform to which the server computer 130 or the user terminal device 120 belongs, and the user interface may be programmed in the second application software.

[0130] FIGS. 8A to 8E are diagrams illustrating user interfaces for classifying photos of a facility, in an embodiment of the present disclosure.

[0131] When a user selects a menu (not illustrated) for classifying the facility photos from the user interface, the user terminal device 120 may display a screen of the user interface for classifying the facility photos as illustrated in FIG. 8A.

[0132] The menu for classifying the facility photos may be displayed on a first screen of a facility photo classification user interface illustrated in FIG. 8A.

[0133] When a user selects an area h1 of the first screen and then selects a facility photo file from a dialog, the user terminal device 120 may import the facility photo file selected by the user and display a file name in an area h2. The user terminal device 120 may display the number of imported facility photo files in an area h3.

[0134] When a user selects a facility photo file in the area h2 or selects all imported facility photo files by selecting an area h4, the user terminal device 120 may display the number of selected facility photo files in an area h5.

[0135] When a user selects an area h6, the user terminal device 120 may delete the selected facility photo file from a list displayed in the area h2.

[0136] When a user selects a filter in an area h7, the user terminal device 120 may display only the facility photo file that meets conditions of the filter in the area h2. The filter may be for selecting a facility photo file that matches, for example, date and position information.

[0137] When a user selects an area h8, the user terminal device 120 may automatically classify the facility photos by section and member. When a user selects an area h9, the user terminal device 120 may display a user interface that allows the user to manually classify the facility photos.

[0138] When a user selects an area h10, the user terminal device 120 may automatically or manually store results of classifying the facility photos.

[0139] In an area h11, icons corresponding to the facility photos may be displayed along with a map.

[0140] When a user selects an area h12, the user terminal device 120 may display a general map in the area h11, and when the user selects an area h13, the user terminal device 120 may display a satellite map in the area h11. When a user selects an area h14, the user terminal device 120 may display a cadastral map in the area h11.

[0141] When a user selects an area h15 and performs dragging and dropping on the map in the area h11, the user terminal device 120 may calculate a distance on the map in proportion to a dragged length.

[0142] When a user adjusts a slider in an area h16, the user terminal device 120 may display the map in the area h11 by enlarging or reducing the map in response to an adjusted position.

[0143] An area h17 may display a mode (for example, "MOVE" or "EDIT") to indicate whether the map may be moved or edited. An area h18 may display latitude and longitude of a position which is selected on the map or at which the cursor is located.

[0144] As illustrated in FIG. 8B, an icon may be displayed at a position where the facility photos were taken, on the map in the area h11.

[0145] When a user selects one facility photo from the area h2 or selects one icon on the map in the area h11, the user terminal device 120 may display a pop-up window h19. The pop-up window h19 may display a name of the facility photo, a photo-taking date, an image of the facility photo, and the latitude, longitude, and altitude of a position where the facility photo was taken.

[0146] When a user selects the area h8 of the first screen, the user terminal device 120 may display a dialog for selecting a classification criterion.

[0147] When a user selects a menu for classifying the facility photos based on a distance in the dialog, the user may additionally input the distance. The user terminal device 120 may classify the facility photos corresponding to the icons located in the input distance from each reference point of the map into one group.

[0148] When a user selects a menu for classifying the facility photos based on the map in the dialog, the user terminal device 120 may classify the facility photos corresponding to an icon located at one of a location, a lot number, a category, and a boundary line of a land into one group.

[0149] The user terminal device 120 may group the facility photos by using a machine learning model. Alternatively, the user terminal device 120 may transmit the facility photos to the server computer 130, and the server computer 130 may group the received facility photos. The machine learning model may be one of, for example, K-means clustering, K-nearest neighbor, and a deep neural network.

[0150] As illustrated in FIG. 8C, the user terminal device 120 may display icons corresponding to the grouped facility photos on the map in the area h11 in the same manner for each group.

[0151] For example, one or more of a shape and a color of the icon may be displayed in the same manner. Alternatively, the user terminal device 120 may display each group as a shape, such as a circle, an ellipse, or a polygon that includes an icon therein. In addition, an area h20 located near the shape may display a name of the group and the number of facility photos belonging to the group.

[0152] The user terminal device 120 may generate a folder for each group and classify and move the facility photos. A name of the folder may be the name of the group.

[0153] As illustrated in FIG. 8D, when a user selects the area h9 of the first screen, the user terminal device 120 may display a menu for manually classifying groups.

[0154] When a user selects an icon displayed on a map in the area h11 or draws a shape including an icon on the map, the user terminal device 120 may classify a facility photo corresponding to the selected icon or an icon included in the shape into one group. When a user sets three or more points on the map, icons included inside a closed curve formed by points may be classified into one group.

[0155] When an icon is selected repeatedly or is previously included in another shape, the user terminal device 120 may display a warning message indicating that the facility photos corresponding to the icon may not be classified into another group and stop the classification.

[0156] An area h21 where a group manually classified by a user may be ungrouped may be displayed for each group on the map in the area h11. When a user selects the area h21, the user terminal device 120 may declassify the facility photos belonging to the group. A user may re-classify the declassified facility photos.

[0157] When a user selects re-classification in an area h22 after completing the classification task, the user terminal device 120 may move the facility photos to a folder corresponding to the classified group.

[0158] When there are unclassified facility photos, the user terminal device 120 may display a dialog notifying this. When a user selects a classification cancel menu from the dialog, the user terminal device 120 may highlight or focus on an icon corresponding to the unclassified facility photos on the map in the area h11. Otherwise, when a user selects a classification progress menu from the dialog, the user terminal device 120 may move only the classified facility photos to a folder corresponding to the classified group.

[0159] As illustrated in FIG. 8E, when a user completes the classification task and selects the area h9 of the first screen, the user terminal device 120 may display a menu for editing the classified group.

[0160] When a user selects the area h21 displayed in each group of icons on the map of the area h11, the user terminal device 120 may declassify the facility photos corresponding to the icons from the corresponding group. In the example, by selecting the areas h21 respectively illustrated in a group G1 and a group G2 in FIG. 8D, it can be seen that the icons belonging to the corresponding groups are declassified as illustrated in FIG. 8E.

[0161] A user may reclassify the declassified icons, and a procedure thereof is the same as described with reference to FIG. 8D.

[0162] FIG. 9 is a diagram briefly illustrating a method of classifying photos for safety inspection of a facility, in an embodiment of the present disclosure. In a method, each of step S2100 to step S2400 may be optionally included and performed.

[0163] In a facility photo importing step S2100, a user may select importing of one or more facility photos taken by a drone by using the user terminal device 120.

[0164] The user terminal device 120 may display a list of facility photo files imported from a user interface for classifying facility photos. In addition, the user terminal device 120 may display icons corresponding to the facility photos on a map in the user interface.

[0165] In a facility photo classification step S2200, a user may select automatic or manual classification of the facility photos by using the user terminal device 120.

[0166] When a user selects the automatic classification, the user terminal device 120 may classify the facility photos based on a distance or a cadastral map. For example, when the facility photos are classified based on a distance, the facility photos corresponding to icons located within the distance input by a user from each reference point on the map may be classified into one group. Alternatively, when the facility photos are classified based on a cadastral map, the facility photos corresponding to icons located within any one of a location, a lot number, a category, and a boundary line of a land may be classified into one group.

[0167] When a user selects the manual classification, and when the user selects the icon displayed on the map or draws a shape including an icon on the map, the user terminal device 120 may classify the facility photos corresponding to the selected icon or an icon included in the shape into one group.

[0168] The user terminal device 120 may display grouped icons in the user interface in the same manner, or may display an area including the grouped icons as a shape.

[0169] In a facility photo classification editing step S2300, a user may declassify the facility photo by using the user terminal device 120 and reclassify the declassified facility photos into another group.

[0170] The user terminal device 120 may provide a menu for declassifying the facility photo corresponding to an icon previously classified in the user interface from the corresponding group. A user may select a declassification menu and return to the facility photo classification step S2200 to reclassify the facility photo.

[0171] In a facility photo movement step S2400, the user terminal device 120 may move the facility photos to a folder corresponding to the classified group.

[0172] The user terminal device 120 may add metadata to a facility photo file, and the metadata may include, for example, section and member information of the group in which the facility photos are classified.

[0173] The user terminal device 120 may receive, from a user, a command to detect damage in the facility photos. The server computer 130 may transmit third application software or a web page for detecting damage in the facility photos to the user terminal device 120.

[0174] The user terminal device 120 may execute the third application software installed thereon. The user terminal device 120 may display a user interface according to the third application software. In the user interface, a user may input a command to detect damage in facility photos. The third application software is a web browser and may receive a web page including a user interface from the server computer 130. Alternatively, the third application software may be provided by an application store of a platform to which the server computer 130 or the user terminal device 120 belongs, and the user interface may be programmed in the third application software.

[0175] FIGS. 10A to 10E are diagrams illustrating user interfaces for detecting damage of a facility, in an embodiment of the present disclosure.

[0176] When a user selects a menu (not illustrated) for detecting damage from facility photos in a user interface, the user terminal device 120 may display a screen of the user interface for detecting damage from the facility photos as illustrated in FIG. 10A.

[0177] A first screen of a facility damage detection user interface illustrated in FIG. 10A may display a menu for managing a project.

[0178] When a user selects an area i1 of the first screen, the user terminal device 120 may generate a project which is a list of facility photo files for detecting damage. When a user selects an area i2, the user terminal device 120 may import a stored project.

[0179] When a user selects an area i3, the user terminal device 120 may display a dialog for selecting a facility photo file. When a user selects one or more facility photo files, the user terminal device 120 may import the facility photo file selected by the user and display the imported facility photo file in an area i4. The user terminal device 120 may display the number of imported facility photo files in an area i5.

[0180] When a user selects a facility photo file from the area i4 or selects all imported facility photo files by selecting an area i6, the user terminal device 120 may display the number of selected facility photo files in an area i7.

[0181] When a user selects a filter in an area i8, the user terminal device 120 may display, in the area i4, only the facility photo file that meets a filter condition. The filter may be to select a facility photo file that meets, for example, date and position information.

[0182] When a user selects a facility photo file in the area i4, the user terminal device 120 may display a selected facility photo image in an area i9.

[0183] When a user selects an area i10, the user terminal device 120 may display damage information of a facility in a pop-up window. The pop-up window that displays the damage information is described with reference to FIG. 10D.

[0184] When a user selects an area i11, the user terminal device 120 may display metadata of a facility photo displayed in the area i9 in the pop-up window illustrated in FIG. 10B, or hide the displayed pop-up window.

[0185] As illustrated in FIG. 10B, the metadata displayed in a pop-up window j1 may include, for example, the type or format of a facility photo file, the number or lengths of horizontal and vertical pixels, a focal length, a width and height of a camera sensor, a photo-taking distance, a ground sample distance (GSD), and so on. The photo-taking distance may indicate a distance between the drone 110 and a facility that is a subject.

[0186] When a user modifies the photo-taking distance in an area j2 of the pop-up window and then selects an area j3, the user terminal device 120 may modify the photo-taking distance of the facility photo in response to a value input by the user, and change a distance between pixels (a distance in the horizontal direction X and the vertical direction Y, cm / px) of the ground sample resolution (GSD) in response to the input photo-taking distance.

[0187] When a user adjusts an area i12 of a first screen, the user terminal device 120 may enlarge or reduce a facility photo image displayed in the area i9 in response to the adjusted position to display the enlarged or reduced facility photo image.

[0188] The reduced facility photo image may be displayed in an area i13, and the facility photo image displayed in the area i9 may be displayed by being highlighted in a square shape. When a user adjusts the area i12 to enlarge the facility photo image, the user terminal device 120 may display the square displayed in the i13 area in a reduced size, and conversely, when the facility photo image is reduced, the square may be displayed in an enlarged size.

[0189] When a user selects an area i14, the user terminal device 120 may store a result of detecting damage from the facility photo. A user may store the project in one of a cloud service and the user terminal device 120.

[0190] A menu for detecting damage of a facility may be displayed in a second screen of the facility damage detection user interface illustrated in FIG. 10C.

[0191] When a user selects an analysis menu (not illustrated), the user terminal device 120 may display the second screen in which an area k1 to an area k3 are displayed on the first screen instead of the area i1 to the area i3 and the area i14. The area i4 to the area i13 may be displayed on the second screen in the same manner as in the first screen.

[0192] When a user selects an "all file analysis" menu from the area k1 of the second screen, the user terminal device 120 may detect damage of all facility photo files displayed in the area i4. Alternatively, when a user selects "single file analysis," the user terminal device 120 may detect damage of the facility photo file displayed in the area i9. Alternatively, when a user selects "selected file analysis," the user terminal device 120 may detect damage of the facility photo file selected in the area i4.

[0193] When a user selects "list initialization" in the area k2, the user terminal device 120 may delete all facility photo files from the area i4. When a user selects "selected file deletion," the user terminal device 120 may delete the facility photo file selected from the area i4.

[0194] When a user selects "all file result extraction" in the area k3, the user terminal device 120 may output damage information detected from all facility photo files displayed in the area i4 as a file. When a user selects "selected file result extraction", the user terminal device 120 may output damage information detected from the facility photo file selected from the area i4 as a file. The user terminal device 120 may mark the detected damage on an image file or CAD file and output the marked image file or CAD file, and store a list of damage information in a document file or spreadsheet file and output the list.

[0195] When a user selects one menu from the area k1, the user terminal device 120 may detect damage from facility photos. Alternatively, the user terminal device 120 may transmit the facility photo to the server computer 130, and the server computer 130 may detect damage from the received facility photo. The user terminal device 120 or the server computer 130 may detect damage from the facility photo by using a machine learning model. The machine learning model may be one of, for example, a convolutional neural network, a model derived from the convolutional neural network, and a YOLO (you only look once) model.

[0196] The machine learning model may include multiple neural networks. Among the multiple neural networks, a first neural network may separate a background from a facility image by removing the background from the facility photo. A second neural network may detect damage from the separated facility image. Images used in a learning process of the first neural network and the second neural network may be different from each other.

[0197] After detecting the damage, the user terminal device 120 may display the detected damage on a facility photo image in the area i9. The user terminal device 120 may display the damaged area in a first color (for example, blue). In addition, numbers (1, 2, 3, 4, ...) may be displayed for each damage. The user terminal device 120 may display the damaged area or a boundary of the area in different shapes in response to the type of damage.

[0198] When a user selects the area i10, the user terminal device 120 may display a list of the detected damage information in a pop-up window illustrated in FIG. 10D, or hide the displayed pop-up window.

[0199] The list of the detected damage information may be displayed in an area l2 of the pop-up window l1. A number, type, width, and length of damage may be displayed in the list. The number of pieces of damage information displayed in the list may be displayed in an area l3.

[0200] When a user selects one damage from the area l2, the user terminal device 120 may highlight or focus on the selected damage item in the area i9.

[0201] A user may select damage information one by one from the area l2, or select all pieces of damage information in an area l4. When a user selects an area l5, the user terminal device 120 may delete the selected damage information from the list displayed in the area l2.

[0202] A menu for drawing damage of a facility may be displayed on a third screen of the facility damage detection user interface illustrated in FIG. 10E.

[0203] When a user selects a damage drawing menu (not illustrated), the user terminal device 120 may display the third screen in which an area m1 to an area m8 appear on the first screen instead of the area i1 to the area i3 and the area i14. The area i4 to the area i13 may be displayed on the third screen in the same manner as on the first screen.

[0204] When a user selects the area m1, the user terminal device 120 may allow the user to draw damage on the area i9. After selecting the area m1, the user may draw damage on the facility photo image displayed on the area i9 by using an input device, such as a touch screen, a digital pen, or a mouse. The user terminal device 120 may display the damage drawn by a user in a second color (for example, purple).

[0205] When a user draws damage and then performs a certain input (for example, pressing an Enter key on a keyboard or attempting a double-click by using a mouse), the user terminal device 120 may display a dialog for inputting a width of the damage. When a user inputs the width of the damage, the user terminal device 120 may analyze the type of the drawn damage, calculate a length, and then display the type, length, and width of the damage in the pop-up window l1.

[0206] The user terminal device 120 may determine the type of damage by using a machine learning model. The machine learning model may be one of, for example, a convolutional neural network and a model derived from the convolutional neural network. In addition, the user terminal device 120 may calculate a length of the drawn damage in proportion to a size of the image adjusted by a user in the area i12.

[0207] When a user selects one or more damages from the area i9 and selects the area m2, the user terminal device 120 may delete the selected damages from the area i9.

[0208] When a user selects multiple damages from the area i9 and selects the area m3, the user terminal device 120 may merge and display the selected multiple damages. The user terminal device 120 may display the type of merged damage as "merger", calculate a length by adding up lengths of all damages before merging, and determine a width as the largest width of the damages before merging.

[0209] When a user selects the merged damage from the area i9 and selects the area m4, the user terminal device 120 may separate the merged damage. The user terminal device 120 may restore the separated damage to the type, length, and width information before merging.

[0210] A user may select the area m5 and draw the damage in the area i9 as a shape (a circle, a triangle, a square, or so on). In addition, a user may select the area m6 and draw the damage in the area i9 as a line.

[0211] When a user selects the area m7, the user terminal device 120 may select all damages. When a user selects the area m8, the user terminal device 120 may deselect the damage.

[0212] FIG. 11 is a flowchart briefly illustrating a method of detecting damage of a facility from a safety inspection photo of the facility, according to an embodiment of the present disclosure. In a method, each of step S3100 to step S3400 may be optionally included and performed.

[0213] In a facility photo importing step (S3100), a user may select importing of one or more facility photos taken by a drone by using the user terminal device 120.

[0214] The user terminal device 120 may display a list of facility photo files imported from a user interface for detecting damage from a facility photo. In addition, the user terminal device 120 may display a facility photo image on the user interface.

[0215] In a facility damage detection step S3200, a user may input a command for detecting damage from the facility photo by using the user terminal device 120.

[0216] The user terminal device 120 or the server computer 130 may detect damage from the selected facility photo in response to the user's command. The user terminal device 120 or the server computer 130 may detect damage from the facility photo by using a machine learning model (for example, a convolutional neural network, a model derived from the convolutional neural network, or a YOLO (you only look once) model).

[0217] In a facility damage editing step S3300, a user may modify and delete the detected damage, or generate a new damage.

[0218] The user terminal device 120 may provide a menu for a user to modify and delete information (a type, a width, a length, a photo-taking distance, and so on) of the detected damage, merge multiple damages, or separate the merged damages. The user terminal device 120 may modify and delete the detected damage in response to the user's input.

[0219] The user terminal device 120 may provide a menu for a user to draw damage on a user interface. The user terminal device 120 may generate a new damage in response to the damage drawn by a user.

[0220] In a result extraction step S3400, the user terminal device 120 may store and output information on detected or user-edited damage as a file.

[0221] The user terminal device 120 may display and output the detected damage as an image file or a CAD file, and may store a list of damage information in a document file or a spreadsheet file and output the list.

[0222] FIG. 12 is a diagram briefly illustrating a configuration of a drone, in an embodiment of the present disclosure.

[0223] A drone 110 may include a detection unit 111, a driving unit 112, a photo-taking unit 113, a control unit 114, and a communication unit 115.

[0224] The detection unit 111 may measure a state of the drone 110 and environmental information around the drone 110. The detection unit 111 may include multiple sensors and may obtain information by measuring a movement speed, inertia, and inclination of the drone 110. In addition, the detection unit 111 may obtain information by measuring an angle, temperature, humidity, illuminance, and air pressure based on latitude, longitude, altitude, and a north point of a position at which the drone 110 is located.

[0225] The driving unit 112 may generate power such that the drone 110 may fly. The driving unit 112 may include, for example, a plurality of motors and gears, a propeller, and a battery. The plurality of motors rotate according to energy accumulated in the battery, and the propeller connected to shafts of the motors through the gears rotate together, and accordingly, the drone 110 may move. A direction in which the drone 110 moves may be changed by adjusting an angle of a rotation axis of the propeller by using the motors.

[0226] The photo-taking unit 113 may take photos of a facility and environment outside the drone 110. The photo-taking unit 113 may include a digital camera and may receive light reflected from the facility and environment, convert the light into an electrical signal, and store the electrical signal as binary data.

[0227] The control unit 114 may control operations of other components. The control unit 114 may include a microprocessor or a microcontroller and may read flight, detection, and photo-taking commands and instruct a component corresponding thereto to process the commands.

[0228] The communication unit 115 may receive a control command transmitted to the drone 110 and transmit the data measured by the drone 110 or the data obtained by taking photos by using the drone 110. The communication unit 115 may include, for example, a data transceiver that supports cellular, Wi-Fi, Bluetooth, Zigbee, satellite communication protocols or transmits or receives data through a communication link of a certain frequency band (900 MHz, 1.3 MHz, 2.4 MHz, or 5.8 GHz).

[0229] A drone control device 140 may allow a user to control flight of the drone 110. The drone control device 140 may be implemented in a joystick or touch screen manner, and may receive an input for controlling the flight of the drone 110 from a user and convert the input into an electrical signal. In addition, the drone control device 140 may include the same data transceiver as the drone 110. The data transceiver may transmit a command to control the flight of the drone 110 or receive data from the drone 110.

[0230] FIG. 13 is a diagram briefly illustrating a configuration of a computing device, in an embodiment of the present disclosure.

[0231] A computing device 200 may include at least one processor 210, a bus 250, a communication interface 240, a memory 220 that loads a computer program 260 executed by the processor 210, and a storage 230 that stores the computer program 260. Only the components related to the embodiment of the present disclosure are illustrated in the drawing. However, a person skilled in the art will understand that other general components may be further included in addition to the components illustrated in the drawing.

[0232] The processor 210 controls all operations of respective components of the computing device 200. The processor 210 may be configured to include a CPU (Central Processing Unit), an MPU (Micro Processor Unit), an MCU (Micro Controller Unit), a GPU (Graphics Processing Unit), or at least one of processors of any other type well known in the art of the present disclosure. In addition, the processor 210 may perform operations for at least one application or program to perform methods / operations according to various embodiments of the present disclosure. Also, the computing device 200 may include at least one processor.

[0233] The memory 220 stores various types of data, commands, and / or information. The memory 220 may load at least one program 260 from the storage 230 to perform methods / operations according to various embodiments of the present disclosure. An example of the memory 220 may be RAM, but is not limited thereto.

[0234] The bus 250 provides a communication function between components of the computing device 200. The bus 250 may be implemented by various types of buses, such as an address bus, a data bus, and a control bus.

[0235] The communication interface 240 supports wired and wireless Internet communication of the computing device 200. The communication interface 240 may also support various communication methods other than the Internet communication. To this end, the communication interface 240 may be configured to include a communication module well known in the technical field of the present disclosure.

[0236] The storage 230 may non-temporarily store at least one computer program 260. The storage 230 may be configured to include a non-volatile memory, such as ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), or flash memory, a hard disk, an SSD (Solid State Drive), a removable disk, or any form of a computer-readable recording medium well known in the technical field to which the present disclosure belongs.

[0237] The computer program 260 may include at least one instruction in which methods / operations according to various embodiments of the present disclosure are implemented.

[0238] When the computer program 260 is loaded into the memory 220, the processor 210 may perform methods / operations according to various embodiments of the present disclosure by executing the at least one instruction.

[0239] The technical idea of the present disclosure described above may be implemented as a computer-readable code on a computer-readable medium. A computer-readable recording medium may be, for example, a removable recording medium (a CD, a DVD, a Blu-ray disk, a USB storage device, a removable hard disk) or a fixed recording medium (ROM, RAM, computer-equipped hard disk). The computer program recorded on the computer-readable recording medium may be transmitted to another computing device through a network, such as the Internet, and installed on the other computing device, thereby being used by the other computing device.

[0240] Although embodiments of the present disclosure are described with reference to the attached drawings, those skilled in the art to which the present disclosure belongs will understand that the present disclosure may be implemented in other specific forms without changing the technical idea or essential features thereof. Therefore, it should be understood that the embodiments described above are examples in all respects and not restrictive. The protection scope of the present disclosure should be interpreted by the claims below, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of the technical ideas defined by the present disclosure.

Claims

1.A method for detecting damage of a facility comprising:an A step of displaying, by a user terminal device, a user interface for detecting damage from a facility photo and importing the facility photo; anda B step of inputting, by the user terminal device, a facility photo selected by a user to a machine learning model to detect damage and displaying a detected damage on the user interface.2.A method for detecting damage of a facility comprising:an A step of displaying, by a user terminal device, a user interface for detecting damage from a facility photo and importing the facility photo; anda B step of inputting, by the user terminal device, a facility photo selected by a user to a machine learning model to detect damage and displaying a detected damage on the user interface.3.The method for detecting damage of the facility of claim 1, further comprising:a C step of editing damage information in response to an input of the user, after the step B.4.The method for detecting damage of the facility of claim 1, further comprising:a C step of editing damage information in response to an input of the user, after the step B.5.The method for detecting damage of the facility of claim 4, whereina distance between pixels of a ground reference resolution of the facility photo is changed in response to a photo-taking distance input by the user.6.The method for detecting damage of the facility of claim 3, wherein,in the C step,new damage is generated in response to damage drawn by the user on the user interface.7.The method for detecting damage of the facility of claim 3, wherein,after the B step or the C step,the user terminal device displays the damage in an image file or a CAD file and stores the image file of the CAD file, and stores the damage information in a document file or a spreadsheet file and exports the document file or the spreadsheet file.8.The method for detecting damage of the facility of claim 3, wherein,after the B step or the C step,the user terminal device displays the damage in an image file or a CAD file and stores the image file of the CAD file, and stores the damage information in a document file or a spreadsheet file and exports the document file or the spreadsheet file.9.The system for detecting damage of the facility of claim 8, whereinthe machine learning model includes a first neural network that deletes a background from the facility photo to separate the background from the facility image, and a second neural network that detects damage of a separated facility image.10.The system for detecting damage of the facility of claim 8, whereinthe machine learning model includes a first neural network that deletes a background from the facility photo to separate the background from the facility image, and a second neural network that detects damage of a separated facility image.11.The system for detecting damage of the facility of claim 8, whereinthe user terminal device displays the damage in an image file or a CAD file and stores the image file of the CAD file, and stores the damage information in a document file or a spreadsheet file and exports the document file or the spreadsheet file.

Images