System for automatically inspecting building defects on basis of artificial intelligence and augmented reality

Abstract

The present invention relates to a system for automatically inspecting building defects on the basis of artificial intelligence and augmented reality. More specifically, the present invention relates to a system for automatically inspecting building defects on the basis of artificial intelligence and augmented reality, the system enabling: a resident to easily register defects on the basis of recommendation content by photographing a space through an photographing module of a first user terminal; a constructor to easily identify and process defect information by using a second user terminal; and a construction company to check and manage in real time, through a learning server, automatically-classified defects. To this end, the present invention provides the system for automatically inspecting building defects on the basis of artificial intelligence and augmented reality, comprising: a first user terminal, which can be possessed by a resident and can inspect defects of a building; a second user terminal, which can be possessed by a constructor and can receive and process information about the defects; a third user terminal which can be possessed by a construction company and can receive and manage the information about the defects and information about the defect processing: a learning server capable of learning defect types according to the type of a construction by using image data of building defects; a defect determination server capable of determining the presence / absence of defects by comparatively calculating an image received from the first user terminal and learning data received from the learning server; and a service server capable of transmitting / receiving information to / from the first user terminal, the second user terminal and the third user terminal.

Description

An automated building defect inspection system based on artificial intelligence and augmented reality

[0001] The present invention relates to an automatic building defect inspection system based on artificial intelligence and augmented reality. More specifically, the system allows residents to easily register defects based on recommendations by taking photos of their spaces using a camera module on a first user terminal, a construction company to easily identify and process defect information using a second user terminal, and a construction company to check and manage automatically classified defects in real time through a learning server.

[0002] In general, under the Apartment Management Act, a defect is defined as a defect that causes cracks, subsidence, damage, lifting, or water leakage in a building due to an error in construction that causes a problem in the safety, function, or appearance of the building or facility.

[0003] In Korea, the average number of new apartments completed each year from 2015 to 2021 is 370,000, and the cumulative number over the past five years is 2,471,619.

[0004] As these apartment residents' claims for rights expand, their sensitivity to defects is increasing, and disputes between residents, developers, and construction companies regarding apartment quality are increasing. The number of defect reports has been steadily increasing over the past five years, and the number of defect reports in 2021 was 7,686, an increase of approximately 81% compared to 4,245 in 2020.

[0005] In the case of a building defect, the occupants, lessees, or other management entities of the apartment complex directly inspect the defect with their own eyes during the defect inspection period given to them upon moving in, and notify the project operator or building owner of this to request that the defect be repaired.

[0006] This type of defect inspection is not only difficult to detect architectural defects because the inspectors are mostly ordinary people who do not have specialized knowledge about buildings, but also often leads to controversies and disputes over whether or not a defect is actually present.

[0007] In addition, since defects are processed using paper documents in the process of receiving and processing defects, there are problems such as the repetition of complex tasks for identifying defects, difficulty in confirming the exact contents of defect data, delay in processing defects due to the data conversion process, and data being omitted during the data conversion process.

[0008] In addition, construction projects are carried out organically by the general contractor and subcontractors, which creates difficulties in managing the contractors, and there is a problem that the construction company's brand image is damaged due to lack of communication with subcontractors regarding the reporting of defects and repair instructions.

[0009] To solve these problems, there is a need for a means to easily register defects and easily identify defect information through linkage with construction companies.

[0010]

[0011] Prior art document: KR Patent Publication No. 10-2539543 (published on June 2, 2023)

[0012] The present invention has been devised to solve the above problems, and its purpose is to provide an artificial intelligence and augmented reality-based automatic building defect inspection system that allows even a user of a first user terminal without knowledge of construction to easily identify defects and to read defects by learning using image data.

[0013] In order to achieve the above object, the present invention provides an artificial intelligence and augmented reality-based automatic building defect inspection system, comprising: a first user terminal that can be carried by a resident and can inspect a building for defects; a second user terminal that can be carried by a construction company and can receive and process information about defects; a third user terminal that can be carried by a construction company and can receive and manage information about defects and information about defect processing; a learning server that can learn types of defects according to types of construction using image data about building defects; a defect determination server that can determine whether there is a defect by comparing and calculating images received from the first user terminal with learning data received from the learning server; and a service server that can transmit and receive information to and from the first user terminal, the second user terminal, and the third user terminal.

[0014] In addition, the first user terminal further includes a defect inspection module including a user information input module capable of inputting information on a building to be inspected for defects and user information; a defect inspection reservation module capable of selecting a building to be inspected for defects, selecting a date and time to inspect the defect inspection, and checking the status of advance inspection reservations; a shooting module having a camera section capable of shooting images and videos of a location to be inspected for defects; a shooting information transmission section capable of transmitting information shot by the shooting module to a judgment server through a communication module; a defect suggestion section capable of suggesting whether there is a defect and the type of defect determined by the judgment server; a defect input section capable of inputting a defect suggested or not suggested by the defect suggestion section; and a defect processing inquiry section capable of checking the progress of defect processing.

[0015] In addition, the learning server further includes a defect image database module in which defect images for construction or building are classified and stored by object classification; an image learning module that performs image learning based on the defect images stored in the defect image database module; and a communication module that can transmit learned information to the judgment server.

[0016] According to the present invention, residents can easily register defects based on artificial intelligence and augmented reality through a first user terminal, construction companies can easily identify and process defect information in conjunction with construction companies, construction companies can automatically identify and manage classified defects in real time through an artificial intelligence-based learning server, and have the effect of efficiently managing and processing defects by digitalizing the defect reception and defect processing process at construction sites.

[0017] FIG. 1 is a diagram showing the relationship between all components of an artificial intelligence and augmented reality-based automatic building defect inspection system according to a preferred embodiment of the present invention.

[0018] FIG. 2 is a diagram illustrating components of each user terminal of an artificial intelligence and augmented reality-based automatic building defect inspection system according to a preferred embodiment of the present invention.

[0019] Figure 3 is a diagram illustrating the entire system flow of an artificial intelligence and augmented reality-based automatic building defect inspection system according to a preferred embodiment of the present invention.

[0020] Figure 4 is a drawing showing a work type mapping screen using a work type mapping module of a third user terminal.

[0021] Figure 5 is a drawing showing a screen for selecting a lake for defect inspection in the work inspection module of the third user terminal.

[0022] Figure 6 is a drawing showing a screen for registering a defect in the work inspection module of the third user terminal.

[0023] Figure 7 is a drawing showing a screen for transmitting defect information from the work inspection module of the third user terminal to the service server.

[0024] Figure 8 is a drawing showing a screen for searching for registered defects in the defect processing management module of a third user terminal.

[0025] Figure 9 is a drawing showing a screen for searching detailed information on a defect registered in the defect processing management module of a third user terminal.

[0026] Figure 10 is a drawing showing a screen for searching detailed information on a defect registered in the defect processing management module of a third user terminal.

[0027] Figure 11 is a drawing showing a screen for checking reservation status in the defect inspection reservation management department of a third user terminal.

[0028] Figure 12 is a drawing showing a screen for checking reservation status and making reservations through the defect inspection reservation module of the first user terminal.

[0029] Figure 13 is a drawing showing a screen for completing a reservation and registering a vehicle through the defect inspection reservation module of the first user terminal.

[0030] Figure 14 is a drawing showing a screen for searching reservation information in the defect inspection reservation management module of a third user terminal.

[0031] Figure 15 is a drawing showing a screen for checking reservation status in the defect inspection reservation management module of a third user terminal.

[0032] Figure 16 is a drawing showing a screen for checking reservation status in the defect inspection reservation management module of a third user terminal.

[0033] Figure 17 is a drawing showing a screen displaying the reservation status on a building elevation drawing in the defect inspection reservation management module of the third user terminal.

[0034] Figure 18 is a drawing showing a screen for performing visit processing in the defect inspection reservation management module of the third user terminal.

[0035] Figure 19 is a drawing showing a screen where generation password information is transmitted from the first user terminal.

[0036] Figure 20 is a drawing showing the screens of the defect registration section and the non-defect guidance section in the defect inspection module of the first user terminal.

[0037] Figure 21 is a drawing showing the screen of the defect list section and defect detail inquiry section of the defect inspection module of the first user terminal.

[0038] Figure 22 is a drawing showing a screen in which a defect is suggested by the defect suggestion section on a screen captured through the shooting module of the first user terminal.

[0039] Figure 23 is a flowchart showing the order in which defects are determined through image conversion, image processing, and pattern matching in the defect detection server.

[0040] Figure 24 is a drawing comparing the image converted and processed with the image that was separated from the defect judgment server.

[0041] Figure 25 is a diagram illustrating the process of performing learning on a learning server.

[0042] Figure 26 is a diagram illustrating the process of processing defect information transmitted through the first user terminal.

[0043] Figure 27 is a drawing showing a screen for searching detailed information on a defect in the defect processing management module of the third terminal.

[0044] Figure 28 is a drawing showing the screen of the dashboard module of the third terminal.

[0045] Figure 29 is a drawing showing a screen where the current status is displayed on the elevation of a building.

[0046] Figure 30 is a drawing showing a screen for transmitting work instruction information to a second user terminal through a third user terminal.

[0047] Figure 31 is a drawing showing a screen for creating a defect confirmation certificate through the defect processing management module of a third user terminal.

[0048] Figure 32 is a drawing showing a screen for checking the status of moving in from the moving in management module of the third user terminal.

[0049] Figure 33 is a drawing showing a screen for reserving a move-in date through the move-in reservation module of the first user terminal.

[0050] Figure 34 is a drawing showing a screen for registering defects through the defect inspection module of the first user terminal after moving in.

[0051] Figure 35 is a drawing showing a screen for searching household information through the household information management module of a third user terminal.

[0052] An embodiment of an automatic building defect inspection system based on artificial intelligence and augmented reality according to the present invention includes a first user terminal that can be carried by a resident and can inspect a building for defects; a second user terminal that can be carried by a construction company and can receive and process information about defects; a third user terminal that can be carried by a construction company and can receive and manage information about defects and information about defect processing; a learning server that can learn types of defects according to types of construction using image data about building defects; a defect determination server that can determine whether there is a defect by comparing and calculating images received from the first user terminal with learning data received from the learning server; and a service server that can transmit and receive information to and from the first user terminal, the second user terminal, and the third user terminal.

[0053] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings. First, when assigning reference numerals to components in each drawing, it should be noted that identical components are assigned the same numerals as much as possible even if they are shown in different drawings. Furthermore, in describing the present invention, if a detailed description of a related known structure or function is judged to obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, although preferred embodiments of the present invention will be described below, it should be understood that the technical idea of ​​the present invention is not limited thereto and can be modified and implemented in various ways by those skilled in the art.

[0054]

[0055] FIG. 1 is a diagram showing the relationship between all components of an artificial intelligence and augmented reality-based automatic building defect inspection system according to a preferred embodiment of the present invention, FIG. 2 is a diagram showing components of each user terminal of an artificial intelligence and augmented reality-based automatic building defect inspection system according to a preferred embodiment of the present invention, FIG. 3 is a diagram showing the entire system flow of an artificial intelligence and augmented reality-based automatic building defect inspection system according to a preferred embodiment of the present invention, FIG. 4 is a diagram showing a work type mapping screen using a work type mapping module of a third user terminal, FIG. 5 is a diagram showing a screen for selecting a unit number for defect inspection in a work type inspection module of a third user terminal, FIG. 6 is a diagram showing a screen for registering a defect in a work type inspection module of a third user terminal, FIG. 7 is a diagram showing a screen for transmitting defect information from a work type inspection module of a third user terminal to a service server, FIG. 8 is a diagram showing a screen for searching for registered defects in a defect processing management module of a third user terminal, and FIG. 9 is a diagram showing a screen for searching for registered defects in a defect processing management module of a third user terminal. A drawing showing a screen for searching detailed information of a defect registered in a defect handling management module, FIG. 10 is a drawing showing a screen for searching detailed information of a defect registered in a defect handling management module of a third user terminal, FIG. 11 is a drawing showing a screen for searching reservation status in a defect inspection reservation management unit of a third user terminal, FIG. 12 is a drawing showing a screen for searching reservation status and making a reservation through a defect inspection reservation module of a first user terminal, FIG. 13 is a drawing showing a screen for completing a reservation and registering a vehicle through a defect inspection reservation module of a first user terminal, FIG. 14 is a drawing showing a screen for searching reservation information in a defect inspection reservation management module of a third user terminal, FIG. 15 is a drawing showing a screen for searching reservation status in a defect inspection reservation management module of a third user terminal, FIG. 16 is a drawing showing a screen for searching reservation status in a defect inspection reservation management module of a third user terminal, FIG. 17 is a drawing showing a screen for searching reservation status in a defect inspection reservation management module of a third user terminal A drawing showing a screen displaying the reservation status on the building elevation.FIG. 18 is a diagram showing a screen for performing visit processing in the defect inspection reservation management module of the third user terminal, FIG. 19 is a diagram showing a screen for transmitting household password information in the first user terminal, FIG. 20 is a diagram showing a screen for a defect registration section and a non-defect guidance section in the defect inspection module of the first user terminal, FIG. 21 is a diagram showing a screen for a defect list section and a defect detailed inquiry section in the defect inspection module of the first user terminal, FIG. 22 is a diagram showing a screen for suggesting defects in a defect suggestion section in a screen captured through a shooting module of the first user terminal, FIG. 23 is a flowchart showing a sequence for determining defects through image conversion, image processing, and pattern matching in the defect judgment server, FIG. 24 is a diagram comparing an image captured by the defect judgment server with an image that has been converted and processed, FIG. 25 is a diagram showing a process for performing learning in the learning server, and FIG. 26 is a diagram showing a process for performing learning in the learning server, transmitted through the first user terminal. A drawing showing the process of processing defect information, FIG. 27 is a drawing showing a screen for searching detailed information on a defect in the defect processing management module of the third terminal, FIG. 28 is a drawing showing a screen of the dashboard module of the third terminal, FIG. 29 is a drawing showing a screen for displaying the current status on the elevation drawing of the building, FIG. 30 is a drawing showing a screen for transmitting work instruction information to the second user terminal through the third user terminal, FIG. 31 is a drawing showing a screen for creating a defect confirmation certificate through the defect processing management module of the third user terminal, FIG. 32 is a drawing showing a screen for searching the status of move-in in the move-in management module of the third user terminal, FIG. 33 is a drawing showing a screen for reserving a move-in date through the move-in reservation module of the first user terminal, FIG. 34 is a drawing showing a screen for registering a defect through the defect inspection module of the first user terminal after moving in, and FIG. 35 is a drawing showing a screen for registering a defect through the household information management module of the third user terminal. This is a diagram showing a screen for searching generation information.

[0056]

[0057] Referring to Fig. 1, the artificial intelligence and augmented reality-based automatic building defect inspection system according to a preferred embodiment of the present invention comprises a first user terminal (100), a second user terminal (200), a third user terminal (300), a learning server (400), a defect determination server (500), and a service server (600).

[0058]

[0059] Hereinafter, components of an artificial intelligence and augmented reality-based automatic building defect inspection system according to a preferred embodiment of the present invention will be described in detail.

[0060] Referring to FIG. 2, the first user terminal (100) can be carried by a resident or an inspector who performs a defect inspection and can perform a defect inspection of a building, and is composed of a user information input module (110), a defect inspection reservation module (120), a photographing module (130), a defect inspection module (140), and a move-in reservation module (150).

[0061] The first user terminal (100) can be a personal communication device including a smart phone, tablet PC, or personal computer, and multiple devices can be used, and each of the multiple residents or inspectors performing the defect inspection can carry one.

[0062]

[0063] The user information input module (110) can input information about the user of the first user terminal, and can input information about the building for which a defect inspection is to be performed, and the information input through the user information input module (110) can be transmitted to the service server (600) and stored in the database module (610).

[0064]

[0065] The defect inspection reservation module (110) can select a building for which a defect inspection is to be performed and make a defect inspection reservation, and is comprised of a reservation status inquiry section, a pre-inspection reservation section, and a visit information confirmation section.

[0066] The reservation status inquiry section allows you to check whether a pre-inspection reservation is available for each date and time, and you can register a pre-inspection reservation by selecting an available date and time in the pre-inspection reservation section. If you visit by vehicle for a pre-inspection, you can register and search the vehicle number in the pre-inspection reservation section.

[0067] In the visit information confirmation section, you can check the visit location, visit date and time, and the common entrance password and household password required for pre-inspection.

[0068]

[0069] The shooting module (130) is equipped with a camera section and can capture images and videos of a location where a defect inspection is to be performed, and is comprised of an augmented reality camera section and a general camera section.

[0070] Referring to Fig. 22, in the augmented reality camera section, if the first user terminal has smooth wireless communication, information on defects can be provided in real time on the screen captured and displayed through the shooting module.

[0071] The general camera section can capture images or video information by taking pictures even when the wireless communication of the first user terminal is smooth or not, and the image information or video information obtained can be provided to the defect determination server through the service when the wireless communication of the first user terminal is smooth.

[0072]

[0073] Referring to FIGS. 20 and 21, the defect inspection module (140) can register defects and includes a photographing information transmission section, a defect suggestion section, a defect input section, a defect registration section, a defect list section, a defect processing inquiry section, and a non-defect guidance section.

[0074] The shooting information transmission unit can transmit image information and video information captured through the shooting module to the defect judgment server via the service server through the communication module equipped in the first user terminal.

[0075] The defect suggestion section can be proposed by displaying the presence or absence of a defect and the type of defect determined by the defect judgment server based on the image information and video information transmitted to the defect judgment server.

[0076] In the defect input section, you can directly enter defects suggested in the defect suggestion section or defects that were not suggested.

[0077] In the defect registration section, defects entered in the defect input section can be transmitted to the service server, and the information transmitted to the service server can be stored in the database module and transmitted to a third-party user terminal.

[0078] The defect list section can search for information on defects suggested in the defect suggestion section and information on defects entered in the defect input section, and can search for registered and unregistered defects.

[0079] The defect handling inquiry section allows you to search for information on handling of registered defects, including information on the contractor performing the defect repair, defect repair processing information, and images and videos after the defect repair.

[0080] The Non-Defective Information Department can provide guidance on cases where something appears abnormal to the naked eye but is not a defect.

[0081] Since most users of the first user terminal do not have specialized knowledge of construction, the defect inspection module is equipped with a non-defect guidance section that can provide guidance on non-defective situations, so that users can receive guidance on non-defective situations.

[0082] For example, if the pattern of the vanity top is different, you can inform them that the stone used for the vanity top is processed natural stone, not artificial stone, and that the natural pattern of natural stone may cause differences in the pattern and color depending on the generation. Or, if the water from the bathroom cleaning spray does not come out, you can inform them of situations that are not defects and suggest solutions, such as "Please check first that the faucet located at the bottom behind the toilet is open" before registering a defect.

[0083]

[0084] Referring to Figure 33, the move-in reservation module (150) can directly reserve the move-in date and time through the first user terminal to ensure orderly and smooth move-in.

[0085] Information on move-in reservations entered through the move-in reservation module is transmitted to the service server and stored in the database module, and can be transmitted to a third-party user terminal and viewed through the move-in management module.

[0086]

[0087] Referring to FIG. 2, the second user terminal (200) can be possessed by the construction company, can receive and process information on defects, and is composed of a user information input module (210), a defect confirmation module (220), and a defect processing module (230).

[0088]

[0089] The construction company that possesses the second user terminal refers to the construction company's partner company or subcontractor, which is a company for each type of construction.

[0090]

[0091] The second user terminal may be a personal communication device including a smartphone, tablet PC, or personal computer, and multiple devices may be used, and multiple construction companies may each have a second user terminal.

[0092]

[0093] The user information input module (210) can input information of a construction company using a second user terminal (200), and information input through the user information input module (210) of the second user terminal (200) can be transmitted to the service server (600) and stored in the database module.

[0094] The defect confirmation module (220) can check information on registered defects through the defect inspection module (140) of the first user terminal (100) or the work inspection module (350) of the third user terminal (300).

[0095] The defect processing module (230) can input information on defect processing and includes a processing waiting section, a transmission waiting section, a transmission completed section, and a processing unavailable section.

[0096] In the processing waiting list, you can view a list of defects that have not yet been repaired or processed. If you select a specific defect from the list, you can view detailed information about the defect, including the number of members, defect location, type of work, photos and videos, and defect details.

[0097] In addition, you can select a specific defect from the list in the processing waiting list and save information about the defect, including the processing details for the defect, photos and videos after the defect repair, and the defect list with information about the defect processing saved is deleted from the processing waiting list and moved to the transmission waiting list.

[0098] The transmission waiting section can view a list of defects for which information on defect processing has been entered but not yet transmitted to the service server, and can select a specific defect from the list to enter additional information on defect processing, and can transmit defects for which processing has been completed to the service server.

[0099] In the transmission completion section, you can view a list of defects that have been processed, and by selecting a specific defect, you can view detailed information about the defect and detailed information about the processing.

[0100] In the case where a defect in the waiting list for processing is one that cannot be processed by the construction company that owns the second user terminal, or a defect that cannot be processed by the construction company, including a matter that is not a defect, the processing unit can select the defect as unprocessable in the waiting list for processing and move it to the unprocessable list.

[0101]

[0102] Referring to FIG. 2, a third user terminal (300) can be possessed by a construction company, and can receive and manage information on defects and information on defect handling, and is composed of a work type mapping module (310), a defect inspection reservation management module (320), a defect inspection status inquiry module (330), a defect handling management module (340), a work type inspection module (350), and a move-in management module (360).

[0103]

[0104] Referring to FIG. 4, the work type mapping module (310) can match construction companies according to work type, type, building, line, and space.

[0105] In the public works mapping module (310), you can select a building, line, or lake, and after selecting a building, line, or lake, you can select a space including a living room, kitchen, or bedroom.

[0106] After selecting a space, you can select the types of work, including mop holders, interior decoration, painting, flooring, and fire protection equipment for that space, and you can match the construction company by entering or selecting information about the construction company for that type of work.

[0107] For example, the living room floor of room 1401 in building 101 can be mapped to a construction company called Nemo Maru, and the painting and mop holder can be mapped to a circle design, allowing construction companies to be mapped by type of work for each space.

[0108]

[0109] Referring to Fig. 11, the defect inspection reservation management module (320) can check and manage the status of defect inspection reservations in the defect inspection reservation module (120) of the first user terminal (100), and includes a visit status inquiry section and a visit processing section.

[0110] Referring to FIGS. 16 and 17, the visit status inquiry section transmits information on a defect inspection reservation entered through the defect inspection reservation module of the first user terminal to the service server, and the information on the defect inspection reservation transmitted to the service server is stored in the database section, and the visit status of each household, the number of visited, revisited, and unvisited households for the entire site are automatically tallied by the operation module of the service server, and the tally result is transmitted to the third user terminal and can be queried through the visit status inquiry section of the third user terminal.

[0111] Referring to FIGS. 18 and 19, the visit processing unit can check the household entrance password of the household for which a visit reservation has been registered, and when processing a household visit, the household password and common entrance password can be transmitted to the first user terminal.

[0112] The common entrance password and household password transmitted by the visitor processing department can be confirmed through the visitor information confirmation department of the first user terminal.

[0113]

[0114] The defect inspection status inquiry module (330) can inquire about information on registered defects through the defect inspection module (140) of the first user terminal (100).

[0115] In the defect inspection status inquiry module (330), a list of defect information including defect reception number, building, unit, balance, real name, reception date, change date, defect details, inspector, work type, defect type, construction company, and progress status can be searched for for each defect registered through the first user terminal (100).

[0116] When a specific defect is selected from the defect information list of the defect inspection status inquiry module (330), detailed information about the defect, including the image of the defect registered by the inspector, the content of the defect written by the inspector, the content of the administrator's memo, and the modification history, can be searched, and the modification history and other defects can be searched to enable management of the defect history of the relevant generation.

[0117]

[0118] The defect processing management module (340) can manage information on registered defects and includes a work instruction section and a document processing section.

[0119] The work instruction section can select a company to instruct work to handle a registered defect through the first user terminal and transmit information about the defect.

[0120] Referring to Figure 30, when a construction company is selected through a work order section, the corresponding information is transmitted to a service server and stored in a database section, and information on defects and work order information are transmitted to the selected second user terminal, and the corresponding information can be confirmed through the defect confirmation module of the second user terminal.

[0121]

[0122] Referring to Fig. 31, the document processing unit can generate and print documents for defect repairs for defects entered through the defect inspection module of the first user terminal. To facilitate the preparation of completion documents for registered defects through the document processing unit, the printing of a defect confirmation certificate is supported, and in cases where a signature of a resident, including a major defect, is required, a push notification can be sent to the first user terminal to obtain a mobile signature, and conversion to a Ministry of Land, Infrastructure and Transport code can be supported to facilitate document work.

[0123]

[0124] A work inspection can be performed through the work inspection module (350), and is comprised of a lake selection section, a defect registration section, and an offline support section.

[0125] You can select the lake to be inspected in the lake selection section and register the inspection results in the defect registration section.

[0126] In the defect registration book, you can select a space including the living room, kitchen, and bedroom, select the location of the defect including the wall, ceiling, and floor in the space, select the type of work including painting, interior decoration, and fire protection equipment, enter the type of defect, and enter photos and videos of the defect and details about the defect.

[0127] Since wireless communication may not be smooth at construction sites, if wireless communication is smooth, the defect registration information can be immediately transmitted to the service server. If wireless communication is not smooth, the defect registration information can be transmitted to the offline mode support department and then immediately transmitted to the service server when wireless communication is possible.

[0128] When a defect is registered in the work inspection module (350) and transmitted to the service server (600), the service server stores information about the defect in the database and transmits it to the second user terminal.

[0129] Defect information transmitted to the second user terminal (200) can be checked through the defect confirmation module (220) of the second user terminal (200) and is entered into the processing waiting section of the defect processing module (230).

[0130]

[0131] The move-in management module (360) can check the move-in reservation information entered through the move-in reservation module (150) of the first user terminal (100) and check the reservation status.

[0132]

[0133] In the household information management module, you can search for household information including resident name and contact information, facility handover date, balance payment date, key issuance date, move-in date, resident information including residence type, registered defect information, and consultation history.

[0134] You can check schedules such as key issuance date or move-in date, manage meter readings for electricity, water, gas, etc., check resident details at a glance, and manage consultation history.

[0135]

[0136] The learning server (400) can learn the types of defects according to the type of construction using image data on building defects, and is comprised of a defect image database module, an image learning module, and a communication module.

[0137] The defect image database module can store defect images for construction or construction work classified by object category, and defects registered through the defect inspection module of the first user terminal can also be stored in the defect image database module and used for image learning about defects.

[0138] Referring to Figure 25, the image learning module can perform image learning based on the defect image stored in the defect image database module.

[0139] The communication module transmits the learned information to the defect judgment server, and compares the information transmitted from the learning server with the information transmitted from the first user terminal to determine whether there is a defect.

[0140]

[0141] The defect determination server (500) can determine whether there is a defect by comparing and calculating the image or video information captured by the shooting module (130) of the first user terminal (100) with the learning data transmitted from the learning server, and is composed of an image conversion unit, an image processing unit, and a pattern analysis unit.

[0142] Referring to Fig. 23, the image conversion unit converts the image or video information captured by the shooting module of the first user terminal into 4-bit image information. The converted image undergoes image processing to reverse the image and change the image level after primary and secondary noise removal in the image processing unit, and the image information that has undergone the image processing is analyzed by the pattern analysis unit to determine whether there is a defect by analyzing a pattern that deviates by more than a certain % set.

[0143] The determined defect information is transmitted to the service server and stored in the database, and is displayed on the first user terminal through the defect suggestion section of the first user terminal defect inspection module.

[0144]

[0145] The service server (600) can transmit and receive information with a first user terminal, a second user terminal, and a third user terminal, and is comprised of a database module, a construction company matching module, and an operation module.

[0146]

[0147] The database module can store information transmitted from the first user terminal, the second user terminal, and the third user terminal in a database.

[0148] The construction company matching module can match each second user terminal with each work type based on the construction company information for each work type entered in the work type mapping module of the third user terminal, based on the user information of the second user terminal entered in the user information input module of the second user terminal.

[0149] The operation module performs statistical operations using information received from the first user terminal, the second user terminal, and the third user terminal, and enables the initialization of statistical information through components that can inquire about information, including the visit status inquiry section of the third user terminal, the defect inspection status inquiry module, and the dashboard module.

[0150]

[0151] Below, we describe in detail the operation method of the artificial intelligence and augmented reality-based automatic building defect inspection system.

[0152]

[0153] The present invention comprises a first user terminal that can be carried by a defect inspector, a second user terminal that can be carried by a construction company, a third user terminal that can be carried by a construction company, a learning server that can learn about building defects, a defect judgment server that can compare and calculate learning data to determine whether there is a defect, and a service server.

[0154]

[0155] Typically, B / S, a construction company inspection, is conducted 60 days before moving in after the start of construction, O / H, a pre-inspection by the prospective occupants, is conducted 30 days before moving in, and A / S, a post-move-in repair service, is conducted after moving in.

[0156]

[0157] The construction company can match construction companies by type, building, line, and space using the work type mapping module of the third-party user terminal, and construction company matching according to the system of the present invention is typically carried out 180 days prior to occupancy.

[0158] The work type mapping module allows you to select a building, line, or unit. After selecting a building, line, or unit, you can then select a space, including the living room, kitchen, and bedroom. After selecting a space, you can select work types for that space, such as mopping, interior design, painting, flooring, and fire protection equipment. You can then match the work types with construction companies.

[0159] For example, the living room floor of room 1401 in building 101 can be mapped to a construction company called Nemo Maru, and the painting and mop holder can be mapped to a circle design, allowing construction companies to be mapped by type of work for each space.

[0160]

[0161] A construction company inspection will be conducted 60 days prior to moving in, and the inspection will be conducted through the inspection module of the third-party user terminal.

[0162]

[0163] Users of the third-party user terminal can select the unit to be inspected in the unit selection section of the work inspection module and register the inspection results in the defect registration section.

[0164] In the defect registration book, you can select a space including the living room, kitchen, and bedroom, select the location of the defect including the wall, ceiling, and floor in the space, select the type of work including painting, interior decoration, and fire protection equipment, enter the type of defect, and enter photos and videos of the defect and details about the defect.

[0165] Since wireless communication may not be smooth at construction sites, if wireless communication is smooth, the defect registration information can be immediately transmitted to the service server. If wireless communication is not smooth, the defect registration information can be transmitted to the offline mode support department and then immediately transmitted to the service server when wireless communication is possible.

[0166] When a defect is registered in the work inspection module and transmitted to the service server, the service server stores information about the defect in the database and transmits it to the second user terminal.

[0167]

[0168] Defect information transmitted to the second user terminal can be checked through the defect confirmation module of the second user terminal and is entered into the processing waiting section of the defect processing module.

[0169] The user of the second user terminal checks the defect details, and after the defect repair is completed, the defect handling module registers the defect handling results along with photos and videos. The information on the registered defect handling results can be immediately transmitted to the service server if wireless communication is smooth, and if wireless communication is not smooth, the information on the defect handling results is registered in the transmission waiting section and is immediately transmitted to the service server when wireless communication is possible.

[0170] When information on the defect processing result is transmitted from the defect processing module to the service server, the service server stores the information on the defect processing result in the database and transmits it to a third-party user terminal, and the third-party user terminal can retrieve the information on the defect processing result.

[0171]

[0172] Normally, the prospective occupant conducts a pre-inspection around 30 days before moving in, but the user of the first user terminal can make a pre-inspection reservation using the defect inspection reservation module of the first user terminal 45 days before moving in.

[0173]

[0174] Users of the first user terminal can check whether a pre-inspection reservation is available on a specific date and time through the reservation status inquiry section, and can register a pre-inspection reservation by selecting an available date and time in the pre-inspection reservation section.

[0175] Additionally, when visiting by vehicle during a pre-inspection, you can register and check the vehicle number in the pre-inspection reservation department.

[0176] Information about a defect inspection reservation entered in the defect inspection reservation module of the first user terminal is transmitted to the service server, and information about a defect inspection reservation transmitted to the service server is stored in the database and transmitted to the third user terminal.

[0177] When information on a defect inspection reservation is transmitted from the service server to a third-party user terminal, the visit status of each household, the number of visited, revisited, and unvisited households for the entire site are automatically tallied and the tally results can be checked through the visit status inquiry section of the defect inspection reservation management module of the third-party user terminal.

[0178] In addition, when a defect inspection reservation is entered, the visit processing section of the defect inspection reservation management module can check the entrance password of the corresponding household, and when the household visit is processed, the household password and common entrance password can be transmitted to the first user terminal.

[0179] You can check the common entrance password and household password in the visitor information confirmation section of the first user terminal.

[0180] The user of the first user terminal can perform a defect inspection using the defect inspection module of the first user terminal on the defect inspection date.

[0181] Since most inspectors who check for defects through the first user terminal are people who do not have specialized knowledge of construction, through the defect inspection module of the first user terminal, even if they do not have specialized knowledge or special experience, if they take a picture of the relevant part through the photography module, the defect suggestion section can suggest whether there is a defect or the type of defect, and the user of the first user terminal can register the suggested defect through the defect input section, as well as register defects that were not suggested.

[0182] Since most users of the first user terminal do not have specialized knowledge of construction, the defect inspection module is equipped with a non-defect guidance section that provides guidance on non-defective situations, so that users can receive guidance on non-defective situations.

[0183] For example, if the pattern of the vanity top is different, you can inform them that the stone used for the vanity top is processed natural stone, not artificial stone, and that the natural pattern of natural stone may cause differences in the pattern and color depending on the generation. Or, if the water from the bathroom cleaning spray does not come out, you can inform them of situations that are not defects and suggest solutions, such as "Please check first that the faucet located at the bottom behind the toilet is open" before registering a defect.

[0184]

[0185] When a user of the first user terminal takes a photo or video using the camera module, the captured photo or video is transmitted to the service server, and the service server transmits the photo or video to the defect assessment server.

[0186] The photo or video transmitted to the defect judgment server is converted into 4-bit image information, and the image is processed by removing primary and secondary noise, inverting the image, and changing the image level. The image information that has gone through the image processing is analyzed for patterns that deviate by more than a set specific percentage to determine whether there are any defects.

[0187] Since grayscale images are easier to distinguish between cracks and other differences than general color images, the image is converted to a 4-bit grayscale image through an image processing process and then the pattern is analyzed.

[0188] The defect assessment server determines a part that deviates from the range set in a certain pattern by a certain percentage or more as a defect and transmits it to the first user terminal, and displays the determined defect information through the defect suggestion section of the defect inspection module.

[0189] The user of the first user terminal can register the defect information suggested through the defect suggestion section as is, as well as register defect information other than the suggested defect.

[0190] Defect information entered through the first user terminal is transmitted to the service server, and the defect information transmitted to the service server is stored in a database and can also be transmitted to the learning server to be used as learning data.

[0191] That is, when a user of the first user terminal simply takes a picture of the space through the shooting module, the defects in the space are automatically determined, and the presence or absence of a defect and the type of defect are suggested by the defect suggestion unit. Through the defect input unit, not only the suggested defects but also the non-suggested defects can be registered.

[0192] For example, even if the defect suggestion section suggests a defect called ‘paint lifting’ based on information captured through the shooting module, the user can register a defect called ‘paint contamination’ instead of registering the suggested defect.

[0193] When a defect is registered, information about the defect is transmitted to the learning server via the service server, and the learning server can relearn the image information and information about the defect.

[0194] When a defect is registered through the defect inspection module defect input section of the first user terminal, information about the defect is transmitted to the service server and stored in the database section, and can also be transmitted to a third user terminal.

[0195] Information on defects transmitted to a third-party user terminal can be viewed through the defect inspection status inquiry module.

[0196] In the defect inspection status inquiry module of the third user terminal, you can search for a list of defect information including the defect reception number, building, unit, level, real name, reception date, change date, defect details, inspector, work type, defect type, construction company, and progress status for each defect registered through the first user terminal.

[0197] When you select a specific defect from the defect information list in the defect inspection status inquiry module, you can view detailed information about the defect, including the defect image registered by the inspector, the defect content written by the inspector, the administrator's memo, and the modification history.

[0198]

[0199] The above description is merely an illustrative description of the technical idea of ​​the present invention, and those skilled in the art will appreciate that various modifications, changes, and substitutions may be made without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention and the accompanying drawings are not intended to limit the technical idea of ​​the present invention, but rather to explain it, and the scope of the technical idea of ​​the present invention is not limited by these embodiments and the accompanying drawings. The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of the rights of the present invention.

Claims

1. A first user terminal that residents can possess and use to inspect the building for defects; A second user terminal that can be possessed by the construction company and can receive and process information about defects; A third-party user terminal that can be possessed by a construction company and can receive and manage information on defects and information on defect handling; A learning server that can learn the types of defects according to the type of construction using image data on building defects; A defect determination server that can determine whether there is a defect by comparing the image transmitted from the first user terminal with the learning data transmitted from the learning server; A service server capable of transmitting and receiving information with the first user terminal, the second user terminal, and the third user terminal. An automatic building defect inspection system based on artificial intelligence and augmented reality, including:

2. In paragraph 1, The first user terminal is A user information input module that can input information on the building to be inspected and information on the user; A defect inspection reservation module that allows you to select a building for which a defect inspection will be conducted, select the date and time for the defect inspection, and check the status of advance inspection reservations; A shooting module equipped with a camera section that can capture images and videos of the location where a defect inspection is to be performed; A shooting information transmission unit that enables the information captured through the shooting module to be transmitted to the judgment server through the communication module. A defect suggestion section that can suggest whether there is a defect and the type of defect determined by the defect judgment server. A defect input section where defects suggested or not suggested in the defect suggestion section can be entered. A defect processing inquiry section where you can check the progress of defect processing. Defect inspection module including; An automatic building defect inspection system based on artificial intelligence and augmented reality, which further includes:

3. In paragraph 2, The learning server is A defect image database module in which defect images for construction or construction are classified and stored by object classification; An image learning module that performs image learning based on the defect images stored in the defect image database module; A communication module that can transmit learned information to the defect judgment server. An automatic building defect inspection system based on artificial intelligence and augmented reality, which further includes:

Images